CS 349: Data Mining, Search, and the World Wide Web
http://www-db.stanford.edu/~sergey/cs349.html
Tuesdays and Thursdays 4:15 - 5:30 in Bldg 370, Room 370 on the Main Quad
Instructors: Sergey Brin and Lawrence Page
Tues and Thurs 5:30 - 7:00 or by
appointment.
sergey@cs.stanford.edu and page@cs.stanford.edu
Course Assistant: Diane Tang
Gates 416: Mon - Wed 11:15 - 12:15 or by appointment.
dtang@cs.stanford.edu
Description
Over the past two years there has been a close collaboration between the Data Mining
Group (MIDAS) and the Digital Libraries Group at Stanford in the area of Web research.
It has culminated in the WebBase project whose aims are to maintain a local copy
of the World Wide Web (or at least a substantial portion thereof) and to use it
as a research tool for information retrieval, data mining, and other applications.
This has led to the development of the PageRank algorithm, the Google search engine,
the DIPRE algorithm, and a number of other works which represent the cutting edge
of research on the Web today (see WebBase Publications).
The topics of this class are data mining and information retrieval in the context
of the World Wide Web. First, we will cover background material in data mining and
information retrieval that is relevant to the class. Second, we will cover recent
advances made at Stanford (PageRank, DIPRE,...) and elsewhere (Kleinberg, Mitchell,...).
Third and most important students will get the opportunity to work hands on with
the WebBase as this will be a project class. We have already modularized a large
part of the code to give people the opportunity to work with it and will continue
to do so throughout the summer. Several people have already taken advantage of the
code. The current WebBase repository consists of roughly 25 million web pages amounting
to 150 GB of HTML.
Prerequisites
- A strong knowledge of C.
- Working knowledge of C++.
- Very basic statistics, graph theory and linear algebra.
Very Tentative Syllabus
- Introduction: 1
- 9/24 Introduction:
- 9/29 WebBase 1 (slides [ http:/www-db.stanford.edu/~sergey/349/cs349-1.ppt
] )
The Anatomy of a Large-Scale Hypertextual Web Search Engine [ http:/google.stanford.edu/~backrub/google.html
]
- Data Mining: 5
Publications of IBM's QUEST project [ http:/www.almaden.ibm.com/cs/quest/publications.html
]
- 10/1 Market Basket (slides [ http:/www-db.stanford.edu/~sergey/349/cs349-2.ppt
] )
R. Agrawal, T. Imielinski, A. Swami: ``Mining Associations between
Sets of Items in Massive Databases'' [ http:/www.almaden.ibm.com/cs/quest/papers/sigmod93.ps
], Proc. of the ACM SIGMOD Int'l Conference on Management of Data,
Washington D.C., May 1993, 207-216. PDF format [ http:/www.almaden.ibm.com/cs/quest/papers/sigmod93.pdf
]. Abstract [ http:/www.almaden.ibm.com/cs/quest/abstracts.html#ais93b ].
Dynamic Itemset Counting and Implication Rules for Market Basket Data [
http:/www-db.stanford.edu/~sergey/dic.html ]
by Sergey Brin. Rajeev Motwani,
Jeffrey D. Ullman and Shalom Tsur.
We present and algorithm for counting
large itemsets faster than previous algorithms. We rely on partial results
to guide the mining process.
Proceedings of the ACM SIGMOD International
Conference on Management of Data, pp. 255-264, Tuscon, Arizona, May 13-15
1997. (html [ http:/www-db.stanford.edu/~sergey/dic.html ], postscript [
http:/www-db.stanford.edu/~sergey/dic.ps ], gzipped ps [ http:/www-db.stanford.edu/~sergey/dic.ps.gz
], bibtex [ http:/www-db.stanford.edu/~sergey/dic.bib ])
- 10/6 Causality
Scalable Techniques for Mining Causal Structures
by C. Silverstein, S. Brin, R. Motwani, and J. Ullman. VLDB '98.
Abstract
[ http:/www-cs-students/~csilvers/papers/causality-vldb-abstract.txt ] ~
Postscript [ http:/www-cs-students/~csilvers/papers/causality-vldb.ps ]
- 10/8 WebBase 2
- 10/13 Classification and Singular Value Decomposition (slides - html
[ http:/www-db.stanford.edu/~sergey/349/classific/ ] postscript [ http:/www-db.stanford.edu/~sergey/349/classific.ps
])
SGI's MLC++ Library [ http:/www.sgi.com/Technology/mlc/ ]
- 10/15 Clustering Techniques (slides - html [ http:/www-db.stanford.edu/~sergey/349/cluster/
] postscript [ http:/www-db.stanford.edu/~sergey/349/cluster.ps ])
Berkeley
Clustering Demo [ http:/riot.ieor.berkeley.edu/riot/Applications/Clustering/
]
- *** Project Proposals Due ***
- 10/20 Data Mining in the Real World
- Search: 3
- 10/22 Standard IR
- 10/27 New Technologies
- 10/29 Latent Semantic Indexing
Bellcore's LSI site [ http:/superbook.bellcore.com/~std/lsi.html
]
- 11/3 WebBase 3
- *** Milestone Due ***
- Web: 6
- 11/5 Search Engines 1 - basics, size, evaluation
- 11/10 Search Engines 2 - crawling, robots.txt, ...
- 11/12 PageRank, Kleinberg
- 11/17 DIPRE
- 11/19 DEC Research
- 11/24 Classification of Web Pages
- *** Final Project Due ***
Mailing List
http:/www.egroups.com/list/cs349/
Sergey Brin [ sergey@cs.Stanford.EDU ]
Last modified: Sat Oct 24 23:18:37 PDT 1998