CSI5387 Concept Learning Systems/Machine Learning

Instructor

Nathalie Japkowicz
Office: STE 5-029
Phone: 562-5800 ext. 6693
E-mail: nat@site.uottawa.ca

Meeting Times and Locations

Time: Mondays 1:00pm-4pm
Location: CBY B-202

Office Hours and Locations

Times:
Mondays 4:15pm-5:15pm
Thursdays 11:45am-12:45pm
Location: STE 5-029;

Overview

Machine Learning is the area of Artificial Intelligence concerned with the problem of building computer programs that automatically improve with experience. The intent of this course is to present a broad introduction to the principles and paradigms underlying machine learning, including presentations of its main approaches, discussions of its major theoretical issues, and overviews of its most important research themes.

Course Format

The course will consist of a mixture of regular lectures and student presentations. The regular lectures will cover descriptions and discussions of the major approaches to Machine Learning as well as of its major theoretical issues. The student presentations will focus on the most important themes we survey. These themes will mostly be approached through recent research articles from the Machine Learning literature.

Evaluation

Students will be evaluated on short written commentaries and oral presentations of research papers (20%), on a few homework assignments (30%), and on a final class project of the student's choice (50%). For the class project, students can propose their own topic or choose from a list of suggested topics which will be made available at the beginning of the term. Project proposals will be due in mid-semester. Group discussions are highly encouraged for the research paper commentaries and students will be allowed to submit their reviews in teams of 3 or 4. However, homework and projects must be submitted individually.

Pre-Requisites

Students should have reasonable exposure to Artificial Intelligence and some programming experience in a high level language.

Required Textbooks

Ian Witten and Eibe Frank, Data Mining: Practical Machine Learning Tools and Techniques, 2nd Edition, Morgan Kaufmann, ISBN 0120884070, 2005.
Introduction to Machine Learning, Nils J. Nilsson (Draft of a Proposed Textbook available on the Web; Available for free)
Nathalie Japkowicz and Mohak Shah, Performance Evaluation for Classification A Machine Learning and Data Mining Perspective (in progress): “Chapter 6: Statistical Significance Testing”.

Additional References .

Tom Mitchell, Machine Learning, 1997 [The class notes are based on it.]
Michael Berry & Gordon Linoff, Mastering Data Mining, John Wiley & Sons, 2000.
Patricia Cerrito, Introduction to Data Mining Using SAS Enterprise Miner, ISBN: 978-1-59047-829-5, SAS Press, 2006.
K. Cios, W. Pedrycz, R. Swiniarski, L. Kurgan, Data Mining: A Knowledge Discovery Approach, Springer, ISBN: 978-0-387-33333-5, 2007.
Margaret Dunham, Data Mining Introductory and Advanced Topics, ISBN: 0130888923, Prentice Hall, 2003.
U. Fayyad, G. Piatetsky-Shapiro, P. Smyth, R. Uthurusamy, editors, Advances in Knowledge Discovery and Data Mining, AAAI/MIT Press, 1996 (order on-line from Amazon.com or from MIT Press).
Jiawei Han, Micheline Kamber, Data Mining : Concepts and Techniques, 2nd edition, Morgan Kaufmann, ISBN 1558609016, 2006.
David J. Hand, Heikki Mannila and Padhraic Smyth, Principles of Data Mining , MIT Press, Fall 2000
Trevor Hastie, Robert Tibshirani, Jerome Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer Verlag, 2001.
Mehmed Kantardzic, Data Mining: Concepts, Models, Methods, and Algorithms, ISBN: 0471228524, Wiley-IEEE Press, 2002.
Daniel T. Larose, Discovering Knowledge in Data: An Introduction to Data Mining, ISBN: 0471666572, John Wiley, 2004 (see also companion site for Larose book).
Glenn J. Myatt, Making Sense of Data: A Practical Guide to Exploratory Data Analysis and Data Mining, John Wiley, ISBN: 0-470-07471-X, November 2006.
Olivia Parr Rud, Data Mining Cookbook, modeling data for marketing, risk, and CRM. Wiley, 2001.
Pang-Ning Tan, Michael Steinbach, Vipin Kumar, Introduction to Data Mining, Pearson Addison Wesley (May, 2005).
Hardcover: 769 pages. ISBN: 0321321367
Sholom M. Weiss and Nitin Indurkhya, Predictive Data Mining: A Practical Guide, Morgan Kaufmann, 1997
Graham Williams, Data Mining Desktop Survival Guide, on-line book (PDF).
Ian Witten and Eibe Frank, Data Mining, Practical Machine Learning Tools and Techniques with Java Implementations, Morgan Kaufman, ISBN 1558605525, 1999.
Ian Witten and Eibe Frank, Data Mining: Practical Machine Learning Tools and Techniques, 2nd Edition, Morgan Kaufmann, ISBN 0120884070, 2005.

List of Major Approaches Surveyed

Version Spaces
Decision Trees
Artificial Neural Networks
Bayesian Learning
Instance-Based Learning
Support Vector Machines
Meta-Learning Algorithms
Rule Learning/Inductive Logic Programming
Unsupervised Learning/Clustering
Genetic Algorithms

List of Theoretical Issues Considered

Experimental Evaluation of Learning Algorithms
Computational Learning Theory

List of Major Themes Surveyed

To expose you to current topics of interest in the data mining and machine learning community, I have chosen to study the topics listed in the following poll of Dec 1-5, 2008.

KDnuggets : Polls : Important Data Mining Topics

Which of the following Data Mining (DM) topics are most important for your work or research? (Choose top 3) [113 voters]
Scaling up DM algorithms for huge data (46)	41%
Mining text (33)	29%
Automating data cleaning (30)	27%
Dealing with unbalanced and cost-sensitive data (29)	26%
Mining data streams (20)	18%
Mining links and networks (19)	17%
Unified theory of DM (18)	16%
DM for biological problems (16)	14%
DM with privacy (10)	8.9%
Mining images (8)	7.1%
DM for security applications (6)	5.4%
Distributed (multi-agent) DM (4)	3.6%
Other (21)	8%