CSI5388: Topics in Machine Learning

Instructor

Nathalie Japkowicz

Office: STE 5029
E-mail: nat@site.uottawa.ca
Telephone: 562-5800 ext. 6693 (Note: e-mail is more reliable)

Meeting Times and Locations

Time:
Monday, 4:00pm-5:30pm;
Wednesday, 2:00pm-3:30pm
Location: MCD 121

Office Hours and Locations

Times: Monday, 2:45pm-3:45pm and Wednesday 1:00pm-2:00pm, or by appointment
Location: STE 5029

Calendar Description

Machine Learning or data Mining are the areas of Artificial Intelligence concerned with the problem of building computer programs that automatically improve with experience. This course will focus on advanced issues from these fields. Issues such as Feature Selection, Class Imbalances, Cost-sensitivity, One-class learning, Classifier Combination, Performance Evaluation and Visualization (and other topics) will be discussed in depth. Students will be expected to read and criticize articles from the recent literature, complete practical assignments, and proposeand complete a research project.

Pre-requesites

csi5387, although the two courses can be taken at the same time with permission of the instructor.

Overview

Machine Learning is the area of Artificial Intelligence concerned with the problem of building computer programs that automatically improve with experience. This course will cover, in depth, some advanced topics in the field.

The course will consist of a mixture of regular lectures and student presentations. The regular lectures will cover broad introductions to some of the major areas of research currently under investigation. The student presentations will be based on recent research papers that describe new results in these areas.

Students will be evaluated on short written commentaries of research papers (20%), on oral presentations of research papers (20%), and on a final class project of the student's choice (60%). For the class project, students can propose their own topic or choose from a list of suggested topics which will be made available at the begining of the term. Project proposals will be due in mid-semester. Group discussions are highly encouraged for the research paper commentaries and students will be allowed to submit their reviews in teams of 3 or 4. However, projects must be submitted individually.

Topics Covered

Week 1: Review of Machine Learning's main concepts
Week 2: Topics in the Evaluation of Machine Learning algorithms (ROC Curves, Cost Curves) based on: ROC Graphs: Notes and Practical Considerations for Data Mining Researchers, by Tom Fawcett, Submitted to Knowledge Discovery and Data Mining, 2003.
Readings:
- Explicitly Representing Expected Cost: An Alternative to ROC Representation
- Have we overestimated the value of ROC analysis under varying class distributions? , Webb and Ming, 2003
Week 3: Intro to Genetic Algorithms (Fitness Functions, Genetic Operators, Applications)
Readings:
- Timeweaver: a Genetic Algorithm for Identifying Predictive Patterns in Sequences of Events, Gary M. Weiss, 1999
- A Survey of Evolutionary Algorithms for Data Mining and Knowledge Discovery (2001) , Alex A. Freitas.
Week 4: Intro to Unsupervised Learning
Readings:
- Unsupervised Learning with Permuted Data, Kirshner, Parise & Smyth, 2003.
- Automatic Alignment of Local Representations, Yee Whye Teh, Sam Roweis, 2003
Week 5: Combining Supervised and Unsupervised Learning, Intro to Radial-Basis Functions.
Readings:
- Learning with labeled and unlabeled data, Matthias Seeger, 2001.
- Understanding the Behavior of Co-training, Kamal Nigam, Rayid Ghani, 2000.
Week 6: Advice on how to formulate a project proposal, presentation by a librarian on ressources available for ML researcher.
Week 7: Topics in Feature Selection (Wrapper, Filter and Hybrid Methods)
Readings:
- Online Feature Selection using Grafting, Perkins & Theiler, 2003.
- Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution, Yu & Liu, 2003.
Week 8: BREAK
Week 9: Single Class Learning Methods (Autoassociators, single-class SVMs)
Reading:
- Supervised versus Unsupervised Binary-Learning by Feedforward Neural Networks , Japkowicz, N., Machine Learning Volume 42, Issue 1/2, pp. 97-122, January 2001.
- One-Class SVMs for Document Classification Larry M. Manevitz, Malik Yousef; 2(Dec):139-154, 2001.
- Combining One class classifiers, David M. J. Tax and Robert P. W. Duin, Lecture Notes in Computer Science, 2096, 2001
Week 10: The Class Imbalance Problem
Readings:
- Class-Boundary Alignment for Imbalanced Dataset Learning, Wu, G. and Chang, E., ICML-2003 Workshop on Learning from Class Imbalances.
- C4.5 and Imbalanced Datasets: Investigating the effect of sampling method, probalistic estimate, and decision tree structure, Chawla, N., ICML-2003 Workshop on Learning from Class Imbalances.
Week 11: Support Vector Machines and Set Covering Machines
Readings:
- Distributional Word Clusters vs. Words for Text Categorization (2002) Ron Bekkerman Ran El-Yaniv Yoad Winter Naftali Tishby.
- Learning with the Set Covering Machine (2001) , Mario Marchand, John Shawe-Taylor.
Week 12: Combination of Classifiers (Bagging, Random Forests, Boosting, Error-Correcting Codes, Mixtures-of-Experts) Reading:
- Random Forests(Technical Report), Breiman, 1999
- Limited Hierarchical Fusion of Multiple Classifiers for Hyperspectral Data Analysis , Shailesh Kumar and Joydeep Ghosh and Melba M. Crawford, 2002.
Week 13: Project Presentations

Readings

Required

A compilation of selected research papers from the recent literature.

References

Machine Learning, Tom Mitchell, McGraw Hill, 1997.
Introduction to Machine Learning, Nils J. Nilsson (Draft of a Proposed New Textbook available on the Web)
Data Mining, Witten, I & Frank, E., Morgan-Kaufmann, 2000.