CSI5387 Concept Learning Systems/Machine Learning


Nathalie Japkowicz
Office: STE 5-029
Phone: 562-5800 ext. 6693
E-mail: nat@site.uottawa.ca

Meeting Times and Locations

  • Time: Mondays 11:30am-1:00pm; Thursdays 1:00pm-2:30pm;
  • Location: Simard 422

Office Hours and Locations

  • Times: Monday, 1:15pm-2:15pm and Thursdays 2:45pm-3:45pm;
  • Location: STE 5-029;


Machine Learning is the area of Artificial Intelligence concerned with the problem of building computer programs that automatically improve with experience. The intent of this course is to present a broad introduction to the principles and paradigms underlying machine learning, including presentations of its main approaches, discussions of its major theoretical issues, and overviews of its most important research themes.

Course Format

The course will consist of a mixture of regular lectures and student presentations. The regular lectures, based on the textbook, will cover descriptions and discussions of the major approaches to Machine Learning as well as of its major theoretical issues. The student presentations will focus on the most important themes we survey. These themes will mostly be approached through recent research articles from the Machine Learning literature.


Students will be evaluated on short written commentaries and oral presentations of research papers (20%), on a few homework assignments (30%), and on a final class project of the student's choice (50%). For the class project, students can propose their own topic or choose from a list of suggested topics which will be made available at the beginning of the term. Project proposals will be due in mid-semester. Group discussions are highly encouraged for the research paper commentaries and students will be allowed to submit their reviews in teams of 3 or 4. However, homeworks and projects must be submitted individually.


Students should have reasonable exposure to Artificial Intelligence and some programming experience in a high level language.

Required Textbooks

Additional References .

Other Reading Material

Research papers will be available from Conference Proceedings or Journals available from the Web. 

(Links appear in the Syllabus table below, in the Readings column)

List of Major Approaches Surveyed

  • Version Spaces
  • Decision Trees
  • Artificial Neural Networks
  • Bayesian Learning
  • Instance-Based Learning
  • Support Vector Machines
  • Classifier Combinations
  • Rule Learning/Inductive Logic Programming
  • Unsupervised Learning/Clustering
  • Genetic Algorithms

List of Theoretical Issues Considered

  • Experimental Evaluation of Learning Algorithms
  • Computational Learning Theory

List of Major Themes Surveyed

  • Feature Selection
  • Learning from Massive Data sets
  • Cost-Sensitive Learning
  • The class imbalance problem
  • Mining Association Rules
  • Data Visualization
  • Classifier Parallelization
  • Privacy Preserving Data Mining

Course Support:

·         Schedule of Presentations

·         Timetable for Homework

·         Suggested Outline for Paper Commentaries

·         Project Description

·         Guidelines for the Final Project Report

Machine Learning Ressources on the Web:

·         David Aha's Machine Learning Resource Page

·         UCI Machine Learning

·         WEKA

·         Free Book: Information Theory, Inference, and Learning Algorithms, David MacKay





Week 1:

Jan 4-8

Introduction: Organizational Meeting


Week 2:

Jan 9-15

Introduction: Overview of Machine Learning


Approach: Versions Space Learning

Witten & Frank: Chapter 1

 Nilsson: Chapter 3

Week 3:

Jan 16-22

Homework 1 HANDED OUT on Monday

Approach: Decision Tree Learning

Theme: Feature Selection

Witten & Frank, Sections 4.3 & 6.1


Background for the Theme:

Witten & Frank, Section 7.1 [Also, Chapter 2]

Theme Papers:



Week 4:

Jan 23-29

 Theoretical Issue: Experimental Evaluation of Learning Algorithms

No Theme this week: Papers discuss the theoretical issue

Witten & Frank, Chapter 5

Theoretical Issue Papers:

Week 5:

Jan 30- Feb 5
Homework 1 DUE on Monday

Approach: Artificial Neural Networks

Theme: Cost-Sensitive Learning

Texts:                                                                                                                         Witten & Frank, pp. 223-235



Week 6:

Feb 6 - 12

Project Proposal DUE on Thursday

Homework 2 HANDED OUT on Thursday

Approach: Bayesian Learning

Theme: The class imbalance problem

Texts: Witten & Frank, Sections 4.2 and 6.7

Theme Papers:

Week 7:

Feb 13 - 19

Approach: Instance-Based Learning

Theme: Text Mining

Texts: Witten & Frank, Sections 4.7 and 6.4

Theme Papers:


Week 8:

Feb 20 - 26



Week 9:

Feb 27 -  Mar 5

Homework 2 DUE on Monday

Approach: Rule Learning

Theme: Mining Association Rules


Texts: Witten & Frank, Sections 4.4 and 6.2

Theme Papers:



Week 10:

Mar 6 - 12

Homework 3 HANDED OUT on Monday

Approach: Support Vector Machines

Theme: Privacy Preserving Data Mining

Texts: Witten & Frank, Sections 4.6 and 6.3

Theme Papers:


Week 11:

Mar 13 - 19

Approach: Classifier Combination

 Theme: Classifier Parallelization

Texts: Witten & Frank, Section 7.5


·         Strategies for Parallel Data Mining, David Skillicorn

Week 12:

Mar 20 - 26

Homework 3 DUE on Monday

Theoretical Issue: Computational Learning Theory

Theme:  Data Visualization

Texts: See Tom Mitchell’s book

Theme Papers:

Week 13:

Mar 27 – Apr 2

Approach: Unsupervised Learning

Approach:  Genetic Algorithms

Texts: Witten & Frank, Sections 4.8 and 6.6.

Texts: See Tom Mitchell’s book

Week 14:

Apr 3 – 9

Projects Presentation


Week 15:

Apr 10

Projects Presentation