CSI5387 Concept Learning Systems/Machine Learning


Nathalie Japkowicz
Office: STE 5-029
Phone: 562-5800 ext. 6693
E-mail: nat@site.uottawa.ca

Meeting Times and Locations

  • Time: Tuesdays 11:30pm-2:30pm
  • Location: LMX 112

Office Hours and Locations

  • Times:
    Thursdays 12:00pm-2:00pm
  • or by appointment
  • Location: STE 5-029;


Machine Learning is the area of Artificial Intelligence concerned with the problem of building computer programs that automatically improve with experience. The intent of this course is to present a broad introduction to the principles and paradigms underlying machine learning, including presentations of its main approaches, discussions of its major theoretical issues, and overviews of its most important research themes.

Course Format

The course will consist of a mixture of regular lectures and student presentations. The regular lectures will cover descriptions and discussions of the major approaches to Machine Learning as well as of its major theoretical issues. The student presentations will focus on the most important themes we survey. These themes will mostly be approached through recent research articles from the Machine Learning literature.


Students will be evaluated on short written commentaries and oral presentations of research papers (20%), on a few homework assignments (30%), and on a final class project of the student's choice (50%). For the class project, students can propose their own topic or choose from a list of suggested topics which will be made available at the beginning of the term. Project proposals will be due in mid-semester. Group discussions are highly encouraged for the research paper commentaries and students will be allowed to submit their reviews in teams of 3 or 4. However, homework and projects must be submitted individually.


Students should have reasonable exposure to Artificial Intelligence and some programming experience in a high level language.

Required Textbooks

Additional References .


Other Reading Material

Research papers will be available from Conference Proceedings or Journals available from the Web.

(Links appear in the Syllabus table below, in the Readings column)

List of Major Approaches Surveyed

  • Version Spaces
  • Decision Trees
  • Artificial Neural Networks
  • Bayesian Learning
  • Instance-Based Learning
  • Support Vector Machines
  • Meta-Learning Algorithms
  • Rule Learning/Inductive Logic Programming
  • Unsupervised Learning/Clustering
  • Genetic Algorithms

List of Theoretical Issues Considered

  • Experimental Evaluation of Learning Algorithms
  • Computational Learning Theory

List of Major Themes Surveyed

·         Active Learning

·         Anomaly Detection

·         Graph Mining

·         Evaluation

·         Discovery/Mining

·         Miscellaneous

Homework Related material:

· List of Themes/Papers for this year

· Schedule of Presentations

· Assignment 1 (due date: February 7, 2012; extended to February 14, 2012)

· Assignment 2 (due date: March 6, 2012)

· Assignment 3 (due date: March 27, 2012)


Course Support:

· Suggested Outline for Paper Commentaries

· Project Description

· Guidelines for the Final Project Report

· R Code from the Japkowicz and Shah Evaluation Book


Machine Learning Ressources on the Web:

· David Aha's Machine Learning Resource Page

· UCI Machine Learning


· Free Book: Information Theory, Inference, and Learning Algorithms, David MacKay





Week 1:

Jan 9

Introduction 1: Organizational Meeting

Introduction 2: Overview of Machine Learning

Witten & Frank: Chapter 1

Week 2:

Jan 16

Approach: Versions Space Learning

Additional Slides on: inductive learning theory, version spaces, decision trees and neural nets

Approach: Decision Tree Learning

Nilsson: Chapter 3

Witten & Frank, Sections 4.3 & 6.1

Week 3:

Jan 23

Homework 1 HANDED OUT on Tuesday

Theoretical Issue: Experimental Evaluation of Learning Algorithms I

Approach: Artificial Neural Networks


Japkowicz & Shah, Chapters 3, 4

Witten & Frank, pp. 223-235



Week 4:

Jan 30

Theoretical Issue: Experimental Evaluation of Learning Algorithms II

Theme: Evaluation

Text: Japkowicz & Shah, Chapters 5, 6

Theme Readings:

Week 5:

Feb 6

Homework 1 DUE on Tuesday

Approach: Bayesian Learning

Theme: Machine Learning in Medecine: Presentation by Dr. Marina Sokolova
(CHEO Research Institute, Canada)

Texts: Witten & Frank, Sections 4.2 and 6.7


Week 6:

Feb 13

Project Proposal DUE on Tuesday

Homework 2 HANDED OUT on Tuesday

Approach: Instance-Based Learning

Theme: Active Learning

Texts: Witten & Frank, Sections 4.7 and 6.4

Theme Readings:

Week 7

Feb 20



Week 8:

Feb 27

Approach: Rule Learning/Association Mining

Theme: Anomaly Detection

Texts: Witten & Frank, Sections 4.4 and 6.2

Theme Readings:

Week 9:

March 5

Homework 2 DUE on Tuesday

Approach: Support Vector Machines

Theme: Graph Mining

Texts: Witten & Frank, Sections 4.6 and 6.3

Theme Readings:

Week 10:

Mar 12

Homework 3 HANDED OUT on Tuesday

Approach: Classifier Combination

Theme: Discovery/Mining

Texts: Witten & Frank, Section 7.5

Theme Readings:

Week 11:

Mar 19

Theoretical Issue: Computational Learning Theory

Approach: Unsupervised Learning

Theme: Misceallenous

Texts: See Tom Mitchell’s book

Texts: Witten & Frank, Sections 4.8 and 6.6.

Theme Readingss:

·         Semi-Supervised Feature Importance Evaluation with Ensemble Learning, Barkia Hasna, Elghazel Haytham, and Aussem Alex, ICDM’11

Week 12:

Mar 26

Homework 3 DUE on Tuesday

Approach: Genetic Algorithms

Theme: TBA

Texts: See Tom Mitchell’s book

Theme Readings: TBA

Week 13:

Apr 2

Projects Presentation