Dissertation Abstract of Nathalie Japkowicz

Concept-Learning in the Abscence of Counter-Examples: An Autoassociation-Based Approach to Classification

Full Dissertation (in PostScript Format)

The overwhelming majority of research currently pursued within the framework of concept-learning concentrates on discrimination-based learning. Nevertheless, this emphasis can present a practical problem: there are real-world engineering problems for which counter-examples are both scarce and difficult to gather. For these problems, recognition-based learning systems are much more appropriate because they do not use counter-examples in the concept-learning phase and thus require fewer counter-examples altogether. The purpose of this dissertation is to analyze a promising connectionist recognition-based learning system--- autoassociation-based classification---and answer the following questions raised by a preliminary comparison of the autoassociator and its discrimination counterpart, the Multi-Layer Perceptron (MLP), on three real-world domains:

What features of the autoassociator make it capable of performing classification in the absence of counter-examples?
What causes the autoassociator to be significantly more efficient than MLP in certain domains?
What domain characteristics cause the autoassociator to be more accurate than MLP and MLP to be more accurate than the autoassociator?

A study of the two systems in the context of these questions yields the conclusions that 1) Autoassociation-based classification is possible in a particular class of practical domains called non-linear and multi-modal because the autoassociator uses a multi-modal specialization bias to compensate for the absence of counter-examples. This bias can be controlled by varying the capacity of the autoassociator. 2) The difference in efficiency between the autoassociator and MLP observed on this class of domains is caused by the fact that the autoassociator uses a (fast) bottom-up generalization strategy whereas MLP has recourse to a (slow) top-down one, despite the fact that the two systems are both trained by the backpropagation procedure. 3) The autoassociator classifies more accurately than MLP domains requiring particularly strong specialization biases caused by the counter-conceptual class or particularly weak specialization biases caused by the conceptual class. However, MLP is more accurate than the autoassociator on domains requiring particularly strong specialization biases caused by the conceptual class.

The results of this study thus suggest that recognition-based learning, which is often dismissed in favor of discrimination-based ones in the context of concept-learning, may present an interesting array of classification strengths.