Dissertation Abstract of Nathalie Japkowicz
Concept-Learning in the Abscence of Counter-Examples: An
Autoassociation-Based Approach to Classification
Full Dissertation (in PostScript Format)
The overwhelming majority of research currently pursued within
the framework of concept-learning concentrates on discrimination-based
learning. Nevertheless, this emphasis can present a practical problem:
there are real-world engineering problems for which counter-examples are
both scarce and difficult to gather.
For these problems, recognition-based learning systems are much more
appropriate because they do not use counter-examples in the concept-learning
phase and thus require fewer counter-examples altogether.
The purpose of this dissertation is to analyze a promising connectionist
recognition-based learning system--- autoassociation-based
classification---and answer the following questions
raised by a preliminary comparison of the autoassociator and its discrimination
counterpart, the Multi-Layer Perceptron (MLP), on three real-world domains:
- What features of the autoassociator make it capable of performing
classification in the absence of counter-examples?
- What causes the autoassociator to be significantly more efficient than
MLP in certain domains?
- What domain characteristics cause the autoassociator to be more
accurate than MLP and MLP to be more accurate than the autoassociator?
A study of the two systems in the context of these questions yields the
conclusions that
1) Autoassociation-based classification is possible in a particular class of
practical domains called non-linear and multi-modal because the
autoassociator uses a multi-modal specialization bias to compensate for the
absence of counter-examples. This bias can be controlled by varying the
capacity of the autoassociator. 2) The difference in efficiency between the
autoassociator and MLP observed on this class of domains is caused by the fact
that the autoassociator uses a (fast) bottom-up generalization strategy
whereas MLP has recourse to a (slow) top-down one, despite the fact that
the two systems are both trained by the backpropagation procedure. 3) The
autoassociator classifies more accurately than MLP domains requiring
particularly strong specialization biases caused by the
counter-conceptual class or particularly weak specialization
biases caused by the conceptual class. However, MLP is more accurate
than the autoassociator on domains requiring particularly strong
specialization biases caused by the conceptual class.
The results of this study thus suggest that recognition-based learning,
which is often dismissed in favor of discrimination-based ones in
the context of concept-learning, may present an interesting array of
classification strengths.