Margin-based generalization error bounds for threshold decision lists

This paper concerns the use of threshold decision lists for classifying data into two classes. The use of such methods has a natural geometrical interpretation and can be appropriate for an iterative approach to data classification, in which some points of the data set are given a particular classification, according to a linear threshold function (or hyperplane), are then removed from consideration, and the procedure iterated until all points are classified. We analyse theoretically the generalization properties of data classification techniques that are based on the use of threshold decision lists and the subclass of multilevel threshold functions. We obtain bounds on the generalization error that depend on the levels of separation — or margins —achieved by the successive linear classifiers.