Sergey Tulyakov

Overview of the Classifier Combination Research

Classifier combination is a new and fast developing area of research closely connected to such areas as machine learning, pattern classification and artificial intelligence. Given expert opinions of few experts (classifiers) it is usually possible to combine these opinions to make better decisions. The main question is how to do it.

In my research I am interested in finding theoretically optimal combination algorithms. The main considered applications are biometric person authentication and handwritten word recognition. Consequently, a small number of classifiers, biometric matchers or handwritten word recognizers, and a large number of classes, persons or lexicon words, is assumed in this research.

I separate combination algorithms into four complexity types. It turns out that each complexity type has its own optimal combination algorithms. Higher complexity implies stronger optimal combination algorithm, but it might be harder to find and the limited number of training samples might not allow its proper training.

Moreover, systems with different optimization criteria, e.g. identification and verification systems, might also possess different optimal combination algorithms. It is possible for optimal combination algorithm in one system to have suboptimal performance in another system, and even decrease the system performance as compared to a single non-combined classifier.

Thus, we need to search for at least 8 (4 complexity types * 2 operational modes) optimal combination algorithms. Only in one case (low complexity combinations for verification systems) we know the optimal solution (likelihood ratio combination method).