Presentation on theme: "Multiple Classifier Combination for Character Recognition: Revisiting the Majority Voting System and its Variations M.C. Fairhurst University of Kent UK."— Presentation transcript:
Multiple Classifier Combination for Character Recognition: Revisiting the Majority Voting System and its Variations M.C. Fairhurst University of Kent UK A. F. R. Rahman and H. Alam BCL Technologies Inc. USA
Basic Problem Statement Given a number of experts working on the same problem, is group decision superior to individual decisions?
Ghosts from the Past… Jean-Charles de Borda (1781) N. C. de Condorcet (1785) Laplace (1795) Issac Todhunter (1865) CC. L. Dodgson (Lewis Carrol) (1873) M. W. Crofton (1885) E. J. Nanson (1907) Francis Galton (1907)
Is Democracy the answer? Infinite Number of Experts Each Expert Should be Competent
How Does It Relate to Character Recognition? Each Expert has its: Strengths and Weaknesses Peculiarities Fresh Approach to Feature Extraction Fresh Approach to Classification But NOT 100% Correct!
Practical Resource Constraints Unfortunately, We Have Limited Number of Experts Number of Training Samples Feature Size Classification Time Memory Size
Solution Clever Algorithms to Exploit Experts –Complimentary Information –Redundancy: Check and Balance –Simultaneous Use of Arbitrary Features and Classification Routines
Question? –Recent trend is towards complicated decision combination schemes –Exhaustive Classifier Selection –Theoretical analysis in place of empirical methods How sophisticated (read “complex”) algorithms do we really need?
Majority Voting Philosophy Should the decision agreed by the majority of the experts be accepted without giving due credit to the competence of the experts? ---- OR ---- Should the decision delivered by the most competent expert be accepted without giving any importance to the majority consensus?
 Simple Majority Voting Decision accepted if at least k of the experts agree, where If n is even, If n is odd.
 Weighted Majority Voting (Contd.) So if decision to assign the unknown pattern to the class is denoted by with, being the number of classes, then the final combined decision supporting assignment to the class takes the form of: The final decision is therefore:
 Ranked Majority Voting Not only the top choice, but ranked list of other classes Takes account of the negative votes cast by the experts against a particular decision. Each expert not only supplies the top choice (class) decision, but also supplies the ranking of all the other choices considered. The idea is to translate this ranking into ``scores" which would be comparable across all the decisions by all the experts.
 Ranked Majority Voting: Continued ( Class Set Reordering) Highest Rank: Take the highest assigned rank Borda Count: Sum of the number of classes ranked below it by each classifier. Regression Method
Selection of a Database NIST Handwritten Characters Collected Off-line Total 34 Classes (0-9, A-Z, no Distinction between 0/O and I/1) Total Samples of Over 34,000 characters Size Normalized to 32X32
Performance of the Classifiers ExpertAcceptedRecog.ErrorRej. FWS97.3578.7618.592.65 MPC97.6285.7811.842.38 BWS95.5072.3123.194.50 MLP95.1382.3112.824.87
Performance of the Combination Combination MethodAcceptedRecog.ErrorRej. Simple96.5990.596.003.41 Weighted96.8590.646.213.15 Class-wise Weighted96.8690.706.163.14 Restricted Top Choice95.6888.976.714.32 Class-wise Best Decision96.7689.646.793.24 Restricted Generalized96.5490.635.913.46 Enhanced (ENOCORE)97.1490.916.232.86 Ranked (Borda)96.9990.776.223.01 Committee95.9889.636.354.02 Regression97.6890.686.832.32
Comparative Study MethodAcceptedRecogn.ErrorReject BKSM96.2090.845.363.80 Sum Rule 96.4090.216.193.60 GA96.3692.393.973.64 Best of MVS 97.1490.916.232.86
Conclusions Majority Voting Solutions can be very versatile and adaptive Different variations may be adopted for different problem domains The Majority Voting configuration is generic Majority Voting Systems may be as applicable to any task domains with equal effectiveness as other complicated solutions