multiple classifier combination for character recognition: revisiting the majority voting system and...
Post on 14-Dec-2015
225 Views
Preview:
TRANSCRIPT
Multiple Classifier Combination for Character Recognition: Revisiting the Majority Voting System and its
Variations
M.C. Fairhurst
University of Kent
UK
A. F. R. Rahman and H. Alam
BCL Technologies Inc.
USA
Basic Problem Statement
Given a number of experts working on the same problem, is group decision superior to individual decisions?
Ghosts from the Past…
• Jean-Charles de Borda (1781)
• N. C. de Condorcet (1785)• Laplace (1795)• Issac Todhunter (1865) •
•CC. L. Dodgson (Lewis Carrol) (1873) • M. W. Crofton (1885)• E. J. Nanson (1907)• Francis Galton (1907)
Is Democracy the answer?
• Infinite Number of Experts
• Each Expert Should be Competent
How Does It Relate to Character Recognition?
Each Expert has its:
• Strengths and Weaknesses
• Peculiarities
• Fresh Approach to Feature Extraction
• Fresh Approach to Classification
• But NOT 100% Correct!
Practical Resource Constraints
Unfortunately, We Have Limited
• Number of Experts
• Number of Training Samples
• Feature Size
• Classification Time
• Memory Size
Solution
• Clever Algorithms to Exploit Experts– Complimentary Information– Redundancy: Check and Balance– Simultaneous Use of Arbitrary Features and
Classification Routines
Question?– Recent trend is towards complicated decision combination
schemes– Exhaustive Classifier Selection– Theoretical analysis in place of empirical methods
How sophisticated (read “complex”)
algorithms do we really need?
Majority Voting Philosophy
• Should the decision agreed by the majority of the experts be accepted without giving due credit to the competence of the experts?
---- OR ----• Should the decision delivered by the most
competent expert be accepted without giving any importance to the majority consensus?
[1] Simple Majority Voting
Classifier Classifier Classifier
ClassificationDecision
ClassificationDecision
ClassificationDecision
Decision Fusion : Counting Individual Votes to Support aDecision
Final Decision
21
12n
n
k
Decision accepted if at least k of the experts agree, where
If n is even,
If n is odd.
[2] Weighted Majority Voting
OverallWeight of
theClassifier
Classifier
OverallWeight of
theClassifier
Classifier
OverallWeight of
theClassifier
Classifier
ClassificationDecision
ClassificationDecision
ClassificationDecision
Decision Fusion: Counting Weighted Votes for Individual Decisions to Support a FinalDecision
Final Decision
[2] Weighted Majority Voting (Contd.)
So if decision to assign the unknown pattern to the class is denoted by with , being the number of classes, then the final combined decision supporting assignment to the class takes the form of:
The final decision is therefore:
thk thiikd mi 1 m
cmid
thi
nk
ikkcomi dd
,...,2,1
*
comd
comimi
com dd ,..,2,1max
[3] Class-wise Weighted Majority Voting
Class-basedWeight of
theClassifier
Classifier
Class-basedWeight of
theClassifier
Classifier
Class-basedWeight of
theClassifier
Classifier
ClassificationDecision
ClassificationDecision
ClassificationDecision
Decision Fusion: Counting Class-based Weighted Votes for Individual Class Decisions toSupport a Final Decision
Final Decision
[4] Restricted Majority Voting (Top Choice)
OverallClassifierWeight
ClassifierOverall
ClassifierWeight
ClassifierOverall
ClassifierWeight
Classifier
Best Classifier Selection
Decision Fusion: Selection of Best Decision Delivered by BestClassifier
Final Decision
[4] Restricted Majority Voting
(Generalized)
[5] Class-wise Best Decision Selection
Class-basedClassifierWeight
ClassifierClass-based
ClassifierWeight
ClassifierClass-based
ClassifierWeight
Classifier
Best Class-basedClassifier Selection
Decision Fusion: Selection of Best Class-based DecisionDelivered by Best Class-wise Classifier
Final Decision
[6] Enhanced Majority Voting
[7] Ranked Majority Voting
• Not only the top choice, but ranked list of other classes• Takes account of the negative votes cast by the experts against a particular
decision. • Each expert not only supplies the top choice (class) decision, but also supplies the
ranking of all the other choices considered.• The idea is to translate this ranking into ``scores" which would be comparable
across all the decisions by all the experts.
[7] Ranked Majority Voting: Continued
(Class Set Reordering)
• Highest Rank: Take the highest assigned rank
• Borda Count: Sum of the number of classes ranked below it by each classifier.
• Regression Method
Selection of a Database
• NIST Handwritten Characters
• Collected Off-line
• Total 34 Classes (0-9, A-Z, no Distinction between 0/O and I/1)
• Total Samples of Over 34,000 characters
• Size Normalized to 32X32
Performance of the Classifiers
Expert Accepted Recog. Error Rej.
FWS 97.35 78.76 18.59 2.65
MPC 97.62 85.78 11.84 2.38
BWS 95.50 72.31 23.19 4.50
MLP 95.13 82.31 12.82 4.87
Performance of the CombinationCombination Method Accepted Recog. Error Rej.
Simple 96.59 90.59 6.00 3.41
Weighted 96.85 90.64 6.21 3.15
Class-wise Weighted 96.86 90.70 6.16 3.14
Restricted Top Choice 95.68 88.97 6.71 4.32
Class-wise Best Decision 96.76 89.64 6.79 3.24
Restricted Generalized 96.54 90.63 5.91 3.46
Enhanced (ENOCORE) 97.14 90.91 6.23 2.86
Ranked (Borda) 96.99 90.77 6.22 3.01
Committee 95.98 89.63 6.35 4.02
Regression 97.68 90.68 6.83 2.32
Comparative Study
Method Accepted Recogn. Error Reject
BKSM 96.20 90.84 5.36 3.80
Sum Rule
96.40 90.21 6.19 3.60
GA 96.36 92.39 3.97 3.64
Best of MVS
97.14 90.91 6.23 2.86
Conclusions• Majority Voting Solutions can be very versatile
and adaptive• Different variations may be adopted for different
problem domains• The Majority Voting configuration is generic• Majority Voting Systems may be as applicable to
any task domains with equal effectiveness as other complicated solutions
top related