3 ème journée doctorale g&e, bordeaux, mars 2015 wei feng geo-resources and environment lab,...

3ème Journée Doctorale G&E, Bordeaux, Mars 2015

Wei FENG Geo-Resources and Environment Lab, Bordeaux INP (Bordeaux Institute of Technology), France

Supervisor: Samia BOUKIR

CLASSIFICATION OF SATELLITE IMAGES USING MARGIN-BASED ENSEMBLE METHODS. APPLICATION TO LAND COVER MAPPING OF NATURAL SITES


Outline

Context and objectives Ensemble learning and margin theory Mislabeled training data identification and filtering based

on ensemble margin Conclusions

2


Objective = Multiple classifier framework, based on ensemble margin, to effectively and efficiently map remote sensing data

Major challenges in ensemble learning Training data class imbalance Training data redundancy Training data class mislabeling

3

Context and Objectives

ICIP 2013September 2013, Melbourne, Australia 3

Forest image Land cover map


I. Ensemble learning and margin theory

3ème Journée Doctorale G&E, Bordeaux, Mars 2015 5

Condorcet theorem (1785): even if the members of a group have just 50% of chance to individually take the right decision, a majority voting of the same group has nearly 100% of chance to take the right decision!

First use of ensemble concept in machine learning: Hansen & Salamon (IEEE PAMI 1990)

Classification by Random Forests (decision tree-based ensemble): Breiman (Machine Learning 2001)

Marquis de Condorcet French mathematician and philosopher (1743-

1794)

Introduction to ensemble learning


Typical ensemble method phases Production of multiple homogeneous classifiers, and Their combination

6

Introduction to ensemble learning

Typical ensemble creation method = bagging bootstrap sampling (with replacement) over training data to produce diverse classifiers components of ensemble

Typical multiple classifiers combination = majority voting ensemble decision = class with most votes

6


Margin of ensemble methods

Difference between votes to different classes

Classification confidence

One popular ensemble margin = difference between the fraction of classifiers voting correctly and incorrectly

7


II. Mislabeled training data identification and filtering based on ensemble margin


Mislabeling problem in machine learning

I am confused!!

Mislabeling error

9

I am a cow!


Ensemble-based class noise identification

Typical class noise filtering approach: Majority vote filter (Brodley et al. 1999 )

Principle:

10

If more than half (>50%) of all the base classifiers of the ensemble classify an instance incorrectly, then this instance is tagged as mislabeled.

Weakness:

It identifies all the clean training instances that have been wrongly classified by the ensemble classifier as mislabeled false positives


Margin-based mislabeled instance elimination algorithmApproach: noise ranking-based

Noise evaluation function:

N(xi) = |margin (xi)| (xi,yi) S / C(xi) ≠ yi

1. Construct an ensemble classifier C with all training data (xi,yi) S2. Compute the margin of each training instance xi

3. Order all the training instances xi, that have been misclassified, according to their noise evaluation values N(xi), in descending order

4. Eliminate the first M most likely mislabeled instances xi to form a new cleaner training set

5. Evaluate the cleaned training set by classification performance, on a validation set V

6. Select the best filtered training set

Alg

ori

thm

11


Margin-based mislabeled instance correction algorithmApproach: noise ranking-based

Noise evaluation function:

N(xi) = |margin (xi)| (xi,yi) S / C(xi) ≠ yi

1. Construct an ensemble classifier C with all training data (xi,yi) S2. Compute the margin of each training instance xi

3. Order all the training instances xi, that have been misclassified, according to their noise evaluation values N(xi), in descending order

4. Correct the labels of first M most likely mislabeled instances xi using the predicted labels to form a new cleaner training set.

5. Evaluate the repaired training set by classification performance, on a validation set V

6. Select the best corrected training set.

Alg

ori

thm

12


Margin-based mislabeled instance identification results

Data sets

Three remote sensing datasets for land cover mapping of sites of different types

Data set Training Validation Test Variables Classes

Forest 1946 973 1946 4 2

Urban 6800 3400 6800 3 4

Agriculture 2000 1000 2000 36 6

13

Table 1. Data sets


Margin-based mislabeled instance identification resultsNoise filtering performance® Noise-sensitive ensemble classifier = Boosting

® Two types of class noise:

Random noise = 20% (training and validation sets with a percentage of randomly

mislabeled instances)

Actual noise = unknown (amount and type)

® Noise filter strategy: adaptive filtering ADAPTIVE FILTERING eliminates or corrects an amount of ordered detected

mislabeled instances = amount that led to maximum accuracy on validation set

14


Margin-based mislabeled instance identification results: Artificial noise

Table 2. Overall accuracies (%) of boosting classifier with no filter, majority vote filtered and margin-based filtered training sets on artificially corrupted data sets (noise rate=20%)

15

Data set No filter

Noise removal Noise correctionMajority

filterMargin

filterMajority

filter Margin filter

Forest 88.40 87.82 88.40 87.92 88.36

Urban 64.48 64.14 68.39 63.99 66.73

Agriculture 83.38 85.68 88.66 82.66 88.15

Increase in accuracy of up to 5%


Margin-based mislabeled instance identification results: Actual noise

Table 3. Overall accuracies (%) of boosting classifier with no filter, majority vote filtered and margin-based filtered training sets on original data sets

16

Data set No filter

Noise removal Noise correctionMajority

filterMargin

filterMajority

filter Margin filter

Forest 88.75 88.25 88.77 88.12 88.21

Urban 67.34 63.07 69.22 63.68 68.69

Agriculture 90.31 87.85 90.15 85.46 89.70

Increase in accuracy of 2%


Conclusions

Ensemble margin = effective and efficient guideline to ensemble design

Ensemble learning and ensemble margin are effective for land cover mapping

Ensemble margin-based class noise filters are significantly more accurate than majority votes filters.

1717

3 ème journée doctorale g&e, bordeaux, mars 2015 wei feng geo-resources and environment lab,...

Documents

ensemble margin slide

ensemble learning slide

ensemble classifier

journe doctorale ge

popular ensemble margin

ensemble margin conclusions

objectives ensemble

marginbased ensemble