Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Fine-grained Flower and Fungi Classification at CMP

Milan Šulc1 Lukaacuteš Picek2 Jiřiacute Matas1

1 Visual Recognition Group Center for Machine Perception

Dept of Cybernetics FEE Czech Technical University in Prague

2 Dept of Cybernetics FAS University of West Bohemia

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Label Distributions

215

FGVCx Flower and Fungi Classification datasets available for training follow a ldquolong-tail distributionrdquo of classes which may not correspond with the test-time distribution

FGVCx Flowers FGVCx Fungi

0 200 400 600 800Class (sorted)

00

01

02

03

04

05

06

s

am

ple

s

Training set

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

0 10 20Class (sorted by Ntrain)

0

500

1000

1500

2000

2500

im

ag

es

Test set

Training set

Label Distributions

315

0 2000 4000 6000 8000 10000Class (sorted by Ntrain)

0

500

1000

1500

2000

2500

im

ag

es

Test set

Training set

We recently observed a similar problem in the LifeCLEF plant identification challenge majority of training data comes from the web while test images come from a different source

Can we compensate for this imbalance

Figure PlantCLEF 2017 label distribution in the ldquotrustedrdquo training set

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Training neural networks (f with parameters ) by cross-entropy loss minimization means training it to estimate the posterior probabilities

where

Then

CNN Outputs as Posterior Estimates

415

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Experiment on selected subsets of CIFAR-100 with different class priorsHow well do the posterior estimates marginalize over dataset samples

CNN Outputs as Posterior Estimates

515

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Assuming that the probability density function remains unchanged

The mutual relation of the posteriors is

Adjusting Estimates to New Priors

615

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When Test Set Priors Are Unknown

715

How to estimate the test-set priors

Saerens et al [1] proposed a simple EM procedure to maximize the likelihood L(x

0x

1x

2)

This procedure is equivalent [2] to fixed-point-iteration minimization of

the KL divergence between and

[1] Adjusting the outputs of a classifier to new a priori probabilities a simple procedure Marco Saerens Patrice Latinne and Christine Decaestecker Neural computation 141 (2002) 21-41[2] Semi-supervised learning of class balance under class-prior change by distribution matching Marthinus Christoffel Du Plessis and Masashi Sugiyama Neural Networks 50110ndash119 2014

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Test Set Prior Estimation in LifeCLEF

815

Preliminary experiments (using the 2017 test set for validation)

When the whole test set is availableInception-ResNet-v2 829 rarr 858Inception-v4 828 rarr 863

On-line [1] after each new test image

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

915

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

FGVCx Fungi 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1015

Note in the iNaturalist 2017 challenge the winning GMV submission [1] approached the change in priors as follows

ldquoTo compensate for the imbalanced training data the models were further fine-tuned on the 90 subset of the validation data that has a more balanced distributionrdquo

We instead only use the validation set statistics ndash ie uniform class distribution in this case

[1] The iNaturalist Species Classification and Detection Dataset-Supplementary Material Grant Van Horn Oisin Mac Aodha Yang Song Yin Cui Chen Sun Alex Shepard Hartwig Adam Pietro Perona and Serge Belongie Reptilia 32 no 400 5426

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1115

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

0 50000 100000 150000 200000 250000 300000 350000 400000Training steps

350

375

400

425

450

475

500

525

Acc

ura

cy [

]

CNN output accuracy

Known (flat) test distr

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

FGVCx Fungi 2018 Inception v4

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Tricks used in both challenges

1215

Predictions re-weighted simply assuming uniform class priors

Moving average of trained variables (exponential decay)

Training time augmentation- Random crops- Color distortions

Test-time data augmentation14 per image 7 crops 2 (mirror)⨯ ⨯

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Label Distributions

215

FGVCx Flower and Fungi Classification datasets available for training follow a ldquolong-tail distributionrdquo of classes which may not correspond with the test-time distribution

FGVCx Flowers FGVCx Fungi

0 200 400 600 800Class (sorted)

00

01

02

03

04

05

06

s

am

ple

s

Training set

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

0 10 20Class (sorted by Ntrain)

0

500

1000

1500

2000

2500

im

ag

es

Test set

Training set

Label Distributions

315

0 2000 4000 6000 8000 10000Class (sorted by Ntrain)

0

500

1000

1500

2000

2500

im

ag

es

Test set

Training set

We recently observed a similar problem in the LifeCLEF plant identification challenge majority of training data comes from the web while test images come from a different source

Can we compensate for this imbalance

Figure PlantCLEF 2017 label distribution in the ldquotrustedrdquo training set

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Training neural networks (f with parameters ) by cross-entropy loss minimization means training it to estimate the posterior probabilities

where

Then

CNN Outputs as Posterior Estimates

415

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Experiment on selected subsets of CIFAR-100 with different class priorsHow well do the posterior estimates marginalize over dataset samples

CNN Outputs as Posterior Estimates

515

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Assuming that the probability density function remains unchanged

The mutual relation of the posteriors is

Adjusting Estimates to New Priors

615

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When Test Set Priors Are Unknown

715

How to estimate the test-set priors

Saerens et al [1] proposed a simple EM procedure to maximize the likelihood L(x

0x

1x

2)

This procedure is equivalent [2] to fixed-point-iteration minimization of

the KL divergence between and

[1] Adjusting the outputs of a classifier to new a priori probabilities a simple procedure Marco Saerens Patrice Latinne and Christine Decaestecker Neural computation 141 (2002) 21-41[2] Semi-supervised learning of class balance under class-prior change by distribution matching Marthinus Christoffel Du Plessis and Masashi Sugiyama Neural Networks 50110ndash119 2014

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Test Set Prior Estimation in LifeCLEF

815

Preliminary experiments (using the 2017 test set for validation)

When the whole test set is availableInception-ResNet-v2 829 rarr 858Inception-v4 828 rarr 863

On-line [1] after each new test image

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

915

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

FGVCx Fungi 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1015

Note in the iNaturalist 2017 challenge the winning GMV submission [1] approached the change in priors as follows

ldquoTo compensate for the imbalanced training data the models were further fine-tuned on the 90 subset of the validation data that has a more balanced distributionrdquo

We instead only use the validation set statistics ndash ie uniform class distribution in this case

[1] The iNaturalist Species Classification and Detection Dataset-Supplementary Material Grant Van Horn Oisin Mac Aodha Yang Song Yin Cui Chen Sun Alex Shepard Hartwig Adam Pietro Perona and Serge Belongie Reptilia 32 no 400 5426

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1115

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

0 50000 100000 150000 200000 250000 300000 350000 400000Training steps

350

375

400

425

450

475

500

525

Acc

ura

cy [

]

CNN output accuracy

Known (flat) test distr

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

FGVCx Fungi 2018 Inception v4

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Tricks used in both challenges

1215

Predictions re-weighted simply assuming uniform class priors

Moving average of trained variables (exponential decay)

Training time augmentation- Random crops- Color distortions

Test-time data augmentation14 per image 7 crops 2 (mirror)⨯ ⨯

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

0 10 20Class (sorted by Ntrain)

0

500

1000

1500

2000

2500

im

ag

es

Test set

Training set

Label Distributions

315

0 2000 4000 6000 8000 10000Class (sorted by Ntrain)

0

500

1000

1500

2000

2500

im

ag

es

Test set

Training set

We recently observed a similar problem in the LifeCLEF plant identification challenge majority of training data comes from the web while test images come from a different source

Can we compensate for this imbalance

Figure PlantCLEF 2017 label distribution in the ldquotrustedrdquo training set

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Training neural networks (f with parameters ) by cross-entropy loss minimization means training it to estimate the posterior probabilities

where

Then

CNN Outputs as Posterior Estimates

415

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Experiment on selected subsets of CIFAR-100 with different class priorsHow well do the posterior estimates marginalize over dataset samples

CNN Outputs as Posterior Estimates

515

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Assuming that the probability density function remains unchanged

The mutual relation of the posteriors is

Adjusting Estimates to New Priors

615

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When Test Set Priors Are Unknown

715

How to estimate the test-set priors

Saerens et al [1] proposed a simple EM procedure to maximize the likelihood L(x

0x

1x

2)

This procedure is equivalent [2] to fixed-point-iteration minimization of

the KL divergence between and

[1] Adjusting the outputs of a classifier to new a priori probabilities a simple procedure Marco Saerens Patrice Latinne and Christine Decaestecker Neural computation 141 (2002) 21-41[2] Semi-supervised learning of class balance under class-prior change by distribution matching Marthinus Christoffel Du Plessis and Masashi Sugiyama Neural Networks 50110ndash119 2014

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Test Set Prior Estimation in LifeCLEF

815

Preliminary experiments (using the 2017 test set for validation)

When the whole test set is availableInception-ResNet-v2 829 rarr 858Inception-v4 828 rarr 863

On-line [1] after each new test image

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

915

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

FGVCx Fungi 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1015

Note in the iNaturalist 2017 challenge the winning GMV submission [1] approached the change in priors as follows

ldquoTo compensate for the imbalanced training data the models were further fine-tuned on the 90 subset of the validation data that has a more balanced distributionrdquo

We instead only use the validation set statistics ndash ie uniform class distribution in this case

[1] The iNaturalist Species Classification and Detection Dataset-Supplementary Material Grant Van Horn Oisin Mac Aodha Yang Song Yin Cui Chen Sun Alex Shepard Hartwig Adam Pietro Perona and Serge Belongie Reptilia 32 no 400 5426

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1115

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

0 50000 100000 150000 200000 250000 300000 350000 400000Training steps

350

375

400

425

450

475

500

525

Acc

ura

cy [

]

CNN output accuracy

Known (flat) test distr

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

FGVCx Fungi 2018 Inception v4

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Tricks used in both challenges

1215

Predictions re-weighted simply assuming uniform class priors

Moving average of trained variables (exponential decay)

Training time augmentation- Random crops- Color distortions

Test-time data augmentation14 per image 7 crops 2 (mirror)⨯ ⨯

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Training neural networks (f with parameters ) by cross-entropy loss minimization means training it to estimate the posterior probabilities

where

Then

CNN Outputs as Posterior Estimates

415

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Experiment on selected subsets of CIFAR-100 with different class priorsHow well do the posterior estimates marginalize over dataset samples

CNN Outputs as Posterior Estimates

515

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Assuming that the probability density function remains unchanged

The mutual relation of the posteriors is

Adjusting Estimates to New Priors

615

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When Test Set Priors Are Unknown

715

How to estimate the test-set priors

Saerens et al [1] proposed a simple EM procedure to maximize the likelihood L(x

0x

1x

2)

This procedure is equivalent [2] to fixed-point-iteration minimization of

the KL divergence between and

[1] Adjusting the outputs of a classifier to new a priori probabilities a simple procedure Marco Saerens Patrice Latinne and Christine Decaestecker Neural computation 141 (2002) 21-41[2] Semi-supervised learning of class balance under class-prior change by distribution matching Marthinus Christoffel Du Plessis and Masashi Sugiyama Neural Networks 50110ndash119 2014

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Test Set Prior Estimation in LifeCLEF

815

Preliminary experiments (using the 2017 test set for validation)

When the whole test set is availableInception-ResNet-v2 829 rarr 858Inception-v4 828 rarr 863

On-line [1] after each new test image

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

915

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

FGVCx Fungi 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1015

Note in the iNaturalist 2017 challenge the winning GMV submission [1] approached the change in priors as follows

ldquoTo compensate for the imbalanced training data the models were further fine-tuned on the 90 subset of the validation data that has a more balanced distributionrdquo

We instead only use the validation set statistics ndash ie uniform class distribution in this case

[1] The iNaturalist Species Classification and Detection Dataset-Supplementary Material Grant Van Horn Oisin Mac Aodha Yang Song Yin Cui Chen Sun Alex Shepard Hartwig Adam Pietro Perona and Serge Belongie Reptilia 32 no 400 5426

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1115

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

0 50000 100000 150000 200000 250000 300000 350000 400000Training steps

350

375

400

425

450

475

500

525

Acc

ura

cy [

]

CNN output accuracy

Known (flat) test distr

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

FGVCx Fungi 2018 Inception v4

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Tricks used in both challenges

1215

Predictions re-weighted simply assuming uniform class priors

Moving average of trained variables (exponential decay)

Training time augmentation- Random crops- Color distortions

Test-time data augmentation14 per image 7 crops 2 (mirror)⨯ ⨯

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Experiment on selected subsets of CIFAR-100 with different class priorsHow well do the posterior estimates marginalize over dataset samples

CNN Outputs as Posterior Estimates

515

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Assuming that the probability density function remains unchanged

The mutual relation of the posteriors is

Adjusting Estimates to New Priors

615

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When Test Set Priors Are Unknown

715

How to estimate the test-set priors

Saerens et al [1] proposed a simple EM procedure to maximize the likelihood L(x

0x

1x

2)

This procedure is equivalent [2] to fixed-point-iteration minimization of

the KL divergence between and

[1] Adjusting the outputs of a classifier to new a priori probabilities a simple procedure Marco Saerens Patrice Latinne and Christine Decaestecker Neural computation 141 (2002) 21-41[2] Semi-supervised learning of class balance under class-prior change by distribution matching Marthinus Christoffel Du Plessis and Masashi Sugiyama Neural Networks 50110ndash119 2014

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Test Set Prior Estimation in LifeCLEF

815

Preliminary experiments (using the 2017 test set for validation)

When the whole test set is availableInception-ResNet-v2 829 rarr 858Inception-v4 828 rarr 863

On-line [1] after each new test image

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

915

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

FGVCx Fungi 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1015

Note in the iNaturalist 2017 challenge the winning GMV submission [1] approached the change in priors as follows

ldquoTo compensate for the imbalanced training data the models were further fine-tuned on the 90 subset of the validation data that has a more balanced distributionrdquo

We instead only use the validation set statistics ndash ie uniform class distribution in this case

[1] The iNaturalist Species Classification and Detection Dataset-Supplementary Material Grant Van Horn Oisin Mac Aodha Yang Song Yin Cui Chen Sun Alex Shepard Hartwig Adam Pietro Perona and Serge Belongie Reptilia 32 no 400 5426

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1115

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

0 50000 100000 150000 200000 250000 300000 350000 400000Training steps

350

375

400

425

450

475

500

525

Acc

ura

cy [

]

CNN output accuracy

Known (flat) test distr

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

FGVCx Fungi 2018 Inception v4

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Tricks used in both challenges

1215

Predictions re-weighted simply assuming uniform class priors

Moving average of trained variables (exponential decay)

Training time augmentation- Random crops- Color distortions

Test-time data augmentation14 per image 7 crops 2 (mirror)⨯ ⨯

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Assuming that the probability density function remains unchanged

The mutual relation of the posteriors is

Adjusting Estimates to New Priors

615

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When Test Set Priors Are Unknown

715

How to estimate the test-set priors

Saerens et al [1] proposed a simple EM procedure to maximize the likelihood L(x

0x

1x

2)

This procedure is equivalent [2] to fixed-point-iteration minimization of

the KL divergence between and

[1] Adjusting the outputs of a classifier to new a priori probabilities a simple procedure Marco Saerens Patrice Latinne and Christine Decaestecker Neural computation 141 (2002) 21-41[2] Semi-supervised learning of class balance under class-prior change by distribution matching Marthinus Christoffel Du Plessis and Masashi Sugiyama Neural Networks 50110ndash119 2014

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Test Set Prior Estimation in LifeCLEF

815

Preliminary experiments (using the 2017 test set for validation)

When the whole test set is availableInception-ResNet-v2 829 rarr 858Inception-v4 828 rarr 863

On-line [1] after each new test image

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

915

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

FGVCx Fungi 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1015

Note in the iNaturalist 2017 challenge the winning GMV submission [1] approached the change in priors as follows

ldquoTo compensate for the imbalanced training data the models were further fine-tuned on the 90 subset of the validation data that has a more balanced distributionrdquo

We instead only use the validation set statistics ndash ie uniform class distribution in this case

[1] The iNaturalist Species Classification and Detection Dataset-Supplementary Material Grant Van Horn Oisin Mac Aodha Yang Song Yin Cui Chen Sun Alex Shepard Hartwig Adam Pietro Perona and Serge Belongie Reptilia 32 no 400 5426

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1115

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

0 50000 100000 150000 200000 250000 300000 350000 400000Training steps

350

375

400

425

450

475

500

525

Acc

ura

cy [

]

CNN output accuracy

Known (flat) test distr

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

FGVCx Fungi 2018 Inception v4

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Tricks used in both challenges

1215

Predictions re-weighted simply assuming uniform class priors

Moving average of trained variables (exponential decay)

Training time augmentation- Random crops- Color distortions

Test-time data augmentation14 per image 7 crops 2 (mirror)⨯ ⨯

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When Test Set Priors Are Unknown

715

How to estimate the test-set priors

Saerens et al [1] proposed a simple EM procedure to maximize the likelihood L(x

0x

1x

2)

This procedure is equivalent [2] to fixed-point-iteration minimization of

the KL divergence between and

[1] Adjusting the outputs of a classifier to new a priori probabilities a simple procedure Marco Saerens Patrice Latinne and Christine Decaestecker Neural computation 141 (2002) 21-41[2] Semi-supervised learning of class balance under class-prior change by distribution matching Marthinus Christoffel Du Plessis and Masashi Sugiyama Neural Networks 50110ndash119 2014

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Test Set Prior Estimation in LifeCLEF

815

Preliminary experiments (using the 2017 test set for validation)

When the whole test set is availableInception-ResNet-v2 829 rarr 858Inception-v4 828 rarr 863

On-line [1] after each new test image

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

915

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

FGVCx Fungi 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1015

Note in the iNaturalist 2017 challenge the winning GMV submission [1] approached the change in priors as follows

ldquoTo compensate for the imbalanced training data the models were further fine-tuned on the 90 subset of the validation data that has a more balanced distributionrdquo

We instead only use the validation set statistics ndash ie uniform class distribution in this case

[1] The iNaturalist Species Classification and Detection Dataset-Supplementary Material Grant Van Horn Oisin Mac Aodha Yang Song Yin Cui Chen Sun Alex Shepard Hartwig Adam Pietro Perona and Serge Belongie Reptilia 32 no 400 5426

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1115

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

0 50000 100000 150000 200000 250000 300000 350000 400000Training steps

350

375

400

425

450

475

500

525

Acc

ura

cy [

]

CNN output accuracy

Known (flat) test distr

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

FGVCx Fungi 2018 Inception v4

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Tricks used in both challenges

1215

Predictions re-weighted simply assuming uniform class priors

Moving average of trained variables (exponential decay)

Training time augmentation- Random crops- Color distortions

Test-time data augmentation14 per image 7 crops 2 (mirror)⨯ ⨯

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Test Set Prior Estimation in LifeCLEF

815

Preliminary experiments (using the 2017 test set for validation)

When the whole test set is availableInception-ResNet-v2 829 rarr 858Inception-v4 828 rarr 863

On-line [1] after each new test image

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

915

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

FGVCx Fungi 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1015

Note in the iNaturalist 2017 challenge the winning GMV submission [1] approached the change in priors as follows

ldquoTo compensate for the imbalanced training data the models were further fine-tuned on the 90 subset of the validation data that has a more balanced distributionrdquo

We instead only use the validation set statistics ndash ie uniform class distribution in this case

[1] The iNaturalist Species Classification and Detection Dataset-Supplementary Material Grant Van Horn Oisin Mac Aodha Yang Song Yin Cui Chen Sun Alex Shepard Hartwig Adam Pietro Perona and Serge Belongie Reptilia 32 no 400 5426

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1115

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

0 50000 100000 150000 200000 250000 300000 350000 400000Training steps

350

375

400

425

450

475

500

525

Acc

ura

cy [

]

CNN output accuracy

Known (flat) test distr

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

FGVCx Fungi 2018 Inception v4

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Tricks used in both challenges

1215

Predictions re-weighted simply assuming uniform class priors

Moving average of trained variables (exponential decay)

Training time augmentation- Random crops- Color distortions

Test-time data augmentation14 per image 7 crops 2 (mirror)⨯ ⨯

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

915

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

FGVCx Fungi 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1015

Note in the iNaturalist 2017 challenge the winning GMV submission [1] approached the change in priors as follows

ldquoTo compensate for the imbalanced training data the models were further fine-tuned on the 90 subset of the validation data that has a more balanced distributionrdquo

We instead only use the validation set statistics ndash ie uniform class distribution in this case

[1] The iNaturalist Species Classification and Detection Dataset-Supplementary Material Grant Van Horn Oisin Mac Aodha Yang Song Yin Cui Chen Sun Alex Shepard Hartwig Adam Pietro Perona and Serge Belongie Reptilia 32 no 400 5426

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1115

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

0 50000 100000 150000 200000 250000 300000 350000 400000Training steps

350

375

400

425

450

475

500

525

Acc

ura

cy [

]

CNN output accuracy

Known (flat) test distr

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

FGVCx Fungi 2018 Inception v4

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Tricks used in both challenges

1215

Predictions re-weighted simply assuming uniform class priors

Moving average of trained variables (exponential decay)

Training time augmentation- Random crops- Color distortions

Test-time data augmentation14 per image 7 crops 2 (mirror)⨯ ⨯

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1015

Note in the iNaturalist 2017 challenge the winning GMV submission [1] approached the change in priors as follows

ldquoTo compensate for the imbalanced training data the models were further fine-tuned on the 90 subset of the validation data that has a more balanced distributionrdquo

We instead only use the validation set statistics ndash ie uniform class distribution in this case

[1] The iNaturalist Species Classification and Detection Dataset-Supplementary Material Grant Van Horn Oisin Mac Aodha Yang Song Yin Cui Chen Sun Alex Shepard Hartwig Adam Pietro Perona and Serge Belongie Reptilia 32 no 400 5426

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1115

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

0 50000 100000 150000 200000 250000 300000 350000 400000Training steps

350

375

400

425

450

475

500

525

Acc

ura

cy [

]

CNN output accuracy

Known (flat) test distr

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

FGVCx Fungi 2018 Inception v4

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Tricks used in both challenges

1215

Predictions re-weighted simply assuming uniform class priors

Moving average of trained variables (exponential decay)

Training time augmentation- Random crops- Color distortions

Test-time data augmentation14 per image 7 crops 2 (mirror)⨯ ⨯

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

When New Priors Are Known

1115

0 200 400 600 800 1000 1200Class (sorted)

00

01

02

03

04

05

s

am

ple

s

Training set

Validation set

0 50000 100000 150000 200000 250000 300000 350000 400000Training steps

350

375

400

425

450

475

500

525

Acc

ura

cy [

]

CNN output accuracy

Known (flat) test distr

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

FGVCx Fungi 2018 Inception v4

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Tricks used in both challenges

1215

Predictions re-weighted simply assuming uniform class priors

Moving average of trained variables (exponential decay)

Training time augmentation- Random crops- Color distortions

Test-time data augmentation14 per image 7 crops 2 (mirror)⨯ ⨯

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Tricks used in both challenges

1215

Predictions re-weighted simply assuming uniform class priors

Moving average of trained variables (exponential decay)

Training time augmentation- Random crops- Color distortions

Test-time data augmentation14 per image 7 crops 2 (mirror)⨯ ⨯

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Final Ensembles

1315

FGVCx Fungi 6 nets (averaged)

2x Inception-v4 299x299 initialized from ImageNet and LifeCLEF ckpts

2x Inception-v4 598x598 initialized from ImageNet and LifeCLEF ckpts

2x Inception-ResNet-v2 299x299 from ImageNet and LifeCLEF ckpts

FGVCx Flowers 5 nets (modus)

3x Inception-v4 299x299 initialized from ImageNet LifeCLEF iNaturalist ckpts

1x Inception-v4 598x598 initialized from LifeCLEF ckpt

1x Inception-ResNet-v2 299x299 initialized from LifeCLEF ckpt

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

Leaderboard

1415

FGVCx Fungi

FGVCx Flowers

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018

Milan Šulc Fine-grained Flower and Fungi Classification at CMP

bull Standard CNN classifiers (and their ensembles) achievebest results in plant and fungi recognition Future work Learning from Ensembles

bull Important to take into account change in class prior distribution [1] New priors can be estimated on-line as new test-samples appear

bull Q amp A sulcmilacmpfelkcvutcz

1515

Discussion

[1] Improving CNN classifiers by estimating test-time priors Milan Šulc and Jiřiacute Matas arXiv180508235 [csCV] 2018