impact of imagenet model selection on domain adaptation of imagenet model... · 2020. 4. 7. ·...

Impact of ImageNet Model Selection on Domain Adaptation

Youshan Zhang & Brian D. Davison

03/01/2020

Computer Science & Engineering

Outline

5

2

3

4

Problem

Methods

State-of-the-art methods

1 Introduction

Conclusion & Future work

Datasets & Results

6

1

Introduction

Background

Ø Big data & different types of data

Ø Original elementary form & not labeled

Ø Manually label data: time-consuming & expensive

Ø Model reuse based on labeled data

§ Conventional machine learning: lower accuracy

1

2Text Image Audio

What is domain adaptation (DA)?

Human: previous experience and knowledge for reasoning & learning

Machine: apply knowledge from other fields into current applications

Key idea:

Ø How to find similar domain knowledge for transferring?

1

3

Why DA ?

Label data: time-consuming & expensive

Train from scratch: tedious

Design customized model: complex

Data shift/bias

1

4Source Domain Target Domain

Outline

5

2

3

4

Problem

Methods

State-of-the-art methods

1 Introduction

Conclusion & Future work

Datasets & Results

6

5

Problem2

6

XT = { (xj, ?) | j = 1, 2, … , nt}

XS = { (xi, yi) | i = 1, 2, … , ns}

Outline

5

2

3

4

Problem

Methods

State-of-the-art methods

1 Introduction

Conclusion & Future work

Datasets & Results

6

7

State-of-the-art methods3

8

Traditional methods

Ø Feature selection[Blitzer et al., 2006; Long et al., 2014]

Ø Subspace learning[Gopalan et al., 2011; Gonget al., 2012; Zhang et al., 2019b]

Ø Distribution adaptation[Panet al., 2011; Jiang et al., 2017; Wang et al., 2018]

Deep learning based methods

Ø Discrepancy based[Tzeng et al., 2014; Long et al.,2015; Ghifary et al., 2015]

Ø Reconstruction based[Bousmalis et al., 2016]

Ø Adversarial learning based[Ganinet al., 2016; Tzeng et al., 2017; Liu et al., 2019]

Limitations3

9

More or less rely on the backbone networks

Not explore other ImageNet models

Not know which is the best layer

Outline

5

2

3

4

Problem

Methods

State-of-the-art methods

1 Introduction

Conclusion & Future work

Datasets & Results

6

10

ImageNet Models4

11

ImageNet Models4

12

Extracted features visualization 4

13

Domain adaptation methods4

Support vector machines (SVM) & 1-nearest neighbor (1NN)

Geodesic Flow Kernel (GFK) & Geodesic sampling on manifolds (GSM)

CORrelation Alignment (CORAL)

Transfer Joint Matching (TJM)

Balanced distribution adaptation(BDA) & Joint distribution alignment (JDA) & Joint Geometrical and Statistical Alignment (JGSA) & Adaptation Regularization (ARTL) & Manifold Embedded Distribution Alignment (MEDA) & Modified Distribution Alignment (MDA)

14

Significance analysis 4

Correlation coefficient

Coefficient of determination

Ø The higher the better

15

Outline

5

2

3

4

Problem

Methods

State-of-the-art methods

1 Introduction

Conclusion & Future work

Datasets & Results

6

16

Datasets & Results5

17

Datasets

Results5

18

Office + Caltech 10

Results5

19

Office-Home

Results5

20

Office31

Classification Accuracy5

21

Classification Accuracy5

22

Classification Accuracy5

23

Best feature extraction layer5

24

Take home messages5

Features from a higher-performing ImageNet-trained model are more valuable than those from a lower-performing model for unsupervised domain adaptation

The layer prior to the last fully connected layer is the best layer for feature extraction

25

Conclusion & future work6

We are the first to examine how features from many different ImageNet models affect domain adaptation

Search the best architecture for feature extraction

Feature fusion

26

6

27

Thank you!

Questions?