ieee international conference on multimedia and...

Yan Hong, Li Niu, Jianfu Zhang, Liqing Zhang

MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University

IEEE International Conference on Multimedia and Expo6-10 July 2020 || London, United Kingdom || Virtual

Beyond without Forgetting: Multi-Task Learning for Classification with Disjoint Datasets

Catalog

1 Multi-task Learning from Disjoint Datasets

2 Learning from Pseudo Labels

3 Selective Augmentation Method

4 Experiments

Catalog

1

2

3

4

Multi-task Learning from Disjoint Datasets

Learning from Pseudo Labels

Selective Augmentation Method

Experiments

Multi-task Learning

Forgetting Effects

Supervised

Unsupervised

Multi-task Learning without ForgettingSoft label vector

Preserving information

Catalog

2

1

3

4

Multi-task Learning from Disjoint Datasets

Learning from Pseudo Labels

Selective Augmentation Method

Experiments

Multi-task Learning with Augmentation DataOne-hot vector

Augmentation information

Noisy Pseudo Labels

Images from AFLW with the pseudo emotion label “angry”

Distribution Mismatch

Images from AFLW with pose label, captured in good light condition

Images from SFEW with emotion label, captured in poor light condition

Catalog

1

4

3

2

Multi-task Learning from Disjoint Datasets

Learning from Pseudo Labels

Selective Augmentation Method

Experiments

Selecting Data with Confident Pseudo Labels

0.09 0.01 0.03 0.04 0.02 0.81

0 0 0 0 0 1

Soft label vector

Pseudo label vector

Max indexConfidence weight 𝑤𝑤𝑖𝑖𝑐𝑐 = 0.81

Selecting Data with Confident Pseudo Labels

Low local density

High local density

= �1,0, otherwise

Selecting Data with Closer Data Distribution

Label Vector Interpolation

selecting data with confident pseudo labels

selecting data with closer data distribution

interpolation

Catalog

1

2

4

3

Multi-task Learning from Disjoint Datasets

Learning from Pseudo Labels

Selective Augmentation Method

Experiments

Comparison with baselines

Ablation Study

Visual Results

Hight weighs,correct thepseudo facialexpressionlabels

Low weighs,incorrect thepseudo facialexpressionlabels

IEEE International Conference on Multimedia and Expo6-10 July 2020 || London, United Kingdom || Virtual

Thanks for watching!

Yan Hong, Li Niu, Jianfu Zhang, Liqing Zhang

MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University

Beyond without Forgetting: Multi-Task Learning for Classification with Disjoint Datasets