learning data manipulation for augmentation and...
TRANSCRIPT
![Page 1: Learning Data Manipulation for Augmentation and Weightingzhitingh/data/neurips19data-manip.pdfLearning Data Manipulation for Augmentation and Weighting ZhitingHu*, Bowen Tan*, Ruslan](https://reader034.vdocuments.us/reader034/viewer/2022042022/5e79396e7a8e680b93232ac6/html5/thumbnails/1.jpg)
Learning Data Manipulation for Augmentation and Weighting
Zhiting Hu*, Bowen Tan*, Ruslan Salakhutdinov, Tom Mitchell, Eric XingCarnegie Mellon University, Petuum Inc.
![Page 2: Learning Data Manipulation for Augmentation and Weightingzhitingh/data/neurips19data-manip.pdfLearning Data Manipulation for Augmentation and Weighting ZhitingHu*, Bowen Tan*, Ruslan](https://reader034.vdocuments.us/reader034/viewer/2022042022/5e79396e7a8e680b93232ac6/html5/thumbnails/2.jpg)
Learning Data Manipulation• Data manipulation is often a crucial step to improve performance
• Data augmentation for small data problems
• Rule-based: image rotation, cropping; text synonym, …• Learning-based [Ratner et al.,17; Cubuk et al.,19]
• Data weighting for class-imbalance problems
• Rule-based: inverse class frequency, …• Learning-based: meta-learning [Ren et al., 18]; self-paced learning [Jiang et al.,18]
! = #$% & = &′ =augment
Specific to one manipulation type
( = 1 ( = 10
![Page 3: Learning Data Manipulation for Augmentation and Weightingzhitingh/data/neurips19data-manip.pdfLearning Data Manipulation for Augmentation and Weighting ZhitingHu*, Bowen Tan*, Ruslan](https://reader034.vdocuments.us/reader034/viewer/2022042022/5e79396e7a8e680b93232ac6/html5/thumbnails/3.jpg)
Learning Data Manipulation• Data manipulation is often a crucial step to improve performance
• Data augmentation for small data problems
• Rule-based: image rotation, cropping; text synonym, …• Learning-based [Ratner et al.,17; Cubuk et al.,19]
• Data weighting for class-imbalance problems
• Rule-based: inverse class frequency, …• Learning-based: meta-learning [Ren et al., 18]; self-paced learning [Jiang et al.,18]
! = #$% & = &′ =augment
Specific to one manipulation type
( = 1 ( = 10
Can we have a more generic learning method?
![Page 4: Learning Data Manipulation for Augmentation and Weightingzhitingh/data/neurips19data-manip.pdfLearning Data Manipulation for Augmentation and Weighting ZhitingHu*, Bowen Tan*, Ruslan](https://reader034.vdocuments.us/reader034/viewer/2022042022/5e79396e7a8e680b93232ac6/html5/thumbnails/4.jpg)
Background
Connecting the Dots between MLE and RL [Tan et al., 2019]
![Page 5: Learning Data Manipulation for Augmentation and Weightingzhitingh/data/neurips19data-manip.pdfLearning Data Manipulation for Augmentation and Weighting ZhitingHu*, Bowen Tan*, Ruslan](https://reader034.vdocuments.us/reader034/viewer/2022042022/5e79396e7a8e680b93232ac6/html5/thumbnails/5.jpg)
Parameterize Manipulation
• Standard MLE
[Hu et al., NeurIPS2019]
ℒ"#$,&# ', ( = *+(-,.) 01(-, .|3) − 5 KL ' -, . || 8 - 89 . - + ;(')
01 -, . 3 = <1 if -, . ∈ 3−∞ otherwise
Get valid rewards only when samples match training data
![Page 6: Learning Data Manipulation for Augmentation and Weightingzhitingh/data/neurips19data-manip.pdfLearning Data Manipulation for Augmentation and Weighting ZhitingHu*, Bowen Tan*, Ruslan](https://reader034.vdocuments.us/reader034/viewer/2022042022/5e79396e7a8e680b93232ac6/html5/thumbnails/6.jpg)
Parameterize Manipulation
• Standard MLE
[Hu et al., NeurIPS2019]
ℒ"#$,&# ', ( = *+(-,.) 01(-, .|3) − 5 KL ' -, . || 8 - 89 . - + ;(')
01 -, . 3 = <1 if -, . ∈ 3−∞ otherwise
Get valid rewards only when samples match training data
Relax it
![Page 7: Learning Data Manipulation for Augmentation and Weightingzhitingh/data/neurips19data-manip.pdfLearning Data Manipulation for Augmentation and Weighting ZhitingHu*, Bowen Tan*, Ruslan](https://reader034.vdocuments.us/reader034/viewer/2022042022/5e79396e7a8e680b93232ac6/html5/thumbnails/7.jpg)
Parameterize Manipulation
• Standard MLE
• Data augmentation
• Data weighting
[Hu et al., NeurIPS2019]
ℒ"#$,&# ', ( = *+(-,.) 01(-, .|3) − 5 KL ' -, . || 8 - 89 . - + ;(')
01 -, . 3 = <1 if -, . ∈ 3−∞ otherwise
Get valid rewards only when samples match training data
Relax it
0IJKL
-, . 3 = <1 if - ∼ NI - -∗, . -∗, . ∈ 3
−∞ otherwise
0IP -, . 3 = <
QR if -, . = -S, .S -S, .S ∈ 3−∞ otherwise
Any label-preserving transformations
Weight of the T-thtraining example
![Page 8: Learning Data Manipulation for Augmentation and Weightingzhitingh/data/neurips19data-manip.pdfLearning Data Manipulation for Augmentation and Weighting ZhitingHu*, Bowen Tan*, Ruslan](https://reader034.vdocuments.us/reader034/viewer/2022042022/5e79396e7a8e680b93232ac6/html5/thumbnails/8.jpg)
Use a Reward Learning Algorithm
• Intrinsic reward learning [Zheng et al., 18]
• Augment extrinsic reward for better performance
• Update model:
• Update reward:
• Map to data manipulation
• Update model:
• Update data:
[Hu et al., NeurIPS2019]
!"#$%& ', ) = !"# ', ) + !,%&(', ))/0 = / + 1∇3ℒ"#$%&(/,5)
50 = 5 + 1∇,ℒ"#(/′(5))
Train model with augmented reward
New model /′ gets higher extrinsic reward
![Page 9: Learning Data Manipulation for Augmentation and Weightingzhitingh/data/neurips19data-manip.pdfLearning Data Manipulation for Augmentation and Weighting ZhitingHu*, Bowen Tan*, Ruslan](https://reader034.vdocuments.us/reader034/viewer/2022042022/5e79396e7a8e680b93232ac6/html5/thumbnails/9.jpg)
Use a Reward Learning Algorithm
• Intrinsic reward learning [Zheng et al., 18]
• Augment extrinsic reward for better performance
• Update model:
• Update reward:
• Map to data manipulation
• Update model:
• Update data:
[Hu et al., NeurIPS2019]
!"#$%& ', ) = !"# ', ) + !,%&(', ))/0 = / + 1∇3ℒ"#$%&(/,5)
50 = 5 + 1∇,ℒ"#(/′(5))
Train model with augmented reward
New model /′ gets higher extrinsic reward
/0 = / + 1∇3ℒ78&%9(/,5)
50 = 5 + 1∇,ℒ:(/′(5))
Train model on manipulated data
New model /′ gets better valid-set perf.
![Page 10: Learning Data Manipulation for Augmentation and Weightingzhitingh/data/neurips19data-manip.pdfLearning Data Manipulation for Augmentation and Weighting ZhitingHu*, Bowen Tan*, Ruslan](https://reader034.vdocuments.us/reader034/viewer/2022042022/5e79396e7a8e680b93232ac6/html5/thumbnails/10.jpg)
Parameterization of Text Data Augmentation
Food was good , but the service was very disappointing . ! =neg# =Raw data
Random mask # = Food [Mask] [Mask] , but the service was very disappointing .
Augment data # = Food tasted nice , but the service was very disappointing .
Finetune BERT to condition on the label$
[Hu et al., NeurIPS2019]
![Page 11: Learning Data Manipulation for Augmentation and Weightingzhitingh/data/neurips19data-manip.pdfLearning Data Manipulation for Augmentation and Weighting ZhitingHu*, Bowen Tan*, Ruslan](https://reader034.vdocuments.us/reader034/viewer/2022042022/5e79396e7a8e680b93232ac6/html5/thumbnails/11.jpg)
Low-Data Text Classification
[Hu et al., NeurIPS2019]
30
31
32
33
34
35
36
37
38
SST-5 IMDB
Accuracy
Methods
Base
Synonym
Fixed
Ours
60
61
62
63
64
65
66
IMDB
Acc
urac
y
Methods
Base: BERT Synonym Fixed Ours
85
86
87
88
89
90
TREC
Acc
urac
y
Methods
Base: BERT Synonym Fixed Ours
85
86
87
88
89
90
TREC
Acc
urac
y
Methods
Base: BERT Synonym Fixed Ours
Train size: 40 per class; Val size: 2
Use a label-conditional BERT as the augmentation model !" # #∗, &
![Page 12: Learning Data Manipulation for Augmentation and Weightingzhitingh/data/neurips19data-manip.pdfLearning Data Manipulation for Augmentation and Weighting ZhitingHu*, Bowen Tan*, Ruslan](https://reader034.vdocuments.us/reader034/viewer/2022042022/5e79396e7a8e680b93232ac6/html5/thumbnails/12.jpg)
70
74
78
82
86
20:1000 50:1000 100:1000
Acc
urac
y
Imbalance ratio
Base: ResNet
Ours
Ren
Proportion
Class-Imbalance Image ClassificationCIFAR10 binary image classification
[Hu et al., NeurIPS2019]
(weighting)