deep adversarial learning for nlp...conclusion •deep adversarial learning is a new, diverse, and...
TRANSCRIPT
![Page 1: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/1.jpg)
Deep Adversarial Learning for NLP
9:00 – 10:30 Introduction and Adversarial Training, GANs William Wang
10:30 – 11:00 Break -
11:00 – 12:15 Adversarial Examples Sameer Singh
12:15 – 12:30 Conclusions and Question Answering William Wang, Sameer Singh
William Wang Sameer Singh
With contributions from Jiwei Li1Slides: http://tiny.cc/adversarial
![Page 2: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/2.jpg)
Deep Adversarial Learning for NLP
William Wang Sameer Singh
With contributions from Jiwei Li.1Slides: http://tiny.cc/adversarial
![Page 3: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/3.jpg)
Agenda
• Introduction, Background, and GANs (William, 90 mins)
• Adversarial Examples and Rules (Sameer, 75 mins)
• Conclusion and Question Answering (Sameer and William, 15 mins)
2Slides: http://tiny.cc/adversarial
![Page 4: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/4.jpg)
Outline
• Background of the Tutorial
• Introduction: Adversarial Learning in NLP
• Adversarial Generation
• A Case Study of GANs in Dialogue Systems
3
![Page 5: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/5.jpg)
Rise of Adversarial Learning in NLP
• Through a simple ACL anthology search, we found that in 2018, there were 20+ times more papers mentioning “adversarial”, comparing to 2016.
• Meanwhile, the growth of all accepted papers is 1.39 times during this period.
• But if you went to CVPR 2018 in Salt Lake City, there were more than 100 papers on adversarial learning (approximately 1/3 of all adv. learning papers in NLP).
4
![Page 6: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/6.jpg)
Questions I’d like to Discuss
• What are the subareas of deep adversarial learning in NLP?
• How do we understand adversarial learning?
• What are some success stories?
• What are the pitfalls that we need to avoid?
5
![Page 7: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/7.jpg)
Opportunities in Adversarial Learning
• Adversarial learning is an interdisciplinary research area, and it is closely related to, but limited to the following fields of study:
• Machine Learning
• Computer Vision
• Natural Language Processing
• Computer Security
• Game Theory
• Economics
6
![Page 8: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/8.jpg)
Adversarial Attack in ML, Vision, & Security
• Goodfellow et al., (2015)
7
![Page 9: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/9.jpg)
Physical-World Adversarial Attack / Examples (Eykholt et al., CVPR 2018)
8
![Page 10: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/10.jpg)
Success of Adversarial Learning
CycleGAN (Zhu et al., 2017)
9
![Page 11: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/11.jpg)
Failure Cases
CycleGAN (Zhu et al., 2017)10
![Page 12: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/12.jpg)
Success of Adversarial Learning
GauGAN (Park et al., 2019)11
![Page 13: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/13.jpg)
Deep Adversarial Learning in NLP
• There were some successes of GANs in NLP, but not so much comparing to Vision.
• The scope of Deep Adversarial Learning in NLP includes:
• Adversarial Examples, Attacks, and Rules
• Adversarial Training (w. Noise)
• Adversarial Generation
• Various other usages in ranking, denoising, & domain adaptation.
12
![Page 14: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/14.jpg)
Outline
• Background of the Tutorial
• Introduction: Adversarial Learning in NLP
• Adversarial Generation
• A Case Study of GANs in Dialogue Systems
13
![Page 15: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/15.jpg)
Adversarial Examples
• One of the more popular areas of adversarial learning in NLP.
• E.g., Alzantot et al., EMNLP 2018
14
![Page 16: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/16.jpg)
Adversarial Attacks (Coavoux et al., EMNLP 2018)
15
The main classifier
predicts a label y from
a text x, the attacker
tries to recover some
private information z
contained in x from the
latent representation
used by the main
classifier.
![Page 17: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/17.jpg)
Adversarial Training
• Main idea: • Adding noise, randomness, or adversarial loss in optimization.
• Goal: make the trained model more robust.
16
![Page 18: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/18.jpg)
Adversarial Training: A Simple Example
• Adversarial Training for Relation Extraction• Wu, Bamman, Russell (EMNLP 2017).
• Task: Relation Classification.
• Interpretation: Regularization in the Feature Space.
17
![Page 19: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/19.jpg)
Adversarial Training for Relation Extraction
Wu, Bamman, Russell (EMNLP 2017).
18
![Page 20: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/20.jpg)
Adversarial Training for Relation Extraction
Wu, Bamman, Russell (EMNLP 2017).
19
![Page 21: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/21.jpg)
Outline
• Background of the Tutorial
• Introduction: Adversarial Learning in NLP
• Adversarial Generation
• A Case Study of GANs in Dialogue Systems
20
![Page 22: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/22.jpg)
GANs (Goodfellow et al., 2014)
• Two competing neural networks: generator & discriminator
the classifier trying to detect the fake sample
forger trying to produce some counterfeit material
Image: https://ishmaelbelghazi.github.io/ALI/21
![Page 23: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/23.jpg)
GAN Objective
22
D(x): the probability that x came from the data rather than generator
Goodfellow, et al., “Generative adversarial networks,” in NIPS, 2014.
D
G
![Page 24: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/24.jpg)
GAN Training Algorithm
23
Discriminator
Generator
Goodfellow, et al., “Generative adversarial networks,” in NIPS, 2014.
![Page 25: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/25.jpg)
GAN Equilibrium
• Global optimality
• Discriminator
• Generator
24Goodfellow, et al., “Generative adversarial networks,” in NIPS, 2014.
D
G
s.t.
![Page 26: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/26.jpg)
Major Issues of GANs • Mode Collapse (unable to produce diverse samples)
25
![Page 27: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/27.jpg)
Major Issues of GANs in NLP
• Often you need to pre-train the generator and discriminator w. MLE
• But how much?
• Unstable Adversarial Training• We are dealing with two networks / learners / agents
• Should we update them at the same rate?
• The discriminator might overpower the generator.
• With many possible combinations of model choice for generator and discriminator networks in NLP, it could be worse.
26
![Page 28: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/28.jpg)
Major Issues of GANs in NLP
• GANs were originally designed for images• You cannot back-propagate through the generated X
• Image is continuous, but text is discrete (DR-GAN, Tran et al., CVPR 2017).
27
![Page 29: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/29.jpg)
SeqGAN: policy gradient for generating sequences(Yu et al., 2017)
28
![Page 30: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/30.jpg)
Training Language GANs from Scratch
• New Google DeepMind arxiv paper (de Masson d’Autume et al., 2019)
• Claims no MLE pre-trainings are needed.
• Uses per time-stamp dense rewards.
• Yet to be peer-reviewed and tested.
29
![Page 31: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/31.jpg)
Why shouldn’t NLP give up on GAN?
• It’s unsupervised learning.
• Many potential applications of GANs in NLP.
• The discriminator is often learning a metric.
• It can also be interpreted as self-supervised learning (especially with dense rewards).
30
![Page 32: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/32.jpg)
Applications of Adversarial Learning in NLP• Social Media (Wang et al., 2018a; Carton et al., 2018)
• Contrastive Estimation (Cai and Wang, 2018; Bose et al., 2018)
• Domain Adaptation (Kim et al., 2017; Alam et al., 2018; Zou et al., 2018; Chen and Cardie, 2018; Tran and Nguyen, 2018; Cao et al., 2018; Li et al., 2018b)
• Data Cleaning (Elazar and Goldberg, 2018; Shah et al., 2018; Ryu et al., 2018; Zellers et al., 2018)
• Information extraction (Qin et al., 2018; Hong et al., 2018; Wang et al., 2018b; Shi et al., 2018a; Bekoulis et al., 2018)
• Information retrieval (Li and Cheng, 2018)
• Another 18 papers on Adversarial Learning at NAACL 2019!
31
![Page 33: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/33.jpg)
GANs for Machine Translation
• Yang et al., NAACL 2018
• Wu et al., ACML 2018
32
![Page 34: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/34.jpg)
SentiGAN (Wang and Wan, IJCAI 2018)Idea: use a mixture of generators and a multi-class discriminator.
33
![Page 35: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/35.jpg)
No Metrics Are Perfect: Adversarial Reward Learning (Wang, Chen et al., ACL 2018)
34
![Page 36: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/36.jpg)
AREL Storytelling Evaluation
• Dataset: VIST (Huang et al., 2016).
0%
10%
20%
30%
40%
50%
XE BLEU-RL CIDEr-RL GAN AREL
Turing Test
Win Unsure
-17.5 -13.7
-26.1
-6.3
35
![Page 37: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/37.jpg)
DSGAN: Adversarial Learning for Distant Supervision IE (Qin et al., ACL 2018)
36
![Page 38: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/38.jpg)
DSGAN: Adversarial Learning for Distant Supervision IE (Qin et al., ACL 2018)
37
![Page 39: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/39.jpg)
KBGAN: Learning to Generate High-Quality Negative Examples (Cai and Wang, NAACL 2018)
Idea: use adversarial learning to iteratively learn better negative examples.
38
![Page 40: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/40.jpg)
Outline
• Background of the Tutorial
• Introduction: Adversarial Learning in NLP
• Understanding Adversarial Learning
• Adversarial Generation
• A Case Study of GANs in Dialogue Systems
39
![Page 41: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/41.jpg)
What Should Rewards for Good Dialogue BeLike ?
40
![Page 42: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/42.jpg)
Turing Test
Reward for Good Dialogue
41
![Page 43: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/43.jpg)
How old are you ?
I don’t know what you aretalking about
I’m 25.
A human evaluator/ judge
Reward for Good Dialogue
42
Jl3
![Page 44: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/44.jpg)
How old are you ?
I don’t know what you aretalking about
I’m 25.
Reward for Good Dialogue
43
Jl3
![Page 45: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/45.jpg)
How old areyou ?
I don’t know what you aretalking about
I’m 25.
P= 90% human generated
P= 10% human generated
Reward for Good Dialogue
44
Jl3
![Page 46: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/46.jpg)
Adversarial Learning inImage Generation (Goodfellow et al., 2014)
45
Jl3Jl4
![Page 47: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/47.jpg)
Model BreakdownGenerative Model (G)
how are you ?
I’m fine . EOS
Encoding Decoding
eos I’m fine .
46
![Page 48: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/48.jpg)
Model BreakdownGenerative Model (G)
how are you ?
I’m fine . EOS
Encoding Decoding
eos I’m fine .
Discriminative Model (D)
how are you ? eos I’m fine .
P= 90% human generated
47
![Page 49: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/49.jpg)
Model BreakdownGenerative Model (G)
how are you ?
I’m fine . EOS
Encoding Decoding
eos I’m fine .
Discriminative Model (D)
how are you ? eos I’m fine .
Reward P= 90% human generated
48
![Page 50: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/50.jpg)
Policy Gradient
REINFORCE Algorithm (William,1992)
Generative Model (G)
how are you ?
I’m fine EOS
Encoding Decoding
eos I’m fine .
49
![Page 51: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/51.jpg)
Adversarial Learning for Neural Dialogue Generation
Update the Discriminator
Update the Generator
The discriminator forces the generator to produce correct responses
50
![Page 52: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/52.jpg)
Human Evaluation
The previous RL model only performbetter on multi-turn conversations
51
![Page 53: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/53.jpg)
Results: Adversarial Learning Improves Response Generation
Human Evaluator
vs a vanilla generation model
Adversarial Win
Adversarial Lose
Tie
62% 18% 20%
52
![Page 54: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/54.jpg)
Sample response
Tell me ... how long have you had this falling sickness ?
System Response
53
![Page 55: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/55.jpg)
Sample response
Tell me ... how long have you had this falling sickness ?
System Response
Vanilla-Seq2Seq I don’t know what you aretalking about.
54
![Page 56: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/56.jpg)
Sample response
Tell me ... how long have you had this falling sickness ?
System Response
Vanilla-Seq2Seq I don’t know what you aretalking about.
Mutual Information I’m not a doctor.
55
![Page 57: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/57.jpg)
Sample response
Tell me ... how long have you had this falling sickness ?
System Response
Vanilla-Seq2Seq I don’t know what you aretalking about.
Mutual Information I’m not a doctor.
Adversarial Learning A few months, I guess.
56
![Page 58: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/58.jpg)
Self-Supervised Learning meets Adversarial Learning
• Self-Supervised Dialog Learning (Wu et al., ACL 2019)
• Use of SSL to learn dialogue structure (sequence ordering).
57
![Page 59: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/59.jpg)
Self-Supervised Learning meets Adversarial Learning
• Self-Supervised Dialog Learning (Wu et al., ACL 2019)
• Use of SSN to learn dialogue structure (sequence ordering).
• REGS: Li et al., (2017) AEL: Xu et al., (2017)
58
![Page 60: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/60.jpg)
Conclusion
• Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP.
• GANs have obtained particular strong results in Vision, but yet there are both challenges and opportunities in GANs for NLP.
• In a case study, we show that adversarial learning for dialogue has obtained promising results.
• There are plenty of opportunities ahead of us with the current advances of representation learning, reinforcement learning, and self-supervised learning techniques in NLP.
59
![Page 62: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/62.jpg)
Thank you!
• Now we will take an 30 mins break.
61
![Page 63: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/63.jpg)
Adversarial Examples in NLP
Sameer [email protected]
@sameer_
sameersingh.org
Slides: http://tiny.cc/adversarial
![Page 64: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/64.jpg)
What are Adversarial Examples?
Sameer Singh, NAACL 2019 Tutorial 2
“panda”
57.7% confidence
“gibbon”
99.3% confidence
[Goodfellow et al, ICLR 2015 ]
![Page 65: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/65.jpg)
What’s going on?
Sameer Singh, NAACL 2019 Tutorial 3[Goodfellow et al, ICLR 2015 ]
Fast Gradient Sign Method
![Page 66: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/66.jpg)
Applications of Adversarial Attacks
• Security of ML Models• Should I deploy or not? What’s the worst that can happen?
• Evaluation of ML Models• Held-out test error is not enough
• Finding Bugs in ML Models• What kinds of “adversaries” might happen naturally?
• (Even without any bad actors)
• Interpretability of ML Models?• What does the model care about, and what does it ignore?
Sameer Singh, NAACL 2019 Tutorial 4
![Page 67: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/67.jpg)
Challenges in NLP
Sameer Singh, NAACL 2019 Tutorial 5
ChangeL2 is not really defined for textWhat is imperceivable? What is a small vs big change?What is the right way to measure this?
EffectClassification tasks fit in well, but …What about structured prediction? e.g. sequence labelingLanguage generation? e.g. MT or summarization
SearchText is discrete,
cannot use continuous optimizationHow do we search over sequences?
![Page 68: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/68.jpg)
Choices in Crafting AdversariesDifferent ways to address the challenges
Sameer Singh, NAACL 2019 Tutorial 6
![Page 69: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/69.jpg)
Choices in Crafting Adversaries
Sameer Singh, NAACL 2019 Tutorial 7
What is a small change?
What does it mean to misbehave?
How do we find the attack?
![Page 70: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/70.jpg)
Choices in Crafting Adversaries
Sameer Singh, NAACL 2019 Tutorial 8
What is a small change?
![Page 71: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/71.jpg)
Change: What is a small change?
Sameer Singh, NAACL 2019 Tutorial 9
CharactersPros:• Often easy to miss• Easier to search overCons:• Gibberish, nonsensical words• No useful for interpretability
WordsPros:• Always from vocabulary• Often easy to missCons:• Ungrammatical changes• Meaning also changes
Phrase/SentencePros:• Most natural/human-like• Test long-distance effectsCons:• Difficult to guarantee quality• Larger space to search
Main Challenge: Defining the distance between x and x’
![Page 72: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/72.jpg)
Change: A Character (or few)
Sameer Singh, NAACL 2019 Tutorial 10[ Ebrahimi et al, ACL 2018, COLING 2018 ]
x = [ ‘I’ ‘ ’ ‘l’ ‘o’ ‘v’ …
x' = [ ‘I’ ‘ ’ ‘l’ ‘i’ ‘v’ …
Edit Distance: Flip, Insert, Delete
x = [ “I love movies” ]
![Page 73: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/73.jpg)
Change: Word-level Changes
Sameer Singh, NAACL 2019 Tutorial 11
x = [ ‘I ’ ‘like’ ‘this’ ‘movie’ ‘ .’ ]
x' = [ ‘I ’ ‘really’ ‘this’ ‘movie’ ‘ .’ ]Word Embedding?
x' = [ ‘I ’ ‘eat’ ‘this’ ‘movie’ ‘ .’ ]Part of Speech?
x' = [ ‘I ’ ‘hate’ ‘this’ ‘movie’ ‘ .’ ]Language Model?
x' = [ ‘I ’ ‘lamp’ ‘this’ ‘movie’ ‘ .’ ]Random word?
Let’s replace this word
[ Alzantot et. al. EMNLP 2018 ]
[Jia and Liang, EMNLP 2017 ]
![Page 74: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/74.jpg)
Change: Paraphrasing via Backtranslation
Sameer Singh, NAACL 2019 Tutorial 12
This is a good movie
x
Este é um bom filme
c’est un bon film
Translate into multiple languages Use back-translators to score candidates
S(x, x’) ∝0.5 * P(x’ | Este é um bom filme) +0.5 * P(x’ | c’est un bon film)
This is a good movie This is a good movieS( , ) = 1
This is a good movie That is a good movieS( , ) = 0.95
S( , ) = 0This is a good movie Dogs like cats
x, x’ should mean the same thing (semantically-equivalent adversaries)
[Ribeiro et al ACL 2018]
![Page 75: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/75.jpg)
Change: Sentence Embeddings
• Deep representations are supposed to encode meaning in vectors• If (x-x’) is difficult to compute, maybe we can do (z-z’)?
Sameer Singh, NAACL 2019 Tutorial 13
D
Decoder(GAN)
Ez
Encoder
z'
x f y
x' f y'
[Zhao et al ICLR 2018]
![Page 76: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/76.jpg)
Choices in Crafting Adversaries
Sameer Singh, NAACL 2019 Tutorial 14
What is a small change?
![Page 77: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/77.jpg)
Choices in Crafting Adversaries
Sameer Singh, NAACL 2019 Tutorial 15
How do we find the attack?
![Page 78: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/78.jpg)
Search: How do we find the attack?
Sameer Singh, NAACL 2019 Tutorial 16
Only access predictions(usually unlimited queries)
Full access to the model(compute gradients)
Access probabilities
Create x’ and test whether the model misbehaves
Create x’ and test whether general direction is correct
Use the gradient to craft x’
Even this is often unrealistic
![Page 79: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/79.jpg)
Search: Gradient-based
Sameer Singh, NAACL 2019 Tutorial 17
𝛻𝐽𝑥
𝐽𝑥 Or whatever the misbehavior is
1. Compute the gradient2. Step in that direction (continuous)3. Find the nearest neighbor4. Repeat if necessary
Beam search over the above…
[ Ebrahimi et al, ACL 2018, COLING 2018 ]
![Page 80: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/80.jpg)
Search: Sampling
Sameer Singh, NAACL 2019 Tutorial 18
1. Generate local perturbations2. Select ones that looks good3. Repeat step 1 with these new ones4. Optional: beam search, genetic algo
[Zhao et al, ICLR 2018 ]
[ Alzantot et. al. EMNLP 2018 ]
[Jia and Liang, EMNLP 2017 ]
![Page 81: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/81.jpg)
Search: Enumeration (Trial/Error)
Sameer Singh, NAACL 2019 Tutorial 19
1. Make some perturbations2. See if they work3. Optional: pick the best one
[Belinkov, Bisk, ICLR 2018 ]
[Iyyer et al, NAACL 2018 ]
[Ribeiro et al, ACL 2018 ]
![Page 82: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/82.jpg)
Choices in Crafting Adversaries
Sameer Singh, NAACL 2019 Tutorial 20
How do we find the attack?
![Page 83: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/83.jpg)
Choices in Crafting Adversaries
Sameer Singh, NAACL 2019 Tutorial 21
What does it mean to misbehave?
![Page 84: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/84.jpg)
Effect: What does it mean to misbehave?
Sameer Singh, NAACL 2019 Tutorial 22
ClassificationUntargeted: any other class
Targeted: specific other class
Other TasksLoss-based: Maximize the loss on the example
e.g. perplexity/log-loss of the prediction
Property-based: Test whether a property holdse.g. MT: A certain word is not generated
NER: No PERSON appears in the output
¡No me ataques!MT: Don't attack me!
NER:
![Page 85: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/85.jpg)
Evaluation: Are the attacks “good”?
• Are they Effective?• Attack/Success rate
• Are the Changes Perceivable? (Human Evaluation)• Would it have the same label?
• Does it look natural?
• Does it mean the same thing?
• Do they help improve the model?• Accuracy after data augmentation
• Look at some examples!
Sameer Singh, NAACL 2019 Tutorial 23
![Page 86: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/86.jpg)
Review of the Choices
• Change• Character level• Word level• Phrase/Sentence level
• Effect• Targeted or Untargeted• Choose based on the task
• Search• Gradient-based• Sampling• Enumeration
• Evaluation
Sameer Singh, NAACL 2019 Tutorial 24
![Page 87: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/87.jpg)
Research HighlightsIn terms of the choices that were made
Sameer Singh, NAACL 2019 Tutorial 25
![Page 88: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/88.jpg)
Noise Breaks Machine Translation!
Change Search Tasks
Random Character Based Passive; add and test Machine Translation
Sameer Singh, NAACL 2019 Tutorial 26[Belinkov, Bisk, ICLR 2018 ]
![Page 89: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/89.jpg)
Hotflip
Sameer Singh, NAACL 2019 Tutorial 27
Change Search Tasks
Character-based(extension to words)
Gradient-based; beam-search Machine Translation, Classification, Sentiment
[ Ebrahimi et al, ACL 2018, COLING 2018 ]
News Classification
Machine Translation
![Page 90: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/90.jpg)
Search Using Genetic Algorithms
[ Alzantot et. al. EMNLP 2018 ] Sameer Singh, NAACL 2019 Tutorial 28
Change Search Tasks
Word-based,language model score
Genetic Algorithm Textual Entailment, Sentiment Analysis
Black-box, population-basedsearch of natural adversary
![Page 91: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/91.jpg)
Natural Adversaries
Sameer Singh, NAACL 2019 Tutorial 29[Zhao et al, ICLR 2018 ]
Change Search Tasks
Sentence, GAN embedding
Stochastic search Images, Entailment,Machine Translation
Textual Entailment
![Page 92: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/92.jpg)
Semantic AdversariesSemantically-Equivalent Adversary
(SEA)Semantically-Equivalent Adversarial Rules
(SEARs)
color → colour
xBacktranslation+ Enumeration
x’ (x, x’)Patternsin “diffs”
Rules
Sameer Singh, NAACL 2019 Tutorial 30[Ribeiro et al, ACL 2018 ]
Change Search Tasks
Sentence via Backtranslation
Enumeration VQA, SQuAD, Sentiment Analysis
![Page 93: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/93.jpg)
Transformation Rules: VisualQA
Sameer Singh, NAACL 2019 Tutorial 31[Ribeiro et al, ACL 2018 ]
![Page 94: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/94.jpg)
Transformation Rules: SQuAD
32Sameer Singh, NAACL 2019 Tutorial[Ribeiro et al, ACL 2018 ]
![Page 95: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/95.jpg)
Transformation Rules: Sentiment Analysis
Sameer Singh, NAACL 2019 Tutorial 33[Ribeiro et al, ACL 2018 ]
![Page 96: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/96.jpg)
Adding a Sentence
Sameer Singh, NAACL 2019 Tutorial 34[Jia, Liang, EMNLP 2017 ]
Change Search Tasks
Add a Sentence Domain knowledge, stochastic search
Question Answering
![Page 97: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/97.jpg)
Some Loosely Related WorkUse a broader notions of adversaries
Sameer Singh, NAACL 2019 Tutorial 35
![Page 98: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/98.jpg)
CRIAGE: Adversaries for Graph Embeddings
[ Pezeshkpour et. al. NAACL 2019 ] Sameer Singh, NAACL 2019 Tutorial 36
Which link should we add/remove,out of million possible links?
![Page 99: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/99.jpg)
“Should Not Change” / “Should Change”
Should Not Change
• like Adversarial Attacks
• Random Swap
• Stopword Dropout
• Paraphrasing
• Grammatical Mistakes
Should Change
• Overstability Test
• Add Negation
• Antonyms
• Randomize Inputs
• Change Entities
Sameer Singh, NAACL 2019 Tutorial 37[Niu, Bansal, CONLL 2018 ]
How do dialogue systems behave when the inputs are perturbed in specific ways?
![Page 100: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/100.jpg)
Overstability: Anchors
Sameer Singh, NAACL 2019 Tutorial 38
Anchor
Identify the conditions under which the classifier has the same prediction
[Ribeiro et al, AAAI 2018 ]
![Page 101: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/101.jpg)
Overstability: Input Reduction
Sameer Singh, NAACL 2019 Tutorial 39[Feng et al, EMNLP 2018 ]
Remove as much of the input as you canwithout changing the prediction!
![Page 102: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/102.jpg)
Adversarial Examples for NLP
Sameer Singh, NAACL 2019 Tutorial 40
• Imperceivable changes to the input• Unexpected behavior for the output• Applications: security, evaluation, debugging
Challenges for NLP• Effect: What is misbehavior?• Change: What is a small change?• Search: How do we find them?• Evaluation: How do we know it’s good?
![Page 103: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/103.jpg)
Sameer Singh, NAACL 2019 Tutorial 41
• More realistic threat models• Give even less access to the model/data
• Defenses and fixes• Spell-check based filtering
• Attack recognition: [Pruthi et al ACL 2019]
• Data augmentation
• Novel losses, e.g. [Zhang, Liang AISTATS 2019]
• Beyond sentences• Paragraphs, documents?
• Semantic equivalency → coherency across sentences
Future Directions
![Page 104: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/104.jpg)
References for Adversarial Examples in NLPRelevant Work (roughly chronological)
• Sentences to QA: [Jia and Liang, EMNLP 2017 ] link
• Noise Breaks MT: [ Belinkov, Bisk, ICLR 2018 ] link
• Natural Adversaries: [Zhao et al, ICLR 2018 ] link
• Syntactic Paraphrases: [Iyyer et al NAACL 2018] link
• Hotflip/Hotflip MT: [ Ebrahimi et al, ACL 2018, COLING 2018 ] link, link
Surveys
• Adversarial Attacks: [Zhang et al, arXiv 2019] link
• Analysis Methods: [ Belinkov, Glass, TAACL 2019 ] link
Sameer Singh, NAACL 2019 Tutorial 42
More Loosely Related Work• Anchors: [Ribeiro et al, AAAI 2018 ] link• Input Reduction: [Feng et al, EMNLP 2018 ] link• Graph Embeddings: [ Pezeshkpour et. al. NAACL ‘19 ] link
• SEARs: [Ribeiro et al, ACL 2018 ] link• Genetic Algo: [ Alzantot et. al. EMNLP 2018 ] link• Discrete Attacks: [Lei et al SysML 2019] link
![Page 105: Deep Adversarial Learning for NLP...Conclusion •Deep adversarial learning is a new, diverse, and inter-disciplinary research area, and it is highly related to many subareas in NLP](https://reader030.vdocuments.us/reader030/viewer/2022040922/5e9cfcec82de6e636a02aed8/html5/thumbnails/105.jpg)
Thank you!
Sameer [email protected]
@sameer_
Sameersingh.org
Work with Matt Gardner and me
as part of
The Allen Institute for Artificial Intelligence
in Irvine, CA
All levels: pre-docs, PhD interns, postdocs, and research scientists!