bias in nlp systemsjcheung/teaching/fall-2017/...tolga bolukbasi, kai-wei chang, james zou,...
TRANSCRIPT
![Page 1: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/1.jpg)
Bias in NLP Systems
COMP-550
Nov 30, 2017
![Page 2: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/2.jpg)
Outline A4 reading discussions
Bias in NLP systems
Recap
2
![Page 3: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/3.jpg)
A4 Reading Discussion Any clarification questions?
How does this method relate to the work we discussed in class?
What are the strengths of the approach? Limitations?
Is it a good idea to replace parts of the model with a neural network?
• If so, which parts?
3
![Page 4: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/4.jpg)
NLP in the Real World NLP and AI systems are increasingly used to automate fact finding and decision making
• Information retrieval
• Image captioning
• Automated essay grading
• School admissions decisions
• Resume and CV filtering
• Loan and insurance approval
Want to make sure process and decisions are fair and unbiased!
4
![Page 5: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/5.jpg)
Technological Fairness? Hope:
• Use objective measures and statistical techniques to produce a fairer system, free of human biases
Reality:
• Machine learning systems can learn the biases that are inherent in the data
• Even worse: the learned methods can produce results that are more biased than the training data!
• How can this be?
5
![Page 6: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/6.jpg)
Bias in Word Embedding Models word2vec exhibits bias!
This is okay:
man – woman ≈ king – queen
But this is NOT, and also found by word2vec!
man – woman ≈ computer programmer – homemaker
(Bolukbasi et al., 2016)
6
![Page 7: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/7.jpg)
Most Gender-Biased Occupations
(Bolukbasi et al., 2016)
7
![Page 8: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/8.jpg)
Implications of Word Association Bias Above results due to counting of word associations
Maybe this just reflects the bias in the underlying distribution of real life – why is that so bad?
Scenario: information retrieval; search result
• Must produce a ranking of, say, people's home pages to show in a search query.
• e.g. "cmu computer science phd student"
• Given two otherwise identical webpages, an algorithm may pick a website with a man's name (e.g., John) over one with a woman's name (e.g., Mary), because the former is more distributionally similar to computer science!
8
![Page 9: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/9.jpg)
Visual Semantic Role Labelling
imSitu data set (Yatskar et al., 2016)
9
![Page 10: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/10.jpg)
Bias Amplification in Trained Models
Result from (Zhao et al., 2017)
10
![Page 11: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/11.jpg)
Why does Bias Amplification Occur? Training data exhibits some bias
An automatic system is asked to produce a decision under uncertainty
• Ranking websites
• Labelling image as involving male or female participant
With standard loss/evaluation procedures, rational to favour more frequent class, if other information does not disambiguate
11
![Page 12: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/12.jpg)
Debiasing Algorithms General technique:
1. Identify axis or axes of bias (e.g., gender, race, religion, etc.)
2. Modify our learning or inference by adding constraints, such that the biased outcomes (as previously identified) are disfavoured
Let's consider the method of Zhao et al., (2017)
12
![Page 13: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/13.jpg)
Debiasing Activity Recognition Original inference problem:
argmax𝑦∈𝑌 𝑓𝜃 𝑦, 𝑖
• i.e., make the decision 𝑦 (e.g., 𝑦 = {woman, meat, stove, … }) that maximizes the score on test instance 𝑖
Idea: for each activity 𝑣∗ to debias, add a constraint:
13
![Page 14: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/14.jpg)
New Optimization Problem
• where {𝑌𝑖} represents the space of all possible label assignments to all test instances
• constraints are taken from equation (2) for each activity
This is expensive to solve exactly; use an approximate method based on Lagrange multipliers
14
![Page 15: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/15.jpg)
Performance
Reduced bias amplification without much loss in classification performance!
15
![Page 16: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/16.jpg)
Summary of Current Work Bias is a problem in NLP systems
Naïve methods can exacerbate problem
Possible to reduce effect of biases without sacrificing task performance
16
![Page 17: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/17.jpg)
References Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings. NIPS 2016.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, Kai-Wei Chang. Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints. EMNLP 2017.
17
![Page 18: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/18.jpg)
Recap of Course What have we done in COMP-550?
18
![Page 19: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/19.jpg)
19
Computational Linguistics (CL) Modelling natural language with computational models and techniques
Domains of natural language
Acoustic signals, phonemes, words, syntax, semantics, …
Speech vs. text
Natural language understanding (or comprehension) vs. natural language generation (or production)
![Page 20: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/20.jpg)
20
Computational Linguistics (CL) Modelling natural language with computational models and techniques
Goals
Language technology applications
Scientific understanding of how language works
![Page 21: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/21.jpg)
21
Computational Linguistics (CL) Modelling natural language with computational models and techniques
Methodology and techniques
Gathering data: language resources
Evaluation
Statistical methods and machine learning
Rule-based methods
![Page 22: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/22.jpg)
Current Trends and Challenges Speculations about the future of NLP
22
![Page 23: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/23.jpg)
Better Use of More Data Large amounts of data now available
• Unlabelled
• Noisy
• May not be directly relevant to your specific problem
How do we make better use of it?
• Unsupervised or lightly supervised methods
• Prediction models that can make use of data to learn what features are important (neural networks)
• Incorporate linguistic insights with large-scale data processing
23
![Page 24: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/24.jpg)
Using More Sources of Knowledge Old set up:
Better model?
24
Annotated data set Model predictions Feature extraction + Simple supervised
learning
Background text General knowledge bases Domain-specific constraints Directly relevant annotated data
Model predictions
![Page 25: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/25.jpg)
Away From Discreteness Discreteness is sometimes convenient assumption, but also a problem
• Words, phrases, sentences and labels for them
• Symbolic representations of semantics
• Motivated a lot of work in regularization and smoothing
Representation learning
• Learn continuous-valued representations using co-occurrence statistics, or some other objective function
• e.g., vector-space semantics
25
![Page 26: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/26.jpg)
Continuous-Valued Representations cat, linguistics, NP, VP
Advantages:
• Implicitly deal with smoothness, soft boundaries
• Incorporate many sources of information in training vectors
Challenges:
• What should a good continuous representation look like?
• Evaluation is often still in terms of a discrete set of labels
26
![Page 27: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/27.jpg)
Broadening Horizons We are getting better at solving specific problems on specific benchmark data sets.
• e.g., On WSJ corpus, POS tagging performance of >97% matches human-level performance.
Much more difficult and interesting:
• Working across multiple kinds of text and data sets
• Integrating disparate theories, domains, and tasks
27
![Page 28: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/28.jpg)
Connections to Other Fields Cognitive science and psycholinguistics
• e.g., model L1 and L2 acquisition; other human behaviour based on computational models
Human computer interaction and information visualization
• That’s nice that you have a tagger/parser/summarizer/ASR system/NLG module. Now, what do you do with it?
• Multi-modal systems and visualizations
28
![Page 29: Bias in NLP Systemsjcheung/teaching/fall-2017/...Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, Adam Kalai. Man is to Computer Programmer as Woman is to Homemaker?](https://reader036.vdocuments.us/reader036/viewer/2022081404/5f04ed4a7e708231d410664f/html5/thumbnails/29.jpg)
That’s It!
Good luck on your projects and finals!
29