interpretability - university of colorado boulder
TRANSCRIPT
Interpretability
Advanced Machine Learning for NLPJordan Boyd-GraberNEED FOR INTERPRETABILITY
Slides adapted from Marco Tulio Ribeiro
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 1 of 1
Classification
• But representation is often less important (means to end)
• We really care about end result
• And not doing simple things like decision trees / linear classifier
• That’s why we’re making complicated algorithms
CanexplainthismessJ
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 2 of 1
Classification
• But representation is often less important (means to end)
• We really care about end result
• And not doing simple things like decision trees / linear classifier
• That’s why we’re making complicated algorithms
CanexplainthismessJ
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 2 of 1
LIME
16
Locally-faithfulsimpledecisionboundary
èGoodexplana:onforpredic:on
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. “Why Should ITrust You?” Explaining the Predictions of Any Classifier.LIME: Local Interpretable Model-Agnostic Explanations
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 3 of 1
What’s an Explanation
Whydidthishappen?HowdoIfixit?
Appearin21%oftrainingexamples,almostalwaysinatheism
Appearsin11%oftrainingexamples,alwaysinatheism
11
From: Keith Richards Subject: Christianity is the answer NTTP-Posting-Host: x.x.com I think Christianity is the one true religion. If you’d like to know more, send me a note
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 4 of 1
What’s an Explanation
P()=0.21P()=0.24P()=0.32
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 4 of 1
What makes good Explanation?
• Interpretable: Humans can Understand
• Faithful: Describes Model
• Model Agnostic: Generalize to Many Models
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 5 of 1
Method
• Sample points around xi
• Use complex model to predict labels for each sample• Weigh samples according to distance to xi
• Learn new simple model on weighted samples• Use simple model to explain
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 6 of 1
Perturbing an Example
x'(con(guoussuperpixels)x(3colorchannels/pixel)
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 7 of 1
Perturbing an Example
x(embeddings)
0.5 0.3 1.3 4.4 1.1 ...
x'(words)
Thisisahorriblemovie.
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 7 of 1
Image Example
PerturbedInstances P(Labrador)
OriginalImage
20
0.92
0.001
0.34
P(labrador)=0.21
Locallyweightedregression
ExplanaDon
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 8 of 1
Is this a good Classifier?
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 9 of 1
Is this a good Classifier?
0102030405060708090
100
Didn'ttrustthemodel
"Snowinsight"
%ofsub
jects(ou
tof2
7)
BeforeexplanaIons AKerexplanaIons
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 9 of 1
Improving ML Algorithms
20newsgroups-Train
20newsgroups-test
Hiddenreligiondataset
train predict
Explain
REPEAT
Evaluate
Turkersdon’tknowaboutthisdataset
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 10 of 1
Improving ML Algorithms
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 10 of 1
Improving ML Algorithms
0,5
0,55
0,6
0,65
0,7
0,75
0,8
Nouserinput 1round 2rounds 3rounds
Acc
urac
y on
hid
den
set
Trainon20newsgroups
Trainonhand-cleaned20newsgroups
Trainon20newsgroupsturkerscleandata
Advanced Machine Learning for NLP | Boyd-Graber Interpretability | 10 of 1