![Page 1: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/1.jpg)
SI485i : NLP
Missing Topics and the Future
![Page 2: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/2.jpg)
Who cares about NLP?
• NLP has expanded quickly
• Most top-tier universities now have NLP faculty (Stanford,
Cornell, Berkeley, MIT, UPenn, CMU, Hopkins, etc)
• Commercial NLP hiring: Google, Microsoft, IBM,
Amazon, LinkedIn, Yahoo
• Web startups in Silicon Valley are eating up NLP
students
• Navy, DoD, NSA, NIH: all funding NLP research
2
![Page 3: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/3.jpg)
What NLP topics did we miss?
• Speech Recognition
3
![Page 4: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/4.jpg)
What NLP topics did we miss?
• Speech Recognition
4
![Page 5: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/5.jpg)
What NLP topics did we miss?
• Machine Translation
5
![Page 6: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/6.jpg)
What NLP topics did we miss?
• Machine Translation
6
Start at ~6min in.
http://www.youtube.com/watch?feature=player_embedded&v=Nu
-nlQqFCKg
![Page 7: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/7.jpg)
What NLP topics did we miss?
• Machine Translation
• IBM Models (1 through 5)
7
![Page 8: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/8.jpg)
Machine Translation
• How to model translations?
• Words: P( casa | house )
• Spurious words: P( a | null )
• Fertility: Pn( 1 | house )
• English word translates to one Spanish word
• Distortion: Pd( 5 | 2 )
• The 2nd English word maps to the 5th Spanish word
![Page 9: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/9.jpg)
Distortion
• Encourage translations to follow the diagonal…
• P( 4 | 4 ) * P( 5 | 5 ) * …
![Page 10: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/10.jpg)
Learning Translations
• Huge corpus of “aligned sentences”.
• Europarl
• Corpus of European Parliamant proceedings
• The EU is mandated to translate into all 21 official languages
• 21 languages, (semi-) aligned to each other
• P( casa | house ) = (count all casa/house pairs!)
• Pd( 2 | 5 ) = (count all sentences where 2nd word
went to 5th word)
![Page 11: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/11.jpg)
Machine Translation Technology
• Hand-held devices for military
• Speak english -> recognition -> translation -> generate Urdu
• Translate web documents
• Education technology?
• Doesn’t yet receive much of a focus
![Page 12: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/12.jpg)
What NLP topics did we miss?
• Dialogue Systems
12
Do you think
Anakin likes me? I don’t care.
![Page 13: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/13.jpg)
What NLP topics did we miss?
• Dialogue Systems
• Why? Heavy interest in human-robot communication.
• UAVs require teams of 5+ people for each operating
machine • Goal: reduce the number of people
• Give computer high-level dialogue commands, rather than low-level
system commands
13
![Page 14: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/14.jpg)
What NLP topics did we miss?
• Dialogue Systems
• Dialogue is a fascinating topic. Not only do we need
to understand language, but now discourse cues: • Questions require replies
• Imperatives/Commands
• Acknowledgments: “ok”
• Back-channels: “uh huh”, “mm hmm”
• Belief-Desire-Intention (BDI) Model
• Beliefs: you maintain a set of facts about the world
• Desires: things you want to become true in the world
• Intentions: desires that you are taking action on
14
![Page 15: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/15.jpg)
What NLP topics did we miss?
• Unsupervised Learning
15
![Page 16: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/16.jpg)
What NLP topics did we miss?
• Unsupervised Learning
• Most of this semester used data that had human/gold
labels.
• Bootstrapping was our main counter-example: it is mostly
unsupervised.
• Many many algorithms being researched to learn
language and knowledge without humans, only using
text.
16
![Page 17: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/17.jpg)
El Fin
• Secret 1:
17
![Page 18: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/18.jpg)
El Fin
• Secret 1:
• I intentionally made our labs confusing
18
![Page 19: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/19.jpg)
El Fin
• Secret 1:
• I intentionally made our labs confusing
Under-defined tasks with unclear expected results
19
![Page 20: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/20.jpg)
El Fin
• Secret 1:
• I intentionally made our labs confusing
Under-defined tasks with unclear expected results
• Secret 2:
20
![Page 21: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/21.jpg)
El Fin
• Secret 1:
• I intentionally made our labs confusing
Under-defined tasks with unclear expected results
• Secret 2:
• I tried to teach you skills that have nothing to do with NLP
21
![Page 22: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/22.jpg)
El Fin
• Secret 1:
• I intentionally made our labs confusing
Under-defined tasks with unclear expected results
• Secret 2:
• I tried to teach you skills that have nothing to do with NLP
Experimentation
Error Analysis
22
![Page 23: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/23.jpg)
El Fin
• Secret 1:
• I intentionally made our labs confusing
Under-defined tasks with unclear expected results
• Secret 2:
• I tried to teach you skills that have nothing to do with NLP
Experimentation
Error Analysis
• Secret 3:
23
![Page 24: SI485i : NLP › Users › cs › nchamber › courses › nlp › ...• Dialogue is a fascinating topic. Not only do we need to understand language, but now discourse cues: •Questions](https://reader033.vdocuments.us/reader033/viewer/2022042323/5f0e0cf57e708231d43d5f7d/html5/thumbnails/24.jpg)
El Fin
• Secret 1:
• I intentionally made our labs confusing
Under-defined tasks with unclear expected results
• Secret 2:
• I tried to teach you skills that have nothing to do with NLP
Experimentation
Error Analysis
• Secret 3:
• I appreciate the hard work you put into the class
24