CHAPTER 13CHAPTER 13

NATURAL LANGUAGE PROCESSING

Machine Translation

Information Extraction

Question Answering

Some Early NLP History

Why is Language Hard?• Ambiguity

Eye Drops Off Shelf Miners Refuse to Work After Death Killer Sentenced to Die for Second Time in 1

Years Iraqi Head Seeks Arms Ban on Nude Dancing on Governor’s Desk Juvenile Court to Try Shooting Defendant Stolen Painting Found by Tree Local HS Dropouts Cut in Half Hospitals Are Sued by 7 Foot Doctors

Models of Language

Language Modeling

Unigram ModelsUnigram Models

Bigram Models

Human Processing

Why is Language Hard?

Parsing as Search: Top-Down

Corpus-Based Methods

Semantic Interpretation• On to meaning!

A very basic approach to computational semantics Truth-theoretic notion of semantics (Tarskian) Assign a “meaning” to each word Word meanings combine according to the parse

structure People can and do spend entire courses on this

topic We’ll spend under an hour!

• What’s NLP and what’s general AI? Designing meaning representations? Computing those representations? Reasoning with them?

Problem: Ambiguities

• Headlines: Iraqi Head Seeks Arms Ban on Nude Dancing on Governor’s Desk Juvenile Court to Try Shooting Defendant Teacher Strikes Idle Kids Stolen Painting Found by Tree Kids Make Nutritious Snacks Local HS Dropouts Cut in Half Hospitals Are Sued by 7 Foot Doctors

• Why are these funny?

Machine Translation

Just a Code?

• “Also knowing nothing official about, but having guessed andinferred considerable about, the powerful new mechanizedmethods in cryptography—methods which I believe succeedeven when one does not know what language has beencoded—one naturally wonders if the problem of translationcould conceivably be treated as a problem in cryptography.When I look at an article in Russian, I say: ‘This is reallywritten in English, but it has been coded in some strangesymbols. I will now proceed to decode.’ ”

• Warren Weaver (1955:18, quoting a letter he wrote in 1947)

Levels of Transfer

Memory: Theory

Time: Theory

Problem: ScaleProblem: Scale

Problem: Sparsity