helping computers find meaning they lost in translation

Helping ComputersFind Meaning They

Lost in Translation

Katie Kuksenok UW Science Now!University of Washington March 8, 2012

Hi! I’m Katie!Computer Science and Engineering

PhD student

How people interact with computers

Richness of written language

PosterityTranquility

Impeachment

liable Indictment

Concurrence

Google SearchesBy Language

Google SearchesBy Language

Machine Translation

Source language

Machine Translation

-------------------------------------------------------------------------------------

---------

translation

Source language Target language

Machine Translation+ Interaction

-------------------------------------------------------------------------------------

---------

translation reading

Source language Target language

Machine Translation+ Interaction

-------------------------------------------------------------------------------------

--------- editing

translation reading

Source language Target language

Machine Translation+ Interaction

-------------------------------------------------------------------------------------

--------- editing

translation reading

Source language Target language

The Translating Machine

The Translating MachineFirst?

Artificial Intelligence…where to begin?

What would a person do?

How can we describe that to a computer?

Rules & Dictionaries

?

Did it work?

Did it work?The spirit is willing, but the flesh is weak

Did it work?The spirit is willing, but the flesh is weak

(Russian)

Did it work?The spirit is willing, but the flesh is weak

The vodka is good, but the meat is rotten

(Russian)

Rules & Dictionaries

Rosetta Stone

?

Translate this:Source farok crrrok hihok yorok clok kantok yurpTarget ____ ____ _____ _____ ____ ______ ___

Translate this:Source farok crrrok hihok yorok clok kantok yurpTarget ____ ____ _____ _____ ____ ______ ___

Source wiwok farok izok stokTarget sus clientes estan enfadados

Source lalok farok ororok lalok sprok izok enemok Target los clientes y los asociados son enemigos

Based on 2 parallel pairs:

Source wiwok farok izok stokTarget sus clientes estan enfadados

Source lalok farok ororok lalok sprok izok enemok Target los clientes y los asociados son enemigos

Translate this:Source farok crrrok hihok yorok clok kantok yurpTarget ____ ____ _____ _____ ____ ______ ___

Based on 2 parallel pairs:

Source wiwok farok izok stokTarget sus clientes estan enfadados

Source lalok farok ororok lalok sprok izok enemok Target los clientes y los asociados son enemigos

Translate this:Source farok crrrok hihok yorok clok kantok yurpTarget ____ ____ _____ _____ ____ ______ ___

Based on 2 parallel pairs:

Source wiwok farok izok stokTarget sus clientes estan enfadados

Source lalok farok ororok lalok sprok izok enemok Target los clientes y los asociados son enemigos

Translate this:Source farok crrrok hihok yorok clok kantok yurpTarget ____ ____ _____ _____ ____ ______ ___

Based on 2 parallel pairs:

clientes

Machine Learning

Statistics + Data =Surprisingly Good Results

More Data + Better Data = Better Results

DATA

Statistical Machine Translationstate of the art

1954 1966 1991 2012

PosterityTranquility

Impeachment

liable Indictment

Concurrence

How do peoplelearn different languages?

By Instruction? By Example?

By Failing

By FailingTogether

Interactive Machine Translation

?

Interactive Machine Translation

HumanComputation

Human Computationsource Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.

Human Computationsource Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.

reorder [vixen] Saw a [vixen] he hungry crow been in the fig tree, and asked you why did.reorder [a] vixen Saw [a] the hungry crow been in the fig tree, and asked you why did.retype {A} vixen Saw the hungry crow been in the fig tree, and asked you why did.retype A vixen {saw} the hungry crow been in the fig tree, and asked you why did.

Human Computationsource Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.

reorder [vixen] Saw a [vixen] he hungry crow been in the fig tree, and asked you why did.reorder [a] vixen Saw [a] the hungry crow been in the fig tree, and asked you why did.retype {A} vixen Saw the hungry crow been in the fig tree, and asked you why did.retype A vixen {saw} the hungry crow been in the fig tree, and asked you why did.delete A vixen saw the hungry crow [been] in the fig tree, and asked you why did.delete A vixen saw the hungry crow in the fig tree, and asked [you] why did.

replace A vixen saw the hungry crow in the fig tree, and asked [what was he doing] did.

Human Computationsource Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.

reorder [vixen] Saw a [vixen] he hungry crow been in the fig tree, and asked you why did.reorder [a] vixen Saw [a] the hungry crow been in the fig tree, and asked you why did.retype {A} vixen Saw the hungry crow been in the fig tree, and asked you why did.retype A vixen {saw} the hungry crow been in the fig tree, and asked you why did.delete A vixen saw the hungry crow [been] in the fig tree, and asked you why did.delete A vixen saw the hungry crow in the fig tree, and asked [you] why did.

replace A vixen saw the hungry crow in the fig tree, and asked [what was he doing] did.retype A vixen saw the hungry crow in the fig tree, and asked {why} was he doing did.

reorder A vixen saw the hungry crow in the fig tree, and asked why [he] was [he] doing did.retype A vixen saw the hungry crow in the fig tree, and asked why he was {there} did.delete A vixen saw the hungry crow in the fig tree, and asked why he was there [did].

final A vixen saw the hungry crow in the fig tree, and asked why he was there.

Human Computationsource Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.

initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.

reorder [vixen] Saw a [vixen] he hungry crow been in the fig tree, and asked you why did.

[a] vixen Saw [a] the hungry crow been in the fig tree, and asked you why did.

retype {A} vixen Saw the hungry crow been in the fig tree, and asked you why did.

A vixen {saw} the hungry crow been in the fig tree, and asked you why did.

delete A vixen saw the hungry crow [been] in the fig tree, and asked you why did.

A vixen saw the hungry crow in the fig tree, and asked [you] why did.

replace A vixen saw the hungry crow in the fig tree, and asked [what was he doing] did.

retype A vixen saw the hungry crow in the fig tree, and asked {why} was he doing did.

reorder A vixen saw the hungry crow in the fig tree, and asked why [he] was [he] doing did.

retype A vixen saw the hungry crow in the fig tree, and asked why he was {there} did.

delete A vixen saw the hungry crow in the fig tree, and asked why he was there [did].

final A vixen saw the hungry crow in the fig tree, and asked why he was there.

x 100

Human Computation

x 100…= Human Insight as Data

source Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.

initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.

reorder [vixen] Saw a [vixen] he hungry crow been in the fig tree, and asked you why did.

[a] vixen Saw [a] the hungry crow been in the fig tree, and asked you why did.

retype {A} vixen Saw the hungry crow been in the fig tree, and asked you why did.

A vixen {saw} the hungry crow been in the fig tree, and asked you why did.

delete A vixen saw the hungry crow [been] in the fig tree, and asked you why did.

A vixen saw the hungry crow in the fig tree, and asked [you] why did.

replace A vixen saw the hungry crow in the fig tree, and asked [what was he doing] did.

retype A vixen saw the hungry crow in the fig tree, and asked {why} was he doing did.

reorder A vixen saw the hungry crow in the fig tree, and asked why [he] was [he] doing did.

retype A vixen saw the hungry crow in the fig tree, and asked why he was {there} did.

delete A vixen saw the hungry crow in the fig tree, and asked why he was there [did].

final A vixen saw the hungry crow in the fig tree, and asked why he was there.

DATA

DATA Human Insight as Data

Human Computation + Translation

How can we capture the editing process ?

source Vio una zorra al hambriento cuervo eternizado en la higuera, y le preguntó qué hacía.

initial Saw a vixen the hungry crow been in the fig tree, and asked you why did.

reorder [vixen] Saw a [vixen] he hungry crow been in the fig tree, and asked you why did.

[a] vixen Saw [a] the hungry crow been in the fig tree, and asked you why did.

retype {A} vixen Saw the hungry crow been in the fig tree, and asked you why did.

A vixen {saw} the hungry crow been in the fig tree, and asked you why did.

delete A vixen saw the hungry crow [been] in the fig tree, and asked you why did.

A vixen saw the hungry crow in the fig tree, and asked [you] why did.

replace A vixen saw the hungry crow in the fig tree, and asked [what was he doing] did.

retype A vixen saw the hungry crow in the fig tree, and asked {why} was he doing did.

reorder A vixen saw the hungry crow in the fig tree, and asked why [he] was [he] doing did.

retype A vixen saw the hungry crow in the fig tree, and asked why he was {there} did.

delete A vixen saw the hungry crow in the fig tree, and asked why he was there [did].

final A vixen saw the hungry crow in the fig tree, and asked why he was there.

InteractionDrag and drop to reorder

Dropdowns with suggestions

InteractionDrag and drop to reorder

vsDropdowns with suggestions

Free type

Interaction

Suggested Alternative Translations

Many Hypotheses

Interaction

Suggested Alternative Translations

Interaction

Suggested Alternative Translations

Interaction + TranslationTranslator expertise

Nov

ice

Expe

rt

Nov

ice

Expe

rt

Caitra (2010)

Interaction + TranslationTranslator expertise

Nov

ice

Expe

rt

Caitra (2010)

TransType (2000)

Interaction + TranslationTranslator expertise

Nov

ice

Expe

rt

Google Translate

Caitra (2010)

TransType (2000)

Interaction + TranslationTranslator expertise

Google translate

data powersbetter systems

Nov

ice

Expe

rt

Google Translate

Caitra (2010) Active Crowd Translation (2010)

TransType (2000)

Interaction + TranslationTranslator expertise

data powersbetter systems

collaboration + discussion

Nov

ice

Expe

rt

Google Translate

MonoTrans (2011)Caitra (2010) Active Crowd Translation (2010)

TransType (2000)

Interaction + TranslationTranslator expertise

Beyond Human Computation

Beyond Human Computation

Munteanu, Baecker, and Penn, CHI 2008

Beyond Human Computation

Duolingo.com

Interactive Machine Translation

What if people interact with translation?

How can a computer learn from that?

How powerful is this approach?

Production: www.paperhand.org | Photos: mlight.typepad.com

A Sea of Information

Production: www.paperhand.org | Photos: mlight.typepad.com

A City Obsessedwith More and Newer

Production: www.paperhand.org | Photos: mlight.typepad.com

An Insatiable,Hungry Monster

Production: www.paperhand.org | Photos: mlight.typepad.com

An Insatiable,Hungry Monster

Production: www.paperhand.org | Photos: mlight.typepad.com

An Insatiable,Hungry Monster

Access?Production: www.paperhand.org | Photos: mlight.typepad.com

Effective AccessProduction: www.paperhand.org | Photos: mlight.typepad.com

Google SearchesBy Language

Thanks to:• Mentors

– James Fogarty (UW)– Srinivas Bangalore

(AT&T)– Luke Zettlemoyer (UW)– Dan Weld (UW)– Charlotte P. Lee (UW)– Jen Mankoff (CMU)

• And many more– Study Participants– Colleagues– You!

• Supporting Organizations– National Science Foundation– AT&T Labs– Microsoft Research– Google– University Book Store– Engage: The Science Speaker

Series– University of Washington– Pacific Science Center– Town Hall Seattle– Microsoft

Thank You!Questions?

Contact:kuksenok@cs.uw.edu

Links, Citations, etc:students.washington.edu/kuksenok/engage