algorithms for nlp - cs.cmu.edutbergkir/11711fa17/fa17 11-711 lecture 20... · under the new...

MachineTranslationTaylorBerg-Kirkpatrick– CMU

Slides:DanKlein– UCBerkeley

AlgorithmsforNLP

MachineTranslation

MachineTranslation:Examples

LevelsofTransfer

Word-LevelMT:Examples

§ lapolitiquedelahaine. (Foreign Original)§ politicsofhate. (ReferenceTranslation)§ thepolicyofthehatred. (IBM4+N-grams+Stack)

§ nousavonssignéleprotocole. (Foreign Original)§ wedidsignthememorandumofagreement. (ReferenceTranslation)§ wehavesignedtheprotocol. (IBM4+N-grams+Stack)

§ oùétaitleplansolide? (Foreign Original)§ butwherewasthesolidplan? (ReferenceTranslation)§ wherewastheeconomicbase? (IBM4+N-grams+Stack)

PhrasalMT:Examples

Metrics

MT:Evaluation§ Humanevaluations:subjectmeasures,

fluency/adequacy

§ Automaticmeasures:n-grammatchtoreferences§ NISTmeasure:n-gramrecall(workedpoorly)§ BLEU:n-gramprecision(noonereallylikesit,but

everyoneusesit)§ Lotsmore:TER,HTER,METEOR,…

§ BLEU:§ P1=unigramprecision§ P2,P3,P4=bi-,tri-,4-gramprecision§ WeightedgeometricmeanofP1-4§ Brevitypenalty(why?)§ Somewhathardtogame…§ Magnitudeonlymeaningfulonsamelanguage,corpus,

numberofreferences,probablyonlywithinsystemtypes…

AutomaticMetricsWork(?)

SystemsOverview

Corpus-BasedMT

Modeling correspondences between languages

Sentence-aligned parallel corpus:

Yo lo haré mañanaI will do it tomorrow

Hasta prontoSee you soon

Hasta prontoSee you around

Yo lo haré pronto I will do it soon

I will do it around

See you tomorrow

Machine translation system:

Model of translation

Phrase-BasedSystemOverview

Sentence-aligned corpus

cat ||| chat ||| 0.9 the cat ||| le chat ||| 0.8dog ||| chien ||| 0.8 house ||| maison ||| 0.6 my house ||| ma maison ||| 0.9language ||| langue ||| 0.9 …

Phrase table(translation model)Word alignments

Many slides and examples from Philipp Koehn or John DeNero

WordAlignment

WordAlignment

WordAlignment

What is the anticipated cost of collecting fees under the new proposal?

En vertu des nouvelles propositions, quel est le coût prévu de perception des droits?

x zWhat

is the

anticipatedcost

ofcollecting

fees under

the new

proposal?

En vertu delesnouvelles propositions, quel est le coût prévu de perception de les droits?

UnsupervisedWordAlignment§ Input:abitext:pairsoftranslatedsentences

§ Output:alignments:pairsoftranslatedwords

§ Whenwordshaveuniquesources,canrepresentasa(forward)alignmentfunctiona fromFrenchtoEnglishpositions

nous acceptons votre opinion .

we accept your view .

1-to-ManyAlignments

EvaluatingModels§ Howdowemeasurequalityofaword-to-wordmodel?

§ Method1:useinanend-to-endtranslationsystem§ Hardtomeasuretranslationquality§ Option:humanjudges§ Option:referencetranslations(NIST,BLEU)§ Option:combinations(HTER)§ Actually,nooneusesword-to-wordmodelsaloneasTMs

§ Method2:measurequalityofthealignmentsproduced§ Easytomeasure§ Hardtoknowwhatthegoldalignmentsshouldbe§ Oftendoesnotcorrelatewellwithtranslationquality(likeperplexityinLMs)

AlignmentErrorRate§ AlignmentErrorRate

Sure align.

Possible align.

Predicted align.

=

=

=

IBMModel1:Allocation

IBMModel1(Brown93)§ Alignments:ahiddenvectorcalledanalignment specifieswhichEnglish

sourceisresponsibleforeachFrenchtargetword.

A:

IBMModels1/2

Thank you , I shall do so gladly .

1 3 7 6 9

1 2 3 4 5 76 8 9

Model ParametersTransitions: P( A2 = 3)Emissions: P( F1 = Gracias | EA1 = Thank )

Gracias , lo haré de muy buen grado .

8 8 88

E:

F:

ProblemswithModel1

§ There’sareasontheydesignedmodels2-5!

§ Problems:alignmentsjumparound,aligneverythingtorarewords

§ Experimentalsetup:§ Trainingdata:1.1Msentences

ofFrench-Englishtext,CanadianHansards

§ Evaluationmetric:alignmenterrorRate(AER)

§ Evaluationdata:447hand-alignedsentences

IntersectedModel1

§ Post-intersection:standardpracticetotrainmodelsineachdirectionthenintersecttheirpredictions[OchandNey,03]

§ Secondmodelisbasicallyafilteronthefirst§ Precisionjumps,recalldrops§ Endupnotguessinghard

alignments

Model P/R AERModel 1 E®F 82/58 30.6Model 1 F®E 85/58 28.7Model 1 AND 96/46 34.8

JointTraining?§ Overall:

§ Similarhighprecisiontopost-intersection§ Butrecallismuchhigher§ Moreconfidentaboutpositingnon-nullalignments

Model P/R AERModel 1 E®F 82/58 30.6Model 1 F®E 85/58 28.7Model 1 AND 96/46 34.8Model 1 INT 93/69 19.5

IBMModel2:GlobalMonotonicity

MonotonicTranslation

Le Japon secoué par deux nouveaux séismes

Japan shaken by two new quakes

LocalOrderChange

Le Japon est au confluent de quatre plaques tectoniques

Japan is at the junction of four tectonic plates

IBMModel2§ Alignmentstendtothediagonal(broadlyatleast)

§ Otherschemesforbiasingalignmentstowardsthediagonal:§ Relativevsabsolutealignment§ Asymmetricdistances§ Learningafullmultinomialoverdistances

EMforModels1/2

§ Model1Parameters:Translationprobabilities(1+2)Distortionparameters(2only)

§ Startwith uniform,including§ Foreachsentence:

§ ForeachFrenchpositionj§ CalculateposterioroverEnglishpositions

§ (orjustusebestsinglealignment)§ Incrementcountofwordfj withwordei bytheseamounts§ Alsore-estimatedistortionprobabilitiesformodel2

§ Iterateuntilconvergence

Example

HMMModel:LocalMonotonicity

PhraseMovement

Des tremblements de terre ont à nouveau touché le Japon jeudi 4 novembre.

On Tuesday Nov. 4, earthquakes rocked Japan once again

A:

TheHMMModel

Thank you , I shall do so gladly .

1 3 7 6 9

1 2 3 4 5 76 8 9

Model ParametersTransitions: P( A2 = 3 | A1 = 1)Emissions: P( F1 = Gracias | EA1 = Thank )

Gracias , lo haré de muy buen grado .

8 8 88

E:

F:

TheHMMModel

§ Model2preferredglobalmonotonicity§ Wewantlocalmonotonicity:

§ Mostjumpsaresmall

§ HMMmodel(Vogel96)

§ Re-estimateusingtheforward-backwardalgorithm§ Handlingnullsrequiressomecare

§ Whatarewestillmissing?

-2 -1 0 1 2 3

HMMExamples

AERforHMMs

Model AERModel 1 INT 19.5HMM E®F 11.4HMM F®E 10.8HMM AND 7.1HMM INT 4.7GIZA M4 AND 6.9

Models3,4,and5:Fertility

IBMModels3/4/5

Mary did not slap the green witch

Mary not slap slap slap the green witch

Mary not slap slap slap NULL the green witch

n(3|slap)

Mary no daba una botefada a la verde bruja

Mary no daba una botefada a la bruja verde

P(NULL)

t(la|the)

d(j|i)

[from Al-Onaizan and Knight, 1998]

Examples:TranslationandFertility

Example:Idioms

il hoche la tête

he is nodding

Example:Morphology

SomeResults§ [OchandNey03]

PhraseMovement

TheHMMModel