Upload File

machine translation - sameer singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...•...

  • Home
  • Documents
  • Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March
46
Machine Translation Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February 28, 2017 Based on slides from Jason Eisenstein, Chris Dyer, Alan Ritter, Yejin Choi, and everyone else they copied from.

Upload: others

Post on 03-Jul-2020

2 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

MachineTranslation

Prof.SameerSinghCS295:STATISTICALNLP

WINTER2017

February28,2017

BasedonslidesfromJasonEisenstein,ChrisDyer,AlanRitter,YejinChoi,andeveryoneelsetheycopiedfrom.

Page 2: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Upcoming…

• Homework4isdueonMarch13• Write-upanddatareleasingsoon.Homework

• Statusreportduein1weeks:March7,2017• Instructionscomingtoday!• Almostfinalreport,only5pages

Project

• Papersummaries:February28,March14• Summary1gradedSummaries

CS295:STATISTICALNLP(WINTER2017) 2

Page 3: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Outline

MachineTranslation

IntroductiontoStatisticalMT

IBMTranslationModels

CS295:STATISTICALNLP(WINTER2017) 3

Page 4: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Outline

MachineTranslation

IntroductiontoStatisticalMT

IBMTranslationModels

CS295:STATISTICALNLP(WINTER2017) 4

Page 5: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

MachineTranslation

CS295:STATISTICALNLP(WINTER2017) 5

IhavealwaysimaginedParadiseasakindoflibrary.

Yo,quemefigurabaelParaíso/Bajolaespeciedeunabiblioteca.

Page 6: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Challenges:WordOrder

CS295:STATISTICALNLP(WINTER2017) 6

EvenforSVO English:IwillbuyitFrench:Jevais l’acheter (Iwillitbuy)

English:IboughtitFrench:Jel’ai achet´ e(Iithavebought)

SVOvsSOV English:IBMboughtLotusJapanese:IBMLotusbought

Page 7: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Challenges:LexicalAmbiguity

CS295:STATISTICALNLP(WINTER2017) 7

bill

pico

cuenta

Page 8: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Challenges:Pronouns

CS295:STATISTICALNLP(WINTER2017) 8

InSpanish,youcanrecoverthepronounfromverbinflection:Vivimos en Atlanta→We liveinAtlanta

IAgain,discoursecontextisoftencrucial:Vive en Atlanta→She/he/it livesinAtlanta

Englishpossessivepronounstakethegenderoftheowner:Marieridesher bike

Frenchpossessivepronounstakethegenderoftheobject:Mariemonte surson vélo

DifferentPronouns

DroppingPronouns

Page 9: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Challenges:Tenses

CS295:STATISTICALNLP(WINTER2017) 9

Thepreterite tenseisforeventswithadefinitetime,e.g.Ibikedtoworkthismorning

Theimperfectisforeventswithindefinitetimes,e.g.Ibikedtoworkalllastsummer

TotranslateEnglishtoSpanish,wemustpicktherighttense.

Page 10: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Challenges:Idioms

CS295:STATISTICALNLP(WINTER2017) 10

Whyintheworld

Kickthebucket

Lendmeyourears

DeadAsADoornail

AsCoolAsaCucumber

HoldYourHorses

StorminaTeacupBob'sYourUncle

BlueintheFace

HeadInTheClouds

Page 11: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Rules forMachineTranslation

CS295:STATISTICALNLP(WINTER2017) 11

Rulesfortranslatingmuch ormany intoRussian:

if precedingwordishow returnskol’koelseifprecedingwordisas return stol’ko zheelseif wordismuch

ifprecedingwordisvery returnnilelseiffollowingwordisanounreturnmnogo

else (wordismany)ifprecedingwordisaprepositionandfollowingwordisnounreturnmnogiielsereturnmnogo

Panov (1960)

Page 12: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

TheVauquios Triangle

CS295:STATISTICALNLP(WINTER2017) 12

Page 13: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Outline

MachineTranslation

IntroductiontoStatisticalMT

IBMTranslationModels

CS295:STATISTICALNLP(WINTER2017) 13

Page 14: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

StatisticalMachineTranslation

CS295:STATISTICALNLP(WINTER2017) 14

Page 15: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

ParallelCorpus:Examples

CS295:STATISTICALNLP(WINTER2017) 15

Page 16: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

ParallelCorpus:Examples

CS295:STATISTICALNLP(WINTER2017) 16

Page 17: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

ParallelCorpus:Examples

CS295:STATISTICALNLP(WINTER2017) 17

Page 18: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

ParallelCorpus:Examples

CS295:STATISTICALNLP(WINTER2017) 18

Page 19: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

TheRosettaStone

CS295:STATISTICALNLP(WINTER2017) 19

Page 20: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

WarrenWeaver(1949)

CS295:STATISTICALNLP(WINTER2017) 20

Page 21: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

ParallelCorpus:Examples

CS295:STATISTICALNLP(WINTER2017) 21

Page 22: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

ParallelCorpus:Examples

CS295:STATISTICALNLP(WINTER2017) 22

Page 23: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

NoisyChannelModel

CS295:STATISTICALNLP(WINTER2017) 23

“NoisyChannel” Decoder

Page 24: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

NoisyChannelModel

CS295:STATISTICALNLP(WINTER2017) 24

“NoisyChannel” Decoder

Page 25: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Example:NoisyChannel

CS295:STATISTICALNLP(WINTER2017) 25

Page 26: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Example:NoisyChannel

CS295:STATISTICALNLP(WINTER2017) 26

Page 27: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

ComponentsofanMTsystem

CS295:STATISTICALNLP(WINTER2017) 27

LanguageModel

TranslationModel

DecodingAlgo

Page 28: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

ComponentsofanMTsystem

CS295:STATISTICALNLP(WINTER2017) 28

Page 29: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

EvaluatingMT

CS295:STATISTICALNLP(WINTER2017) 29

Page 30: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

HumanEvaluation

CS295:STATISTICALNLP(WINTER2017) 30

Fluency

Adequacy

A:furiousnAgA onwednesday ,thetribalminimumpur oftenschoolsalsowasburnt

B:furiousnAgA onwednesday thetribalpur minitenschoolsofthemwasalsoburnt

Page 31: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

AutomatedEvaluation

CS295:STATISTICALNLP(WINTER2017) 31

Fluency

Adequacy

Page 32: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

BLEUScore

CS295:STATISTICALNLP(WINTER2017) 32

Page 33: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

BLEUScore:Example

CS295:STATISTICALNLP(WINTER2017) 33

‘extensionofisi inuttar pradesh ’

‘isi ’sexpansioninuttar pradesh ’‘thespreadofisi inuttar pradesh ’‘isi spreadinginuttar pradesh ’thespreadofisi inuttar pradesh

Page 34: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

BLEUScore:Example

CS295:STATISTICALNLP(WINTER2017) 34

‘extensionofisi inuttar pradesh ’

‘isi ’sexpansioninuttar pradesh ’‘thespreadofisi inuttar pradesh ’‘isi spreadinginuttar pradesh ’thespreadofisi inuttar pradesh

Page 35: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

BLEU’snotbad…

CS295:STATISTICALNLP(WINTER2017) 35G.Doddington,NIST

Page 36: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Outline

MachineTranslation

IntroductiontoStatisticalMT

IBMTranslationModels

CS295:STATISTICALNLP(WINTER2017) 36

Page 37: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Statistical TranslationModel

CS295:STATISTICALNLP(WINTER2017) 37

Andtheprogramwasimplemented

Laprogrammation aété mise en application

Page 38: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

WordAlignment:Direct

CS295:STATISTICALNLP(WINTER2017) 38

Page 39: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

WordAlignment:1-to-Many

CS295:STATISTICALNLP(WINTER2017) 39

Page 40: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

WordAlignment:Reordering

CS295:STATISTICALNLP(WINTER2017) 40

Page 41: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

WordAlignment:Inserting

CS295:STATISTICALNLP(WINTER2017) 41

Page 42: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

WordAlignment:Dropping

CS295:STATISTICALNLP(WINTER2017) 42

Page 43: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

TranslatingwithAlignments

CS295:STATISTICALNLP(WINTER2017) 43

Page 44: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

Example:TranslationProb

CS295:STATISTICALNLP(WINTER2017) 44

Page 45: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

IBMModels

CS295:STATISTICALNLP(WINTER2017) 45

Model1

Model2

Model3/4/5

Page 46: Machine Translation - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0228...• Write-up and data releasing soon. Homework • Status report due in 1 weeks: March

WordAlignmentAlgorithm

CS295:STATISTICALNLP(WINTER2017) 46


ffi - University of California, San Diegocseweb.ucsd.edu/classes/wi17/cse151-a/DecisionTree.pdf · Bq,cqr- aY\& 6uU{ll4s {o.Zh p\or1s?,r-q+ zsx) '1 arn5r,at frptrc'(t 't) '(l'o)'(o'o)

Drugs Targeting the CNSclasses.biology.ucsd.edu/bimm118.WI17/PPT Lecture Notes/Lecture 14… · Drugs Targeting the CNS ... • Neurons: excites some neurons, ... – Characteristic

CSE 237A Winter 2017: Individual Project Part 2 of 2: Sensor ...cseweb.ucsd.edu/classes/wi17/cse237A-a/project/part2/cse237aWI1… · CSE 237A Winter 2017: Individual Project Part

Steroid-based Drugs •Adrenocortical Hormonesclasses.biology.ucsd.edu/bimm118.WI17/PPT Lecture... · • Carpal Tunnel Syndrome • Dermatitis ... –Replacement therapy (Turner

Lecture 12: Deep semantics dependencies & semantic roles StatNLP, Winter 2008 Roger Levy UCSD Linguistics

Autonomic Nervous System - Class Websitesclasses.biology.ucsd.edu/bimm118.WI17/PPT Lecture Notes/Lecture 7.… · Autonomic Nervous System Transmitters: ... (“rhinitis medicamentosa”)

Home | Computer Science - Outline Outline in Codecseweb.ucsd.edu/~viscomp/classes/cse167/wi17/slides/... · 2017-01-23 · § Will need to implement matrix stacks like in OpenGL

CSE 123 Discussion 1: Project 1 Overviewcseweb.ucsd.edu/classes/wi17/cse123-a/discussions/week1.pdfHow to download assignment Once you are granted access login to GitHub. Switch to

Internet of Things (IoT) - University of California, San Diegocseweb.ucsd.edu/classes/wi17/cse237A-a/handouts/8_iot.pdf · in smart spaces using intelligent interfaces to connect

Chemotherapeutic Agents - Class Websitesclasses.biology.ucsd.edu/bimm118.WI17/PPT Lecture Notes/Lecture 16.pdf · Antibiotics - Protein synthesis inhibitors Protein synthesis inhibitors:

Syntactic Parsing - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0207-syntactic... · Syntactic Parsing Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 February

Linear Regression - Sameer Singhsameersingh.org/courses/gml/fa17/slides/04-lin...Linear regression Define form of function f(x) explicitly Find a good f(x) within that family 0 10

Lecture21: Links and Signalingcseweb.ucsd.edu/classes/wi17/cse123-a/lectures/123-wi17... · 2017. 3. 3. · 2 Lecture21 Overview Quality of Service Signaling Channel characteristics

Statistical and Learning Methods in Natural Language …cs.haifa.ac.il/~shuly/teaching/04/statnlp/chunking.pdfIntroduction Text Chunking using TBL Memory-based shallow parsing Sequential

Language Modeling - Sameer Singhsameersingh.org/.../wi17/slides/lecture-0124-lang-models.pdfLanguage Modeling Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER 2017 January 24, 2017

Drug Metabolismclasses.biology.ucsd.edu/bimm118.WI17/PPT Lecture Notes...Drug Metabolism - Oxidation • Humans have 18 families of cytochrome P450 genes and 43 subfamilies: – CYP1

The Suicideof the West - Home on the RANGErangemagazine.com/features/winter-17/range-wi17-sr-rachel_carson.pdf · pest-killing properties were discovered, and ... The Suicideof the

Question Answering (and Textual Entailment)sameersingh.org/courses/statnlp/wi17/slides/lecture-0314... · 2020-04-19 · Question Answering (and Textual Entailment) Prof. Sameer Singh

•Anesthesia •Narcotic Analgetics •Local Anesthetics …classes.biology.ucsd.edu/bimm118.WI17/PPT Lecture Notes/Lecture 15.pdfthe brain => PG E produced => affects thermoregulation

CSE 258 Lecture 8 - Home | Computer Science and …cseweb.ucsd.edu/classes/wi17/cse258-a/slides/lecture8.pdfCSE 258 –Lecture 8 Web Mining and Recommender Systems Extensions of latent-factor

Summary - University of California, San Diegocseweb.ucsd.edu/~viscomp/classes/cse167/wi17/slides/...Summary Monte Carlo methods robust and simple (at least until nitty gritty details)

CSE 20 DISCRETE MATH - Home | Computer Science and …cseweb.ucsd.edu/classes/wi17/cse20-ab/Lect8_Wi17.pdf ·  · 2017-02-03• Hypothetical syllogism ... conditional statement

CSE 258 - cseweb.ucsd.educseweb.ucsd.edu/classes/wi17/cse258-a/slides/... · Assignment 1 • Two recommendation tasks • Due Feb 27 (four weeks -2 days from today) • Submissions

Tajana Simunic Rosing Department of Computer Science and ... › classes › wi17 › cse237A-a › handouts › 01... · ARM Software Development Toolkit (SDT) –tools for CL and

CSE 151 Intro. to AI: A Statistical Approachcseweb.ucsd.edu/classes/wi17/cse151-a/slides.pdfCSE 151 Intro. to AI: A Statistical Approach Instructor: Kamalika Chaudhuri Course Staff

Semantics: Roles and Relations - Sameer Singhsameersingh.org/courses/statnlp/wi17/slides/lecture-0214...Semantics: Roles and Relations Prof. Sameer Singh CS 295: STATISTICAL NLP WINTER

Autonomic Nervous System - University of California, …classes.biology.ucsd.edu/bimm118.WI17/PPT Lecture Notes/Lecture 6.… · Autonomic Nervous System Transmitters: ... – Ganglion-specific

Discourse and Summarization - Sameer Singhsameersingh.org/courses/...discourse-summarization.pdf · Discourse Relations 15 1. In today’s society, college is ambiguous. 2. We need

Sequence Labeling, Contd - Sameer Singhsameersingh.org/.../wi17/slides/lecture-0202-sequence-labeling-contd… · Sequence Labeling, Contd Prof. Sameer Singh CS 295: STATISTICAL NLP

CSE 158 Lecture 13 - University of California, San Diegocseweb.ucsd.edu/classes/wi17/cse158-a/slides/lecture13_annotated.pdf · Web Mining and Recommender Systems Triadic closure;

CSE 124 January 27, 2017 - Home | Computer Sciencecseweb.ucsd.edu/~gmporter/classes/wi17/cse124/... · CSE 124 January 27, 2017 Winter 2017, UCSD Prof. George Porter

CSE 258 Lecture 14cseweb.ucsd.edu/classes/wi17/cse258-a/slides/lecture14.pdf · 2017-03-07 · gets one user).92.75.24.67.97.59.58 bid / quality of the recommendation. Advertising

Adversarial Examples in NLP - Sameer Singhsameersingh.org/files/ppts/naacl19-advnlp-part2-sameer... · 2021. 7. 5. · Sameer Singh, NAACL 2019 Tutorial 17 𝛻𝐽𝑥 𝐽𝑥 Or

Hardware/Software Codesigncseweb.ucsd.edu/classes/wi17/cse237A-a/handouts/10.hwsw.pdf · e.g. hardware components for the DCT or the FFT. Index set J of hardware component instances

Gastrointestinal Pharmacology - University of California ...classes.biology.ucsd.edu/bimm118.WI17/PPT Lecture Notes/Lecture 10.pdf · Gastrointestinal Pharmacology Laxative abuse: