![Page 1: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/1.jpg)
Discriminative Modeling Discriminative Modeling extraction Sets for Machine extraction Sets for Machine TranslationTranslationAuthorJohn DeNero and Dan Klein UC BerkeleyPresenterJustin Chiu
![Page 2: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/2.jpg)
ContributionContributionExtraction set
◦Nested collections of all the overlapping phrase pairs consistent with an underlying word-alignment
Advantages over word-factored alignment model◦Can incorporate features on phrase pairs,
more than word link◦Optimize a extraction-based loss function
really direct to generating translationPerform better than both supervised
and unsupervised baseline
![Page 3: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/3.jpg)
Progress of Statistical MTProgress of Statistical MTGenerate translated sentences
word by wordUsing while fragments of training
example, building translation rules◦Aligned at the word level ◦Extract fragment-level rules from word
aligned sentence pair Tree to string translation
Extraction Set Models◦Set of all overlapping phrasal
translation rule + alignment
![Page 4: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/4.jpg)
OutlineOutlineExtraction Set ModelsModel EstimationModel InferenceExperiments
![Page 5: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/5.jpg)
EXTRACTION SET EXTRACTION SET MODELSMODELS
![Page 6: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/6.jpg)
Extraction Set ModelsExtraction Set ModelsInput
◦Unaligned sentence
Output◦Extraction set of phrasal translation
rules◦Word alignment
![Page 7: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/7.jpg)
Extraction Sets from Word Extraction Sets from Word AlignmentsAlignments
![Page 8: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/8.jpg)
Extraction Sets from Word Extraction Sets from Word AlignmentsAlignments
![Page 9: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/9.jpg)
Extraction Sets from Word Extraction Sets from Word AlignmentsAlignments
![Page 10: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/10.jpg)
Possible and Null Alignment Possible and Null Alignment LinksLinksPossible links has two types
◦ Function words that is unique in its language◦ Short phrase that has no lexical equivalent
Null alignment◦ Express content that is
absent in its translation
![Page 11: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/11.jpg)
Interpreting Possible and Null Interpreting Possible and Null Alignment LinksAlignment Links
![Page 12: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/12.jpg)
Interpreting Possible and Null Interpreting Possible and Null Alignment LinksAlignment Links
![Page 13: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/13.jpg)
Linear Model for Linear Model for Extraction SetExtraction Set
![Page 14: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/14.jpg)
Scoring Extraction SetsScoring Extraction Sets
![Page 15: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/15.jpg)
MODEL ESTIMATIONMODEL ESTIMATION
![Page 16: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/16.jpg)
MIRA(Margin-infused Relaxed MIRA(Margin-infused Relaxed Algorithm)Algorithm)
![Page 17: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/17.jpg)
Extraction Set Loss Extraction Set Loss FunctionFunction
![Page 18: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/18.jpg)
MODEL INFERENCEMODEL INFERENCE
![Page 19: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/19.jpg)
Possible DecompositionsPossible Decompositions
![Page 20: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/20.jpg)
DP for Extraction SetsDP for Extraction Sets
![Page 21: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/21.jpg)
DP for Extraction SetsDP for Extraction Sets
![Page 22: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/22.jpg)
Finding Pseudo-Gold ITG Finding Pseudo-Gold ITG AlignmentAlignment
![Page 23: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/23.jpg)
EXPERIMENTSEXPERIMENTS
![Page 24: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/24.jpg)
Five systems for Five systems for comparisoncomparisonUnsupervised baseline◦ Giza++◦ Joint HMMSupervised baseline◦ Block ITGExtraction Set Coarse Pass◦ Does not score bispans that corss
bracketing of ITG derivationsFull Extraction Set Model
![Page 25: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/25.jpg)
DataDataDiscriminative training and
alignment evaluation◦Trained baseline HMM on 11.3 million
words of FBIS newswire data◦Hand-aligned portion of the NIST MT02
test set 150 training and 191 test sentences
End-to-end translation experiments◦Trained on 22.1 million word prarllel
corpus consisting of sentence up to 40 of newswire data from GALE program
◦NIST MT04/MT05 test sets
![Page 26: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/26.jpg)
ResultsResults
![Page 27: Discriminative Modeling extraction Sets for Machine Translation Author John DeNero and Dan KleinUC Berkeley Presenter Justin Chiu](https://reader035.vdocuments.us/reader035/viewer/2022062302/5a4d1b567f8b9ab0599a955c/html5/thumbnails/27.jpg)
DiscussionDiscussionSyntax labels v.s wordsWord align to rule Rule to word
alignInformation from two directions65% of type 1 error