machine learning system architecture – microsoft translator, a case study : vishal chowdhary...
DESCRIPTION
Microsoft Translator currently supports 100+ languages. We constantly improve the translation quality, add new scenarios, all with a constant team size. This session describes a production scale machine learning architecture using MS Translator as a case study. You will learn the mental model to approach your ML problem and concrete Do’s and Don’ts for the various components of the ML system architecture.TRANSCRIPT
Machine LearningSystem ArchitectureMicrosoft Translator, a Case Study.
Vishal Chowdhary (@talktovishal, vishalc@microsoft)
Outline
Translation APIs and UX
Translation
Engine
Distributed Runtime
HPC/Hadoop
Cluster
Bing Index
Public corpora
Train statistical
models
Find and gather
parallel data
Common Problems
Our Solution
Examples\Demo
Summarize
Evaluation
Release
(Feature + Model)
Selection
Data Acquisition
Training
Data Acquisition
Evaluation
Release
(Feature + Model)
Selection
Data Acquisition
Training
Web
JRC. Europarl
Microsoft
localization
Preprocess,
Sentence Align MT Store
Extract:
Filter Chain
Train SMT
Engine
Evaluation
Release
(Feature + Model)
Selection
Data Acquisition
Training
Training
Level 5Easy
Level 7Challenging
Level 9Grueling
Level 5Easy
Level 8Demanding
Level 10Mind Boggling
Evaluation
Release
(Feature + Model)
Selection
Data Acquisition
Training
Debugging & Evaluation
Evaluation
Release
(Feature + Model)
Selection
Data Acquisition
Training
Release
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on
the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
blogs.msdn.com/translator
twitter.com/MSTranslator
facebook.com/BingTranslator