a case study on multi-modal machine translation · 2017-11-30 · a case study on multi-modal...
TRANSCRIPT
Translating using ImagesA case study on multi-modal machine translation
Industry Challenge: eBay users create product listings in many different languages
• Machine Translation of product titles is essential• Can images help MT models translate product titles better, while
being flexible enough to exploit large, text only MT corpora and deliver state of the art (SOTA) translation quality?
Our Solution:We built a neural MT model to translate a source sentence by scanning around the source words AND specific parts of an image
• Two independent attention mechanisms in one decoder• Spatial (convolutional) image features• Build on SOTA image processing pipelines
The ADAPT Centre for Digital Content Technology is funded under the SFI Research Centres Programme
(Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.
Dr Iacer Calixto, Dr Atul Nautiyal, Ahmed Abdelkader and Prof Andy WayTo learn more about innovative ADAPT technologies, contact: [email protected]
• Neural machine translation (NMT) models that incorporate images• Flexibility to exploit large text-only machine translation (MT) corpora • Fine-tune models on in-domain multi-modal (MM) data
Industry Benefits
Exploit readily usable MM dataDeliver better translations using
image informationObtain expertise on MM language
processingStay ahead of competition, improve
user experience
Increased user engagementOur models can be directly used to allow users to engage in different
languages seamlessly
Use Cases
Multi-modal data exploitationUse and connect additional multi-modal data together to build better MT
models
Co-Developed with: