roi and the value of machine translation

Lionbridge Technology Overview:Technology Best Practice

Presented by Nicholas McMahonModerated by Paula ShannonJuly 15, 2009

Basic MT Introduction

Company confidential – distribution prohibited without permission

Three Types Of Machine Translation

Rules BasediTranslatoriTranslator

StatisticalGoogleGoogle

HybridSystran Enterprise


Server 7

Basic MT Introduction


Top Three Considerations for MT

MT performance in one l i NOT MTlanguage is NOT MT

performance in another language or even in another

language direction



MT is not an

“every language solution” for any of the MT Engines



MT at an Enterprise Level

is NOT free


Top Three Applications of MT:Chat


Top Three Applications of MT:Chat

MT Chat ApproachesHuman to human instant dialogueHuman to semi auto response based onresponse based on key word findings

EvolutionEarly adoption – Value focus, very little distinct ‘ROI’ modeling


modeling

Top Three Applications of MT:Support Content


Top Three Applications of MT:Support Content

FAQ MT ApproachesRaw MT output for un‐ptranslated contentTiered Quality approach y pp

EvolutionVolume Success and reasonable adoptionVolume Success and reasonable adoption“Self serve” & “New content” increases user patience


Top Three Applications of MT:User Guides


Top 3 Applications of MT

Approaches

TM, MT + Post Editing of content

Considerations

Early Adoption – strong indicators of success

Goal is to MT to be ‘invisible’ inGoal is to MT to be invisible in process


POLLING QUESTION

Are you currently considering or using MT in any of these situations?

Considering MT for Chat

C id i MT f FAQConsidering MT for FAQs

Considering MT for Volume User Content

Using MT for ChatUsing MT for Chat

Using MT for FAQs

Using MT for Volume User Contentg


Enterprise Cost Considerations

“Start up Costs” Costs

Data Identification/ Corpus creation $0k ‐ $100k

Customization or training of content $5k ‐ $250k$ $

Licensing fees $0k ‐ $200k

Integration fees $10k ‐ $200k


Enterprise Cost considerations

“Post Editing” Costs

Training of resources for post editing $5kTraining of resources for post editing $5k ‐ $10k

P t Editi t 65% 85% fPost Editing rates – 65% ‐ 85% of new word rates

Ongoing customization/ corpus g g / pmanagement annually $5k ‐ $200k


ROI modeling for MT within Instant communication

User 1 User 2

My Second figure is an animal My role of a young handsome y gwith a tail and four feet

y f y gbeau is a boy with a tail and 4 feet

h ki d f i d i h d ? h dWhat kind of meaning and nice boy?

What do you mean? A handsome boy?

My second figures looks like an My second figure is like anMy second figures looks like an animal

My second figure is like an animal

It has four feet and a tail It has four feet and a tail

Oh, I understand and am the 8th I got it, it is my 8th figure


Source: NTT Labs and Kyoto University,Effects of Machine Translation on Collaborative Work, Dec 06


Establishing the value of instant communications

Abilit t l t tAbility to resolve support requests Cost per multilingual call handling

Ability to communicate betterAbility to communicate betterValue of increased time to market


ROI Modeling for MT within Instant Communication

MT in Chat Benefits VariablesStrong defined reference sets produce better resultsStrong defined language use (specific to a domainuse (specific to a domain or industry)Patience and Flexibility of user (not for complaints!)Favors Value of ‘currency’ of content versus qualityof content versus quality



Results Defined Data Sets

5 ‐ 25% average success rate (resolution of issue)

User Satisfaction generally 30 –50%50%



ROI Model – Statistical MT considered best match

10 Languages, 100,000 chat support requests

$Alignment of data for Corpus Creation: $30kProduct Licenses $100kIntegration of tools $20kIntegration of tools $20kHuman system/ process training $10kSystem tweaking : $20ky g $

Total Cost $180k



Averaged Results for Calculation

10% support cases resolved

Average cost of alternate solution: $15 (FAQ routing/ human Call)

Benefit

100 000 * 10% * $15 $150 000100,000 10% $15 $150,000

Less Costs $180,000,

Basic Financial ROI Year 1 ‐$30,000


POLLING QUESTION

Based on the considered MT chat model do you think the suggested ROI is:

Very Low

LLow

About Right

HighHigh

Very High


ROI modeling for MT applied to FAQs

Establishing the value of FAQs?

Ability of users to autonomously resolve support requests

Increased Customer Satisfaction



Benefits VariablesStrong defined reference sets produce better resultsproduce better resultsQuality of source content constructionExpectations of user –disclaimers on qualityFavors un‐translated contentFavors un translated contentRules based or Hybrid best solutionsProvision of English source linksProvision of English source links provide additional contextCultural expectations


Volume of source data available


Results Defined Data Sets20% ‐ 35% average success rate (resolution of issue)

On average about 8%On average about 8% less than English

User Satisfaction ll 50 65%generally 50 – 65%



ROI Model – Hybrid MT Engine considered

10 Languages, 100,000 FAQs docs 450 words per doc.

$Alignment of data for Corpus Creation: $50kProduct Licenses $250kIntegration of tools/ solution $60kIntegration of tools/ solution $60kRating, feedback reporting $20kHuman system/ process training $10ky / p g $System tweaking : $50k


Total Cost $390k


Averaged Results for Calculation :

25% support cases resolved

Average cost of alternate solution: $35(Human Call Response)

Benefit

100,000* 25% * $35 = $875,000

Less Costs $390000

Basic Financial ROI Year 1 $485,000


POLLING QUESTION

Based on the considered MT FAQ model do you think the suggested ROI is:

Very Low

LLow

About Right

HighHigh

Very High


ROI Modeling for MT applied to User Guides

Establishing the value of MT within User DocumentationImprovement in output volumeReduction in total attracted cost of translationIncreased user content available



Benefits VariablesStrong defined reference sets produce better resultsQuality of source content constructionconstructionKnowledge and skills of human resourcesSystems and tools integration of MT and TMTarget Language complexityTarget Language complexity



Results Defined Data Sets‐10% ‐ +25% productivity improvement

Reduction in total attracted cost per wordp



ROI Model – Rules Based MT considered

4 Languages, 2,000,000 words total

$Alignment of data for Corpus Creation: $5kProduct Licenses $20kHuman system/ process training $20kHuman system/ process training $20kSystem tweaking : $5k

Total Cost $50k



Averaged Results

15 % increased in productivity

Average fully loaded per word cost $0.25

(accounting for fuzzy matches)

Benefit

2 000 000 * 15%* 0 25 $75 7502,000,000 * 15%* 0.25 = $75,750

Less Costs $50,000Less Costs $50,000

Basic Financial ROI Year one $25,750


(about 10% of expected costs)

POLLING QUESTION

Based on the considered MT user guide model do you think the suggested ROI is:

Very Low

LLow

About Right

HighHigh

Very High


MT ROI Take Away

E l Ad ti b t MT b d j t tiEarly Adoption, but MT based projects are reporting

MT solutions are dramatically improving

MT source data is growing exponentiallyMT source data is growing exponentially


Knowledge Center

www.lionbridge.comwww.lionbridge.comDownload White Papers

• Building Stronger Brands Around the World: A Guide to Effective Global Marketing

• Strengthening Global Brands: Key Steps for

Download White Papers

Meaningful Communications around the World

• Building a Global Web Strategy: Best Practices for Developing your International Online Brand

• Machine Translation: Saving Time Lowering Costs

View Webinars

Machine Translation: Saving Time, Lowering Costs, and Improving Services

• Localization 2.0

• The Art and Science of Global Navigation

• The Best Global Web Sites (and Why)

• Localization 2.0

• The Art and Science of Global Navigation

• The Best Global Web Sites (and Why)


roi and the value of machine translation

Business

h d boy

human instant dialogue

chatmt chat approaches

semi auto response

ktraining of resources

nicholas mcmahon

post editing

fp t editit