how to translate different measurements and scores into business intelligence (tom shaw, capita)
TRANSCRIPT
![Page 1: How to translate different measurements and scores into Business Intelligence (Tom Shaw, Capita)](https://reader031.vdocuments.us/reader031/viewer/2022021813/587baafc1a28ab81758b6d33/html5/thumbnails/1.jpg)
Business IntelligenceHow to translate different measurements and scores into
Business Intelligence
Tom Shaw (Capita TI)Paul Mangell (Alpha CRC)Scott Carothers (Kinetic TheTechnologyAgency)
![Page 2: How to translate different measurements and scores into Business Intelligence (Tom Shaw, Capita)](https://reader031.vdocuments.us/reader031/viewer/2022021813/587baafc1a28ab81758b6d33/html5/thumbnails/2.jpg)
The Goal – Improve our Industry using Business Intelligence
?
What Data? How to capture?
What’s relevant? To whom?
How to interpret?
Adequacy
Fluency
TM Leveraging
User Sentiment
MT Leveraging
Accuracy
Style / Readability
Translator Quality
User Engagement
TER, Bleu, Meteor
Per Word
Per Hour CTR
PE Distance
TAT
![Page 3: How to translate different measurements and scores into Business Intelligence (Tom Shaw, Capita)](https://reader031.vdocuments.us/reader031/viewer/2022021813/587baafc1a28ab81758b6d33/html5/thumbnails/3.jpg)
Translation Quality – Business Intelligence from a LSP perspective
Tier 3 content
Tier 2 content
Tier 1 content
Support content, non-publishable user driven or reference
Publishable content for user assistance or product support related
Highly visible content, such as Marketing or software localization as
well as legal
![Page 4: How to translate different measurements and scores into Business Intelligence (Tom Shaw, Capita)](https://reader031.vdocuments.us/reader031/viewer/2022021813/587baafc1a28ab81758b6d33/html5/thumbnails/4.jpg)
Quality Evaluation Triangle
QUALITY
Users
LSPBuyer
The user experience should be equal or better of that achieved in Source language
Customising metrics and QA process in coordination with the
buyer will assure expectations are measured adequately
The buyer’s “internal clients” play a key role in setting up quality
expectations and measuring success
![Page 5: How to translate different measurements and scores into Business Intelligence (Tom Shaw, Capita)](https://reader031.vdocuments.us/reader031/viewer/2022021813/587baafc1a28ab81758b6d33/html5/thumbnails/5.jpg)
Quality Evaluation - LSP and Buyer
Content type specific QA process and metricsLSP Evaluation Criteria / Model:
Error Typology / Adequacy / Fluency
Territory specific requirements and preferences
Buyer Internal Clients
![Page 6: How to translate different measurements and scores into Business Intelligence (Tom Shaw, Capita)](https://reader031.vdocuments.us/reader031/viewer/2022021813/587baafc1a28ab81758b6d33/html5/thumbnails/6.jpg)
Quality Evaluation - Users
Users User feedback and content engagement levels
Evaluation Criteria / Model:User Activity / Engagement
![Page 7: How to translate different measurements and scores into Business Intelligence (Tom Shaw, Capita)](https://reader031.vdocuments.us/reader031/viewer/2022021813/587baafc1a28ab81758b6d33/html5/thumbnails/7.jpg)
Quality Evaluation Models - Thoughts
1) Quality Evaluation Models have to be closely linked to purpose / end use
2) Standard models not always good enough. Customised to specific needs
3) Metrics / Benchmarks should be set with end user in mind
……………………. Who should drive? LSP<>Buyer, Buyer<>Users….
![Page 8: How to translate different measurements and scores into Business Intelligence (Tom Shaw, Capita)](https://reader031.vdocuments.us/reader031/viewer/2022021813/587baafc1a28ab81758b6d33/html5/thumbnails/8.jpg)
Common Metrics the Buyers Maintain
Most common metrics:
• Which LSP has the assignment
• Delivery Time
• How long did post-editing take?
![Page 9: How to translate different measurements and scores into Business Intelligence (Tom Shaw, Capita)](https://reader031.vdocuments.us/reader031/viewer/2022021813/587baafc1a28ab81758b6d33/html5/thumbnails/9.jpg)
Most Valuable Metrics for the Buyer
1. Aggregate volume
2. Turnaround time
3. TM Analysis
4. LSP ratings
5. Cost tracking
![Page 10: How to translate different measurements and scores into Business Intelligence (Tom Shaw, Capita)](https://reader031.vdocuments.us/reader031/viewer/2022021813/587baafc1a28ab81758b6d33/html5/thumbnails/10.jpg)
Enterprise Metrics Dashboard
![Page 11: How to translate different measurements and scores into Business Intelligence (Tom Shaw, Capita)](https://reader031.vdocuments.us/reader031/viewer/2022021813/587baafc1a28ab81758b6d33/html5/thumbnails/11.jpg)
Checks and balances in collecting and sharing data
• How will the data be collected, analysed, presented?
• Is all data good data?
• Who needs to know which data set?
• What is the cost of collecting, analysing, and presenting data?
• What us the impact of negative or positive data on the enterprise, and the resource/supplier?
• What should be done in the light of the new data?
• Who will take the actions implied/required?
• Who needs to agree that the data strategy is correct? Buy in v apathy/antipathy
![Page 12: How to translate different measurements and scores into Business Intelligence (Tom Shaw, Capita)](https://reader031.vdocuments.us/reader031/viewer/2022021813/587baafc1a28ab81758b6d33/html5/thumbnails/12.jpg)
Challenges / Considerations / Points to Discuss?• What is a quality level? Who sets / defines it?
• All supply chain to work towards the same goal
• Linking QE results from LSP for different levels with End User Feedback
• External factors affecting the results – e.g. translation quality directly linked to # sales?
• Not all languages can be treated equally – e.g. acceptance of MT in Japan vs Spain
• High quality / perfect really always required?
• Variable quality levels – impact on brand?
• How do high QA scores correlate to user engagement?
• How to customise your metrics based on the communication objectives
• Professional translators offering a ‘less than perfect’ service