We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byDwayne Powell
Modified about 1 year ago
100 years of progress and innovation © 2011 IBM Corporation The value of Post Editing - IBM Case Study Frank X. Rojas, Jian Ming Xu, Santi Pont Nesta, Álex Martínez Corrià, Salim Roukos, Helena Chapman, Saroj K. Vohra June 2011
© 2011 IBM Corporation2 IBM Case Study – MT Post Editing Introduction MT Innovation Process Overview Findings Conclusion / Recommendations
© 2011 IBM Corporation3 IBM World Wide Translation Operations 24 Centers World Wide ~115 Translation Suppliers Process ~2.8 B Words Translate ~0.4 B Words ~ 60 language pairs One Stop Shop for all Translation Services Marketing Material Web Product Integrated Information Publications Legal/Safety/ Contracts Machine Translation Multimedia Francization Cultural Consultancy Centralized DTP Overall End to End Process Management
© 2011 IBM Corporation4 IBM Professional Translation Services Professional Memory 72% 85% Re-Use Unit Cost >50% Reduction Traditional Technology Process Mgmt Human Skill Consistent Quality Standards Global Brand Identity Professional Quality Standards Future: –Ability to reduce cost using conventional methods reaching limits –Business pressure for additional cost elimination –Looking to MT Technology as next wave to reach business goals
© 2011 IBM Corporation5 - MT portal - Generic crowdsourcing - Text translation services June 2008 Historical Perspective RTTS introduced in 2006 as platform for speech and text translation, developed by IBM Research 2010 MT piloting Pilot: SPA, ITA, FRE, GER New E2E process Partnership: WWTO/n.Fluent 8.6 M words 2011 MT Training Pilot: GER, BPR, JPN, CHS MT payment profiles ready 16.0 M words target eSupport (www) “Translate This Page” JPN pilot / rule engine n.Fluent customized with WWTO translation memories eSupport “Translate This Page” switch to n.Fluent RTTS licensed to IBM partners Initial n.Fluent/WWTO Spanish MT pilot Improve efficiency of professional translators
© 2011 IBM Corporation6 MT Critical Success Metrics Necessary and sufficient condition to measure success –5.0 M words sampled –Minimum of 3 languages –Net Contribution to ROI by MT Engine: 10% of payable words should be MT –No more than 5% adverse impact to Overall Quality Index –No more than 5% impact to Customer Satisfaction Lack of industry metrics and guidance. –Active research on MT technology... no guidance on operational impacts –A business vacuum existed on how to integrate MT services –No operational process had been defined for MT services
© 2011 IBM Corporation7 IBM’s Watson Q&A computer Google’s autonomous car Technologies to understand and produce natural human speech Instantaneous, high-quality machine translation Smartphones / App phones in the developing world * Andrew McAfee is a principal research scientist in the MIT Sloan School of Business Recent Digital Innovations with Biggest Impact in the Business World*
© 2011 IBM Corporation8 Real-Time Translation Server (RTTS) & n.Fluent Real Time Translation Server (RTTS) IBMs MT Engine RTTS provides machine translation for n.Fluent & other applications APIs allow other applications to access these translation services. Customization tools – Domains, chat-specific models, … Commercially licensed to IBM partners Language Pairs to/from English: n.Fluent IBMs MT translation application Providing machine translation services for: Text, web pages, and documents (Word, Excel, …) Instant Messaging chats (via IM plug-in) Mobile translation application (BlackBerry and others) Enabled with LEARNING via crowdsourcing (internal 450K IBMers) Deployed for eSupport self serving tech support (external) العربية 中文 Deutsch English Français Italiano 日本語 Português Русский Español 한국어
© 2011 IBM Corporation9 - MT portal - Generic crowdsourcing - Text translation services June 2008 Historical Perspective RTTS introduced in 2006 as platform for speech and text translation, developed by IBM Research 2010 MT piloting Pilot: SPA, ITA, FRE, GER New E2E process Partnership: WWTO/n.Fluent 8.6 M words 2011 MT Training Pilot: GER, BPR, JPN, CHS MT payment profiles ready 16.0 M words target eSupport (www) “Translate This Page” JPN pilot / rule engine n.Fluent customized with WWTO translation memories eSupport “Translate This Page” switch to n.Fluent RTTS licensed to IBM partners Initial n.Fluent/WWTO Spanish MT pilot Improve efficiency of professional translators
© 2011 IBM Corporation10 TM MT New / Changed 100% Exact Match MT Pre-Process Editing Session MT Post Editing End to End Workflow Upfront & on-going MT tuning via IBM TM professional translations –Professional translation = Best context Matching methods –Traditional TM – breaks down segment level –Machine TM – breaks down block level using MT models – reconstructs segments preserving formats/mark-up tags MT service level integration TM Pre-Process Shipment English TM Match Analysis CAT Translation 1.Show best choice vs vs 2.Select best choice (Post Edit rules) 3. Commit language TESTING QUALITY MT Model & Trans. = Localization Kit (NLV Folder)
© 2011 IBM Corporation1118-sept.-08 MT Pre-processing TM New / Changed 100% Exact Match Build dynamic, domain specific MT model MT MT initial corpus General parallel training corpus Domain specific parallel training corpus ALL segment “no match segments ” Translation of no match segments Initial MT corpus –done before start of project Localization kit TM MT New / Changed 100% Exact Match
© 2011 IBM Corporation1218-sept.-08 Xxx xxx xx xxx xxx xxx. La aplicación desprotege los archivos antes de exportarlos. Yy yyy yyy TM Editing Environment TM Environment Xxx xxx xx xxx xxx xxx. The application unprotects files before exporting them. Yy yyy yyy Translation Memory 0 - The application unprotects files before exporting them. 1[m] – La aplicación desprotege archivos antes de exportarlos. 2[f 85%] - La aplicación protege los archivos antes de exportarlos TM Environment [Ctrl + 1] Typed Translator options Ignore fuzzy and MT Post edit MT Post edit fuzzy Two Seconds Rule: Translators are trained on several strategies to make a quick choice TM MT
© 2011 IBM Corporation13 Productivity Measurements Start segment –Choose action End segment MT productivity evaluation log (MTeval Log) –N events –Words | Time | Existing Proposal | Used Proposal |... Examine productivity per payment category –SUM(Words) / SUM(Time) –Use of IBM Business Analytic Tool (SPSS) –Trim events that fall into 5% (slowest) and 95% (fastest) percentile 1.accept match [~0 time] 2.edit match [X time] 3.reject match [manual translation] Each event EM : Exact RM : Replace FM : Fuzzy MT : Machine NP : No Proposal A) = “best” Existing Proposal B) = “alternative” Existing Proposal C) = reject all Existing Proposal, 100% human labor
© 2011 IBM Corporation14 Total # events : 2,309 (377+1,932) Total words: 24,150 Total time: 27,362 –3,911 w/ MT match 11,377 w/ MT match –20,239 w/o MT match 15,985 w/o MT match MT impact to productivity –MT : 0.44 words/sec [1777 words / 4071 sec] –NP 0.21 w/ MT match 0.32 w/o MT match Baseline (placebo) MT Leverage : 71.8% [1777 / ( )] Single Shipment EXAMPLE rate(MT) / rate(NP) : 1.37 i.e. Translator can complete 37% more words in the same time. Key metrics
© 2011 IBM Corporation15 MT Impact on Fuzzy Match : 4Q10 Findings When FM & MT matches exist simultaneously Productivity: rate(MT) / rate(NP): a.Case : Translator edits FM b.FM-MT Combined case c.Case: Translator edits MT ** Findings subject to change with additional sampling. Overall –Machine matches not as good as professional (fuzzy) matches –No statistical impact to fuzzy productivity to include MT matches. SPA highest sample case 28.6% 4.4% 57.6% 46.9% FM-MT Pick Rate: FREGERITASPA Productivity ratio FM FM-MT MT
© 2011 IBM Corporation16 MT Key Metrics: 4Q10 Findings 8.6 M words sampled in real time translation service. SPA : Qualified MT engine 4Q10 ITA : Qualified MT engine 4Q10 FRA : Qualified MT engine 1Q11 While rate(MT) / rate(NP) is high, the findings were not statistically significant in 4Q. GER : Insufficient productivity from MT engine ** Findings subject to change with additional sampling.
© 2011 IBM Corporation17 Overall Savings Assessment Overall savings % –Word savings due to MT efficiency Convert time savings MT payment factor % –MT payment factor X [MT % words + NP % words] Results in less payable words. MT productivity savings drives a overall savings –These are not the same due to MT % distribution. Supply chain has to consider cost of MT services ** Findings subject to change with additional sampling.
© 2011 IBM Corporation18 Pay for MT Words Translated not MT Matches We pay for final results (MT payable words) not MT matches –MT matches considered “opinion” until chosen by a human –Too many opinions & opinions by immature MT models are less efficient. Actual MT payable words have value beyond the specific project –Post Edited words are reused in future and unknown MT context Engine has to deliver consistent MT payable words –Minimum needed to quality an MT engine for compensation High MT productivity[rate(MT) / rate(NP)] High MT leverage[% of MT matches used] –Compensation to be based on MT payment factor
© 2011 IBM Corporation19 Variance across Languages There is no single maturity path when modeling MT engines across many languages. IBM Pilot: each trained MT engine is a unique asset. –Some languages require more modeling/tuning than others. –Language pairs that service “Loose -> Structured” languages are struggling German requires more effort than Spanish Are there limitations to statistical MT engines? –New thinking may need to be explored? Each MT engine will have separate MT payment factors.
© 2011 IBM Corporation20 Perspective of MT Post Edit Pilots Translation Service Hierarchy Professional Translation Services (Professional LSP) Community Translation Services (Controlled Social Crowd) Volunteer Translation Services (General Crowds) Free Services (Individual) Quality / Reliability LOWER HIGHER General Domain Specific internal IBM All IBM external/internal Pubs / UI external (2011 Pilots) internal IBM n.Fluent “machine” WWTO “human” New Memory Assets MT Post Editing has impacts across entire Translation Service Hierarchy
© 2011 IBM Corporation21 1.Professional (Human) memories are the best assets and deliver the highest quality. 2.Professional memories are a key asset for MT success. 3.All Memory assets need to be protected and managed. 4.Flow of memories between Professional and Machine must be properly balanced. 5.Dynamic modeling offers significant advantage over static modeling. 6.Continuous business analytics is needed to optimize machine assets. 7.A single cost model per language is needed, independent of MT services/engines. 8.An aggressive yet cautious approach is warranted to go forward. MT Post Editing Project – Key Lessons MT Post Editing does improve productivity and efficiency of a localization supply chain.
Nicki Sommer Principal Sales Consultant NCOAUG, August 15, 2008 Whats new in release 12 manufacturing?
Dr. Chen, Electronic Commerce Pearson/Prentice Hall & Dr. Chen, Electronic Commerce Chapter 14 E-Commerce Strategy and Global EC.
Chapter 8 E-Strategy, Internet Communities, and Global EC Prentice Hall, 2003.
E-Sourcing Today A Perspective on the Role and Scope of e- Sourcing and the State of the e-Sourcing Marketplace.
1 Roberto Cencioni Kimmo Rossi Challenge 2 – Objective 2.2 Language based Interaction DG Information Society and Media Unit INFSO.E1 Language Technologies.
Project Management Dr. Anbang Qi Prof. of International Business School of Nankai University.
What is the Value of Architecture Andrew L Macaulay Global Head of Architects Community March 2006 In collaboration with Microsoft Architect Insight Conference.
ERP and E-Business- An Overview Based on the book Enterprise Resource Planning Solutions and Management by Flona Fui-Hoon Nah, Idea Group Publishing 2001.
E-Marketing, 3rd edition Judy Strauss, Adel I. El-Ansary, and Raymond Frost Chapter 2: Strategic E-Marketing © Prentice Hall 2003.
Management Information Systems MANAGING THE DIGITAL FIRM, 12 TH EDITION BUILDING INFORMATION SYSTEMS Chapter 13 VIDEO CASES Case 1: IBM: Business Process.
Wyeth is now a part of Pfizer Inc. The merger of local Wyeth and Pfizer entities may be pending in various jurisdictions and is subject to completion of.
UNIT I FUNDAMENTAL OF E-COMMERCE 1.1INTRODUCTION TO E-COMMERCE 1.2 DRIVING FORCES OF E-COMMERCE 1.3 BENEFITS AND LIMITATIONS OF E-COMMERCE 1.4 DATA MINING.
1 Services. 2 Agenda Overview –Managing the Transitions of The Networked Learning Environment Blackboard Consulting –Who We Are and What We Do Blackboard.
Invitation to Join Open Health Tools Draft by Skip McGaughey
Management Information Systems MANAGING THE DIGITAL FIRM, 12 TH EDITION GLOBAL EDITION BUILDING INFORMATION SYSTEMS Chapter 13 VIDEO CASES Case 1: IBM:
1 SAS Global Forum 2009 Marty Ellingsworth (iiA) The views expressed by the presenter does not necessarily represent the views, positions, or opinions.
Click to add text IBM Case Manager An Architectural Overview Mike Marin Distinguished Engineer IBM.
MFG Assessment Application: Assessment Criteria and Metrics 1 Performance assessment criteria and metrics may be used as the basis for determining the.
1 E-Strategy, Internet Communities, and Global EC.
Notes accompany this presentation. Please select Notes Page view. These materials can be reproduced only with written approval from Gartner. Such approvals.
WORKSHOP II: Planning for Course Redesign Review of Workshop Homework Innovative Course Redesign Practices Break-out Sessions: Redesign Plans Preparing.
Measuring innovation TRAINING WORKSHOP ON SCIENCE, TECHNOLOGY AND INNOVATION INDICATORS Cairo, Egypt September 2009.
Page 1 FEB-2010 Management information System: ERP What is ERP System Presented to : Dr. Mahmoud Youssef Presented By : Omar Khaled Wael Omar Sherif Aly.
What is Trade Promotions? Created August 5, 2004.
Copyright © 2007 by South-Western, a division of Thomson Learning, Inc. All rights reserved. Developed by Cool Pictures and MultiMedia Presentations PowerPoint.
Addingvelocityraisingqualitycuttingcosts. Opticentre is the first BPO Globalization technology centre of excellence Opticentre helps clients make lasting.
© 2016 SlidePlayer.com Inc. All rights reserved.