We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byDwayne Powell
Modified about 1 year ago
100 years of progress and innovation © 2011 IBM Corporation The value of Post Editing - IBM Case Study Frank X. Rojas, Jian Ming Xu, Santi Pont Nesta, Álex Martínez Corrià, Salim Roukos, Helena Chapman, Saroj K. Vohra June 2011
© 2011 IBM Corporation2 IBM Case Study – MT Post Editing Introduction MT Innovation Process Overview Findings Conclusion / Recommendations
© 2011 IBM Corporation3 IBM World Wide Translation Operations 24 Centers World Wide ~115 Translation Suppliers Process ~2.8 B Words Translate ~0.4 B Words ~ 60 language pairs One Stop Shop for all Translation Services Marketing Material Web Product Integrated Information Publications Legal/Safety/ Contracts Machine Translation Multimedia Francization Cultural Consultancy Centralized DTP Overall End to End Process Management
© 2011 IBM Corporation4 IBM Professional Translation Services Professional Memory 72% 85% Re-Use Unit Cost >50% Reduction Traditional Technology Process Mgmt Human Skill Consistent Quality Standards Global Brand Identity Professional Quality Standards 1 2 3 Future: –Ability to reduce cost using conventional methods reaching limits –Business pressure for additional cost elimination –Looking to MT Technology as next wave to reach business goals
© 2011 IBM Corporation5 - MT portal - Generic crowdsourcing - Text translation services June 2008 Historical Perspective 2006 2007 2008 2009 2011 2012 2010 RTTS introduced in 2006 as platform for speech and text translation, developed by IBM Research 2010 MT piloting Pilot: SPA, ITA, FRE, GER ------------------------------------- New E2E process Partnership: WWTO/n.Fluent 8.6 M words 2011 MT Training Pilot: GER, BPR, JPN, CHS ------------------------------------- MT payment profiles ready 16.0 M words target eSupport (www) “Translate This Page” JPN pilot / rule engine n.Fluent customized with WWTO translation memories eSupport “Translate This Page” switch to n.Fluent RTTS licensed to IBM partners Initial n.Fluent/WWTO Spanish MT pilot ------------------------------------- Improve efficiency of professional translators
© 2011 IBM Corporation6 MT Critical Success Metrics Necessary and sufficient condition to measure success –5.0 M words sampled –Minimum of 3 languages –Net Contribution to ROI by MT Engine: 10% of payable words should be MT –No more than 5% adverse impact to Overall Quality Index –No more than 5% impact to Customer Satisfaction Lack of industry metrics and guidance. –Active research on MT technology... no guidance on operational impacts –A business vacuum existed on how to integrate MT services –No operational process had been defined for MT services
© 2011 IBM Corporation7 IBM’s Watson Q&A computer Google’s autonomous car Technologies to understand and produce natural human speech Instantaneous, high-quality machine translation Smartphones / App phones in the developing world * Andrew McAfee is a principal research scientist in the MIT Sloan School of Business Recent Digital Innovations with Biggest Impact in the Business World*
© 2011 IBM Corporation8 Real-Time Translation Server (RTTS) & n.Fluent Real Time Translation Server (RTTS) IBMs MT Engine RTTS provides machine translation for n.Fluent & other applications APIs allow other applications to access these translation services. Customization tools – Domains, chat-specific models, … Commercially licensed to IBM partners Language Pairs to/from English: n.Fluent IBMs MT translation application Providing machine translation services for: Text, web pages, and documents (Word, Excel, …) Instant Messaging chats (via IM plug-in) Mobile translation application (BlackBerry and others) Enabled with LEARNING via crowdsourcing (internal 450K IBMers) Deployed for eSupport self serving tech support (external) العربية 中文 Deutsch English Français Italiano 日本語 Português Русский Español 한국어
© 2011 IBM Corporation9 - MT portal - Generic crowdsourcing - Text translation services June 2008 Historical Perspective 2006 2007 2008 2009 2011 2012 2010 RTTS introduced in 2006 as platform for speech and text translation, developed by IBM Research 2010 MT piloting Pilot: SPA, ITA, FRE, GER ------------------------------------- New E2E process Partnership: WWTO/n.Fluent 8.6 M words 2011 MT Training Pilot: GER, BPR, JPN, CHS ------------------------------------- MT payment profiles ready 16.0 M words target eSupport (www) “Translate This Page” JPN pilot / rule engine n.Fluent customized with WWTO translation memories eSupport “Translate This Page” switch to n.Fluent RTTS licensed to IBM partners Initial n.Fluent/WWTO Spanish MT pilot ------------------------------------- Improve efficiency of professional translators
© 2011 IBM Corporation10 TM MT New / Changed 100% Exact Match MT Pre-Process Editing Session MT Post Editing End to End Workflow Upfront & on-going MT tuning via IBM TM professional translations –Professional translation = Best context Matching methods –Traditional TM – breaks down content @ segment level –Machine TM – breaks down segments @ block level using MT models – reconstructs segments preserving formats/mark-up tags MT service level integration TM Pre-Process Shipment English TM Match Analysis CAT Translation 1.Show best choice vs vs 2.Select best choice (Post Edit rules) 3. Commit language TESTING QUALITY MT Model & Trans. = Localization Kit (NLV Folder)
© 2011 IBM Corporation1118-sept.-08 MT Pre-processing TM New / Changed 100% Exact Match Build dynamic, domain specific MT model MT MT initial corpus General parallel training corpus Domain specific parallel training corpus ALL segment “no match segments ” Translation of no match segments Initial MT corpus –done before start of project Localization kit TM MT New / Changed 100% Exact Match
© 2011 IBM Corporation1218-sept.-08 Xxx xxx xx xxx xxx xxx. La aplicación desprotege los archivos antes de exportarlos. Yy yyy yyy TM Editing Environment TM Environment Xxx xxx xx xxx xxx xxx. The application unprotects files before exporting them. Yy yyy yyy Translation Memory 0 - The application unprotects files before exporting them. 1[m] – La aplicación desprotege archivos antes de exportarlos. 2[f 85%] - La aplicación protege los archivos antes de exportarlos TM Environment [Ctrl + 1] Typed Translator options Ignore fuzzy and MT Post edit MT Post edit fuzzy Two Seconds Rule: Translators are trained on several strategies to make a quick choice TM MT
© 2011 IBM Corporation13 Productivity Measurements Start segment –Choose action End segment MT productivity evaluation log (MTeval Log) –N events –Words | Time | Existing Proposal | Used Proposal |... Examine productivity per payment category –SUM(Words) / SUM(Time) –Use of IBM Business Analytic Tool (SPSS) –Trim events that fall into 5% (slowest) and 95% (fastest) percentile 1.accept match [~0 time] 2.edit match [X time] 3.reject match [manual translation] Each event EM : Exact RM : Replace FM : Fuzzy MT : Machine NP : No Proposal A) = “best” Existing Proposal B) = “alternative” Existing Proposal C) = reject all Existing Proposal, 100% human labor
© 2011 IBM Corporation14 Total # events : 2,309 (377+1,932) Total words: 24,150 Total time: 27,362 –3,911 w/ MT match 11,377 w/ MT match –20,239 w/o MT match 15,985 w/o MT match MT impact to productivity –MT : 0.44 words/sec [1777 words / 4071 sec] –NP 0.21 w/ MT match 0.32 w/o MT match Baseline (placebo) MT Leverage : 71.8% [1777 / (1777+697)] Single Shipment EXAMPLE rate(MT) / rate(NP) : 1.37 i.e. Translator can complete 37% more words in the same time. Key metrics
© 2011 IBM Corporation15 MT Impact on Fuzzy Match : 4Q10 Findings When FM & MT matches exist simultaneously Productivity: rate(MT) / rate(NP): a.Case : Translator edits FM b.FM-MT Combined case c.Case: Translator edits MT ** Findings subject to change with additional sampling. Overall –Machine matches not as good as professional (fuzzy) matches –No statistical impact to fuzzy productivity to include MT matches. SPA highest sample case 28.6% 4.4% 57.6% 46.9% FM-MT Pick Rate: 0.00 1.00 2.00 3.00 4.00 5.00 6.00 7.00 8.00 FREGERITASPA Productivity ratio FM FM-MT MT
© 2011 IBM Corporation16 MT Key Metrics: 4Q10 Findings 8.6 M words sampled in real time translation service. SPA : Qualified MT engine 4Q10 ITA : Qualified MT engine 4Q10 FRA : Qualified MT engine 1Q11 While rate(MT) / rate(NP) is high, the findings were not statistically significant in 4Q. GER : Insufficient productivity from MT engine ** Findings subject to change with additional sampling.
© 2011 IBM Corporation17 Overall Savings Assessment Overall savings % –Word savings due to MT efficiency Convert time savings MT payment factor % –MT payment factor X [MT % words + NP % words] Results in less payable words. MT productivity savings drives a overall savings –These are not the same due to MT % distribution. Supply chain has to consider cost of MT services ** Findings subject to change with additional sampling.
© 2011 IBM Corporation18 Pay for MT Words Translated not MT Matches We pay for final results (MT payable words) not MT matches –MT matches considered “opinion” until chosen by a human –Too many opinions & opinions by immature MT models are less efficient. Actual MT payable words have value beyond the specific project –Post Edited words are reused in future and unknown MT context Engine has to deliver consistent MT payable words –Minimum needed to quality an MT engine for compensation High MT productivity[rate(MT) / rate(NP)] High MT leverage[% of MT matches used] –Compensation to be based on MT payment factor
© 2011 IBM Corporation19 Variance across Languages There is no single maturity path when modeling MT engines across many languages. IBM Pilot: each trained MT engine is a unique asset. –Some languages require more modeling/tuning than others. –Language pairs that service “Loose -> Structured” languages are struggling German requires more effort than Spanish Are there limitations to statistical MT engines? –New thinking may need to be explored? Each MT engine will have separate MT payment factors.
© 2011 IBM Corporation20 Perspective of MT Post Edit Pilots Translation Service Hierarchy Professional Translation Services (Professional LSP) Community Translation Services (Controlled Social Crowd) Volunteer Translation Services (General Crowds) Free Services (Individual) Quality / Reliability LOWER HIGHER General Domain Specific internal IBM All IBM external/internal Pubs / UI external (2011 Pilots) internal IBM n.Fluent “machine” WWTO “human” New Memory Assets MT Post Editing has impacts across entire Translation Service Hierarchy
© 2011 IBM Corporation21 1.Professional (Human) memories are the best assets and deliver the highest quality. 2.Professional memories are a key asset for MT success. 3.All Memory assets need to be protected and managed. 4.Flow of memories between Professional and Machine must be properly balanced. 5.Dynamic modeling offers significant advantage over static modeling. 6.Continuous business analytics is needed to optimize machine assets. 7.A single cost model per language is needed, independent of MT services/engines. 8.An aggressive yet cautious approach is warranted to go forward. MT Post Editing Project – Key Lessons MT Post Editing does improve productivity and efficiency of a localization supply chain.
HP Quality Center Overview.
Copying distribution or use of the contents of this document is prohibited without written authorization from SafeHarbor Technology Corporation. Maximizing.
Translation in the 21 st Century Impacts of MT and social media on language services.
LEADSCALE GROUP Partnership Presentation. Contents 1:Introduction 2:How we can help you 3:Enrichment (Case Study) 4: Overview 5:Q&A.
IBM Envisioning The Future Supply Chain © 2011 IBM Corporation.
The Eyeblaster ACM Advertising Campaign Management.
PROFILE Business Day offers outsourcing services in sales. We have extensive experience and expertise in the key to any business and.
21 st -Century Supply Chains Copyright © 2010 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin.
Centralizing Dell Marketing Translations Wayne Bourland Sr. Manager, Global Localization Team.
Introduction to Grid Application On-Boarding Nick Werstiuk
Gabriela Contreras, Continental Airlines Yvan Hennecart, SDL
Company confidential Prepared by HERE Transit Sr. Product Manager, HERE Transit Product Overview David Volpe.
© 2008 IBM Corporation ® Atlas for Lotus Connections Unlock the power of your social network! Customer Overview Presentation An IBM Software Services for.
IMA CIM Overview. IMA Mission “Provide a knowledge-sharing platform for business professionals where proven Internet.
Copyright © 2009 by SDL Tridion. SDL Tridion®, SDL Tridion R5™, BluePrinting™, SiteEdit™ and WebForms™ are trademarks of SDL Tridion Holding B.V. or its.
ROI When Web Content Management Meets Knowledge Management
© 2000 XTRA Translation Services Is MT technology available today ready to replace human translators?
Click View > Slide Master to insert a photo as a background behind the colored boxes. Driving the XLIFF Standard at Microsoft 3 rd International XLIFF.
An innovative platform to allow translation and indexing of internet sites Localization World
FITT Fostering Interregional Exchange in ICT Technology Transfer Communication & Collaboration Tools.
© 2014 IBM Corporation Smarter Workforce Services Business Process Innovation.
Thanks to Microsoft Azure’s Scalability, BA Minds Delivers a Cost-Effective CRM Solution to Small and Medium-Sized Enterprises in Latin America MICROSOFT.
Materials Management BUS 3 – 141 Quality and Specification Leveraging Technical Excellence Week of Aug 31, 2010.
Help communities share knowledge more effectively across the language barrier Automated Community Content Editing PorTal.
High Performance Through Self Service KronosWorks 10/22/2003.
1. THE TRANSLATION NETWORK Case Studies 2 THE TRANSLATION NETWORK6 Intel OSTC uses Drupal for localization for the Tizen initiative into 6 languages.
Click to edit text Presentation Title Presenter’s name Presenter’s Title Date Be Efficient – Accurate – Connected™
© 2007 IBM Corporation IBM Software Strategy Group IBM Google Announcement on Internet-Scale Computing (“Cloud Computing Model”) Oct 8, 2007 IBM Confidential.
Enterprise Alert on Microsoft Azure Fully Automates Critical Incident Communication and Transforms It into an Intelligent, Reliable, and Mobile Experience.
How Global Companies Can Close the Globalization Gap with DITA
Virtual Event Solutions When Webconferencing is not the Right Tool Enterprises are turning to webcasting and virtual events to deliver their message to.
© 2015 IBM Corporation Introducing. © 2015 IBM Corporation 2 SELF-SERVICE Covers a wide spectrum of users Power users Consumers Creators Level of Management.
IBM WebSphere Everyplace Access for Multiplatforms Managing the e-business Customer Experience.
CISB444 - Strategic Information Systems Planning Chapter 5 : IS/IT Strategic Analysis: Determining Future Potential.
Chapter 14 Global Production, Outsourcing and Logistics 1.
Replace with Application Image Metrix Insights Leverages Internet of Things, Big Data to Deliver Actionable Healthcare Intelligence via Azure and Cortana.
Last Updated 1/17/02 1 Business Drivers Guiding Portal Evolution Portals Integrate web-based systems to increase productivity and reduce.
1 Thomas Lippert Senior Product Manager - Mobile What’s new in SMC 5.0.
1 © 2006 Avaya Inc. All rights reserved. Avaya – Proprietary & Confidential. Technology Enabling the Avaya Vision Technology Enabling the Avaya Vision.
Building an efficient pipeline for your bank communication
Confidential, Copyright and Patent Pending, Ping4 Group, Inc. 1 Investor Summary Investor Summary.
Discover the Newest Solution from Expertime: Magento + PimCore Running on Microsoft Azure MICROSOFT AZURE ISV PROFILE: EXPERTIME Expertime works with clients.
Reduce Costs, Gain Control, and Improve Processes Insert Date Insert Client Logo.
1 Lotus Connections Customer Use Cases The following are stories from early customer engagements. They were created to help people understand the many.
What Is Desktop Virtualization? How Does Application Virtualization Help? How does V3 Systems help? Getting Started AGENDA.
Built on Azure, Moodle Helps Educators Create Proprietary Private Web Sites Filled with Dynamic Courses that Extend Learning Anytime, Anywhere MICROSOFT.
1 The Benefits of an SOA in the Contact Center Brian Garr Program Director, IBM Speech Solutions.
© 2008 IBM Corporation ® IBM Cognos Business Viewpoint Miguel Garcia - Solutions Architect.
Digital Site Management for the Product Management Department (Concurrency Corporation)
Logistics and supply chain strategy planning
© 2017 SlidePlayer.com Inc. All rights reserved.