Using Open Standards: Save Money and Meet Customer Needs Using Open Standards: Save Money and Meet Customer Needs John Watkins, President, ENLASO

Standards are Hard…

Agenda What are Standards Who uses Standards How to use Standards

What are Standards – Evolution SDLs Trados: SDLs computer-assisted translation software products are the de facto standard in enterprise-wide translation... 1 De facto standards arise from market share: influence through prevalence Standards may evolve from de facto standards through the cooperation of the industry and a relevant standards body TMX is a standard for the exchange of translation memory data (independent of the tool used to create the translation memory) This arises through the cooperation of industry and standards bodies to agree upon the specifications for the transfer of translation memory data among tools (be they in proprietary or open source tools) Today a wide variety of translation memory and related tools use TMX to ensure interoperability 1 Ignacio Garcia & Vivian Stevenson, TRADOS and the Evolution of Language Tools, Multilingual, May

What are Standards – Definitions Standards – Remove barriers for the purpose of performing functions that are within an industry – Are approved and maintained by neutral third parties with input from industry, to avoid being locked into a proprietary solution Open Standards – Do the above – With Transparency in development and maintenance – And availability to the public (with accessible rights) – Synergy with Open Source software – We are fortunate that core Standards in use in the localization industry are, indeed, Open Standards

What are Standards – Benefits Open Standards facilitate the interoperability of language services tools – Freedom to work with a wide variety of tools (many proprietary tools support open standards for compatibility) – Processes are developed independent of the tools Customers and providers can work more easily with the various file types The best linguists can be used regardless of their tool preference Consequently – Tools are not constrained – Workflow is easier – Projects can be faster, better, and cheaper

Who Uses Standards – Everyone Customers Eschew proprietary solutions Service Providers Support the wide variety of tools to meet customer and vendor needs Linguists Want flexibility with CAT tool selection Tool Developers Trying to meet everybodys needs

Who Uses Standards – Contribute Size doesnt matter – we can all contribute – Mid-Size Localization Company – Investing in Open Standards Member of GALA Open Standards Initiative Member of OASIS (XLIFF TC) Member of W3C (LT-Web, ITS) Multilingual Web-LT Project (European Commission W3C) – Develop Open Source Tools (Okapi Framework)

Using Standards – Contribute Direct to the Source – OSCAR/LISA -> Disbanded Standards developed by OSCAR under LISA now under the Creative Commons Attribution license – See GALA European Telecommunications Standards Institute (ETSI) Localization Industry Standards (LIS) Industry Specification Group as the successor for the LISA/OSCAR portfolio (TMX, TBX, SRX…): – OASIS (XLIFF, DITA…): – W3C (ITS, MultilingualWeb-LT – ITS 2.0): – European Commission (LT-Web):

Using Standards – Contribute GALA Standards Initiative – OSCAR standards: – Linport (localizaiton kit standardization) – Model Service Elements (localization task standardization) – Coordination and representation QT Launchpad (DFKI - translation quality) W3C MultilingualWeb-LT (W3C - international web standards) ISO TC 37 SC5 SD (ISO translation services) OASIS, Unicode Consortium, OpenTM2, more to come – GALA-Connect (working groups for members): – Quarterly webinars on standards developments: There are lots of ways/places to contribute!

Using Standards 1.Look at an example project 2.Identify Standards involved 3.Use Standards to provide localized files

Using Standards – Examples Example using Four Standards that are stable and work well – Translation memories TMX: Translation Memory eXchange 1 Easily exchange of translation memory among tools – Segmentation SRX: Segmentation Rules eXchange 1 Provide a standard method to describe segmentation rules that are being exchanged among tools – Extracted data ITS: Internationalization Tag Set 2 Used for XML to support the internationalization and localization of XML schemas and documents XLIFF: XML Localisation Interchange File Format 3 To store localizable data and carry it from one step of the localization process to the other, while allowing interoperability among tools 1 See GALA Open Standards: 2 See W3C: 3 See OASIS:

Using Standards – Open Source Open Standards are a logical fit with Open Source tools. We work with the Okapi Framework Project 1 – Rainbow: Toolbox with functions for pre/post processing, file conversion, encoding conversion, QA, etc. – Pensieve: An Okapi TM engine – Ratel: Segmentation editor – And much more 1 See Okapi Framework project site at:

Wordfast TM TMX 1 TMX 2 Trados TM SRX Rules Excel XML File MIF File Translation Memory from Tradosfrom Trados Translation Memory from Wordfastfrom Wordfast Segmentation rulesSegmentation rules for the TMs New version of the documents to translate (from Excel and FrameMaker)ExcelFrameMaker Example Project

You can use the Okapi Framework to: – Manipulate and combine translation memories – Extract text with appropriate filters – Edit segmentation rules and apply them to content – Leverage from TM – Machine translate unmatched text – Create the translation package for the linguists – Rebuild translated files Means to an End

FrameMaker MIF File

Excel XML File

Three Tasks 1.Consolidate client TMs into a single TM 2.Prepare the translation package to send to the linguist 3.Post-process the files for delivery

Translation Memories – TMX TMX (Translation Memory eXchange) is the standard way to store source text (segments) and their corresponding translations Supported by most CAT tools

Wordfast TM – TMX

Trados TM – TMX

Wordfast TM TMX 1 TMX 2 Pensieve TM Rainbow Toolbox Trados TM Combining the two TMs into a single one. Four different tools sharing data through TMX Combine TMs

Three Tasks Consolidate the clients TMs data into a single Pensieve TM 2.Prepare the translation package to send to the linguist 3.Post process the files for delivery

Pipeline (Driven by Rainbow) Excel File MIF File ITS Rules MIF FilterXML Filter Content Extraction ITS rules can be complex, but it provides a clear way for the owner of the source material to specify what needs to be translated. ITS-aware tools can process XML documents without guesswork. XML Extraction – ITS

For XML documents, ITS (Internationalization Tag Set) describes what needs to be extracted and how to extract it W3C MultilingualWeb-LT WG just started to work on the successor of ITS 1.0 Using ITS rules to identify localizable text in the Excel XML documentITS rulesExcel XML document

ITS Rules

Pipeline (Driven by Rainbow) Excel File MIF File ITS Rules Segmentation MIF FilterXML Filter SRX Rules Extraction Sharing segmentation rules is key to sharing TMs Segmentation – SRX

Translation is done at the segment level SRX (Segmentation Rules eXchange) describes where to break or not break the content into segments Having the rules for source segments allows better re-usability of existing TM, giving exact matches Maintain SRX rules with an SRX EditorSRX rulesSRX Editor

Segmentation – SRX Dont break segment after VS. V.S. vs. or v.s.

Pipeline (Driven by Rainbow) Translation Kit Pensieve TM Excel File MIF File ITS Rules Segmentation MIF Filter Pre-translate unmatched from MT Pre-translate from TM Translation Kit Creation XML Filter SRX Rules Microsoft MT Excel XLIFF MIF XLIFF TMXEtc. Extraction Pensieve TM Connector Microsoft MT Connector Translation Kit – XLIFF, TMX

To flow through the translation cycle the extracted content needs to be stored in a common format many tools understand XLIFF (XML Localisation Interchange File Format) is a standard way to represent extracted data TMX files with all the translation candidates found

Translation Kit – XLIFF, TMX

Three Tasks Consolidate the clients TMs data into a single Pensieve TM Prepare the translation package to send to the linguist 3.Post process the files for delivery

Pipeline (Driven by Rainbow) Translation Kit Excel File MIF File Translator Kit Filter Translation Kit Post- Processing XML Filter Excel XLIFF MIF XLIFF TMXEtc. Extraction MIF FilterPost-Processing

Translated FrameMaker MIF

Translated Excel XML

Three Tasks Consolidate the clients TMs data into a single TM Prepare the translation package to send to the linguist Post process the files for delivery

Summary We know – More about our standards – Everybody should care about standards – We can (and do) use them today Next Steps – Consider requiring open standards compliance with the tools you use to ensure portability – Get involved in educational opportunities – Support standards initiatives through the organization(s) that best fit your needs

References TMX 1.4b – Translation Memory eXchange ITS 1.0 – Internationalization Tag Set SRX 2.0 – Segmentation Rules eXchange XLIFF 1.2 – XML Localisation Interchange File Format Okapi Framework (open-source & cross-platform) Globalization and Localization Association (GALA provides access to various standards projects)

Questions? John Watkins, President, ENLASO