4Why PEMT? Increasing demand for PEMT in the market: increasing volume of short-lived documentsdifferent levels of text quality acceptableThe industry perspective on MT:to lower productivity pricesto publish more contentto publish in more languagesto publish in less timeThe TAUS (2010) survey:52% companies in the US, EU & Asia provide PE services regularly74% of the resources they used are freelance translators
5the big pictureHow does MT affect the translation process and the translator?
6Integrating MT in the translation process Phase 1:Translation MemoriesSourcetext0%translatedTranslation Memory (TM)Hybridx%Phase 2:Machine Translation(MT)Hybridtext100%translated but with MT errorsUn-translated segmentsPhase 3:Post-editingTargettext100%translatedPost-editor
7The role of translators in PEMT The role of PEMT experts:edit the outputselect the adequate corpusclean up the data so the output is more suitable for the customerprovide constant feedback to improve the system’s performancethe role changes as MT improvesThe nature of PEMT projects:large contents of highly repetitive nature, short-lived, internal usepre-editing (at the SL level before MT to avoid ambiguous input to the MY system)post-editing (at the TL level after MT to correct errors in the MT output)
8basic concepts in pemtIs post-editing a translation task or a revision task?
9Post-editing vs. Translation reviewing an MT text against an original text and correcting any errors in order to comply with a set of quality criteria in as few edits as possiblethe set of quality criteria ≠ a personal idea of translation qualityas few edits as possible > to increase the productivityPE vs. Translation:Translation: 1 sourcePE: 2 sources (the original & the raw MT output):reject the MT output & translate from scratch (PE closer to Trans than Rev)correct a lot/a few of errors (PE closer to Rev than Trans)accept the proposed translation as is (PE closer to Rev than Trans)PE should be done by a translator, not a monolingual reviewer!
10Post-editing vs. Revision PE:deals with recurring, predictable errorsMT texts put a strain on the post-editing expert, so PE is more cognitively demandingRevision:checks for random mistranslations or omissionshuman errors more difficult to spot but the texts are easier to readPE & Revision both require specific skills and should be tackled by translators trained & experienced in the task!> 100,000 words / 1 month of full-time post-editing
11the job profileWhat skills and qualities do I need to be a good post-editing expert?
12Skills for post-editing (O’Brien 2002) Degree in translation or related subjectsExpert in the subject area and target languageProficient in the source language and contrastive issuesExperience in technical translation/localizationAdvanced word processing skills, full key proficiency (search&replace)Positive, tolerant and open-minded towards MTConfidence in abilities and technical expertiseRecognition of typical and repetitive MR errorsAbility to use macros and coded dictionariesAdvanced terminology management skillsBackground MT knowledge, types of PE and levels of expected qualityPre-editing skills (controlled language & controlled authoring tools)Programming skills for automatically correcting errors
13MT qualityWhat can I expect from MT and what can clients expect from PEMT?
14Common MT errors What MT errors to expect: Depend on the MT system, the content and the language pair used!error analysis time-consuming but:crucial to improve the MT systemcrucial to raise awareness about the post-editing taskSeveral error classifications exist (Schäffer, 2003):Lexical errors (general vocabulary, terminology, polysemy, idioms)Syntactic errors (sentence analysis, word order)Grammatical mistakes (tense, number, gender, case, punctuation)Errors due to defective input (mistakes in the source language)
15Quality in technical translation and localization Functionalist’s approach to quality:the focus is on the customer’s needs and what they pay forquality is variable and is defined by clients, not the society in generalFit-for-purpose! (not what trained translators would consider the best)Quality of MTstandard MT evaluation measures (BLEU, Meteor, NIST, TER):how close the input is to human quality with a single numbernot very reliable in most translation projectsmanual quality assessment needed!crucial for productivity savings & pricingrandom strings checked for grammar, terminology and format (grades 1-5)very specific client’s quality expectations needed! (rapid/full PE)Quality of PEMT is used to save costs, so revision of PE texts is usually not doneCrucial to strike a balance between speed and the quality of PE
16types of post-editingHow much post-editing should I do?
17Different levels of post-editing No post-editingdirectly published on the internet, with disclaimerRapid post-editingsuitable for short-lived documents needed gisting & internal usemin editing, shortest time possible, min no. of changes, to remove blatant & significant errors, no stylistic changesFull post-editingleading to human quality, required for texts for publicationmax editing, all errors and stylistic changes taken into account (but still in less time than translating from scratch)Criteria:the MT system and language pair usedthe domain and structure of the textthe use of the final text, the desired quality and the type of readersthe volume of translation and the time available
18post-editing guidelines What exactly should I correct and how much?
19General guidelines for PE Language- and project-specific guidelines needed for each project!as short and precise as possible:a description of the MT system and the source text useda description of the quality of MT output and the expected quality of the finished translationscenarios when to discard a useless segmenttypical types of errors that need to be correctedchanges to be avoidedterminology issues
20Guidelines for rapid PE Read the source segment firstRead the MT suggestionMake the necessary changes:Make sure the content of the sentence is accurateIf the terminology is incorrect, don’t spend too much time researchingDon’t post-edit word-order if the sentence can be understood as isDon’t change styleDon’t replace words with a synonymDon’t correct grammar mistakes unless the target sentence doesn’t reflect the meaning of the source sentence
21Guidelines for full PE Always very project-specific Use the MT suggestion if:a large piece of the sentence is correctthe raw MT quality is very good with only minor corrections neededthe raw MT quality is not so good but would still be faster to correct it than to translate from scratchthe MT has the correct meaning and is completely understandableDon’t use the MT suggestion if:the raw MT doesn’t make any sense and it would take longer to correct it than to translate from scratchyou need a more than a few seconds to understand itthere are errors that would require rearranging most of the text
22Examples from the guidelines at Microsoft The 5-10 second evaluation:the maximum time you should spend evaluating the validity of the MT suggestionif it is hard to understand already at the beginning, don’t even read the whole sentence, just proceed to translate from scratch instead.The High-5 & Low-5 rule:When you detect a long sentence, do the following:Read the first 5 words. If it’s good, read on until it’s bad, then stop and copy the correct part and continue to translate and forget about reading on.If the first 5 or 6 words aren’t good, skip to read the last 5 or 6 words. It the last part of the sentence is correct, use it, or just start the whole thing from scratch.If both first 5 and last 5 words are incorrect, do not carry on reading through the middle to try to identify correct MT segments. Just discard the MT suggestion and proceed to translate from scratch.
23Post-editing effort and productivity How hard will post-editing be and how much will I gain?
24Post-editing effortKey element to decide if the use of MT is worthwhile or not (Krings, 2001):Temporal PE effortDoes PEMT save time vs. human translation?Does PEMT save time vs. TM fuzzy matches?Depends of the quality of the raw MT output and type of errors!Cognitive PE effortHow complex and cognitively demanding are the corrections?Obvious mistakes (gender) vs. ambiguous complex syntactic structuresTechnical PE effortDoes PE require to delete, insert, reorder or a all 3?Measuring PE effort:temporal: the easiest to measurecognitive & technical PE: eye-trackers, Translog, Think Aloud Protocols (useful in research, less so in the commercial world)
25Post-editing productivity One of the big unknown factors in PEMT projectsnew field, so no standard metrics existproductivity in PE estimated at 4,000-10,000 words/daymany variables to consider:the quality of raw MT output?the productivity of translators in general?the experience of post-editors?the amount of effort to post-edit fuzzy matches?inconclusive results:early studies: show productivity gains up to 3 times compared to HT (Vasconcellos and Leon 1985)recent studies: productivity gain not always achieved (O’Brien 2006, Guerberof 2008)commercial users: many claim high productivity gains but don’t make their methodology availableTest before you commit!