2 Cognitive aspects of translation and post-editing: empirical investigations Erik Angelone, Isabel Lacruz and Greg Shreve KENT STATE UNIVERSITY
3 KeTra’s Three Periods Early: 2007-2009 Recent: 2010-2013 Future: 2013-onwards
4 Early period (2007-2009) Erik Angelone & Greg Shreve Observational studies using Think Aloud Protocols and Screen RecordingsTwo main projects
5 Uncertainty Management and Problem Solving in Translation Questions investigated:1. When, where, and how are problem solving bundles employed in managing uncertainty while translating?2. How do the degree and contour of metacognitive activity associated with the bundles vary between professional and non-professional translators?3. In what situations are the metacognitive activities associated with uncertainty management accounted for by TAPs, and in what situations are they not?
6 Main FindingsThe professionals engaged in sequential, uninterrupted problem recognition - solution proposal - solution evaluation bundling to a greater extent than the students.
7 Problem-Solving Strategies and Error Mitigation in Translation Questions investigated:1. What impact does metacognitive bundling have on overall translation quality, as indicated by the type and frequency of errors in the target text?2. Are certain forms of uncertainty management behavior more conducive than others to limiting translation errors?3. What is the correlation between the textual level of overall uncertainty management and the textual level of translation errors?4. Do apparent process-oriented expertise effects in uncertainty management behavior result in product-oriented expertise effects, i.e., improved translation quality?
8 Main FindingsMetacognitive bundling (uninterrupted problem recognition - solution proposal - solution evaluation sequences) resulted in fewer overall errors and fewer errors of each textual level type.
9 Recent Period (2010-2013) Erik Angelone, Isabel Lacruz, Greg Shreve Experimental studies using eye-tracking, key-logging, and reaction time methodologies
10 Cognitive Effort, Syntactic Disruption, and Visual Interference in a Sight Translation Task Questions investigated:1. Does syntactic complexity of the source text affect sight translation performance?2. Does syntactic complexity of the source text affect written translation?3. Is sight translation more effortful than bilingual reading?
11 Methodologies Eye-tracking Keystroke logging One authentic version and one modified version of each text11 translators took part in these experiments
12 Main FindingsSyntactic complexity affected sight translation: sight translation was more effortful in the syntactically complex texts than in the non-complex textsSyntactic complexity did not affect written translationSight translation was more effortful than bilingual reading
13 Sight Translation and Speech Disfluency: Performance Analysis as a Window to Cognitive Translation ProcessesQuestions investigated:1. Is there a relationship between oral disfluency phenomena and effort indicators in eye-tracking metrics?2. What types of linguistic problems result in disfluencies?
14 Main FindingsPromising methodology: Triangulation of eye-tracking data and disfluency dataThere seems to be an association between speech disfluencies and cognitive effort in sight translationDisfluencies signal production problems resulting from lexical, syntactic, and translation-strategic problems
15 Efficacy of Screen Recordings in the Other-Revision of Translations Questions investigated:1. Are screen recordings efficacious to assist other- revision?2. Are screen recordings more efficacious than IPDR records in other-revision?3. Is there a difference in efficacy depending on error type?
16 Methodologies8 Spanish/English translators and 4 German/English translatorsFirst session: Each participant translated one text using Screen Recordings and another one using IPDRSecond session: Each participant edited one text translated by another translator and had access to either the Screen Recordings or the IPDRThe dependent variable was proportion of errors mitigated in the edited textsErrors were classified by type
17 Main FindingsScreen recordings were more effective than IPDR in mitigating errors in other-revision: The percentage of errors left in the edited text was significantly lower for SR than IPDRThis effect was significant for both German and Spanish translatorsSR were more efficacious only in four of the six error categories
20 Pauses and Cognitive Effort in Post-Editing Questions investigated:1. Do pause patterns in keystroke log reports indicate cognitive effort in post-editing? If so, can these patterns be quantified by metrics suitable for automatic analysis?2. Can levels of cognitive effort in post-editing be detected through pause timing data generated from keystroke log reports?3. Can levels of cognitive effort in post-editing be detected through word timing data generated from keystroke log reports?4. Can levels of cognitive effort in post-editing be detected through pause count data generated from keystroke log reports?
21 Methodologies Keystroke logging Post-editing of English-Spanish and Spanish-English MTAssessment of actual cognitive effort from actions detected, rather than by inferences made from objective assessments of cognitive demandTwo observational studies: one pilot study and one follow- up using three participants with different language and professional characteristics
22 Main FindingsClusters of short pauses associated with high levels of cognitive effortAverage pause ratio (= average pause time/average word time) decreases as cognitive effort increasesAverage pause time decreases as cognitive effort increasesAverage time to process a word increases as cognitive effort increasesPause to word ratio (= number of pauses/number of words) increases as cognitive effort increases
23 Future DirectionsExperimental studies into the nature of cognitive effort in post- editing and translationComparison of editing performance and behavior of non- translator subject-matter-experts and professional non-subject- matter-expert translatorsComparisons of MT revisions and human translationsHow is the mind of the translator different from that of the bilingual/monolingual language user? How are mental representations for language different for a translator?
24 Future Directions Both experimental and observational studies using Eye-tracking in combination with keystroke loggingReaction time paradigms such as:Lexical decision tasksWord translation tasksTranslation verification tasks both for words and for sentencesPaper and pencil tasks such multiple choice testsScreen RecordingsDual task paradigms