1. Third IATIS Regional Workshop – Western Balkans Translator and Interpreter Training September 26, 2014 Faculty of Philosophy, Novi Sad

2. A Corpus-based Analysis of Translator Errors Influenced by CAT Tool Design: an experiment in back-translation by Aleksandar Kavgić & Randall A. Major

3. 1. INTRODUCTORY REMARKS

4. 1. Introductory Remarks CAT Tools – Trados Parallel Corpus – LF Aligner CATMA – “Computer Aided Textual Markup and Analysis”

6. Stop! No one really talks like this. Discourse should be: – a cohesive, interwoven set of connected utterances – sentences should be an archipelago interconnected with bridges built from anaphoric references and links.

9. Training College Students to Be Translators Undergraduate courses in translating various kinds of texts Master level course in Information Technology and Translating Question: why would students who produce (relatively) seamless translations on an undergraduate level, start producing choppy, disconnected ones in their IT&T course?

10. Could it be a problem of SOFTWARE?

12. 2. BACKGROUND

13. 2. Background SDL Trados - the industry standard software for producing coherent, terminologically consistent translations. Combines a textual framework with Translation Memory and Terminology Management

14. Product Development and Changes In 2011, Trados changed its long-running format from ST on top, TT on bottom to a side-by-side format.

15. Old Format (1)

16. Old Format (2)

17. New Format (1)

18. New Format (2)

21. Discussion (1) Some people disliked the change: It takes the focus off the Translation Unit being worked on, and it’s got a distracting amount of external information all around the work area. Some people liked the change: Finally you can see the larger context, both before and after, without scrolling around the document. And now you know at glance how much you still have to do to finish!

22. Discussion (2) Three of the great things about Trados: – Absolute terminological consistency – Repetitive translation is made easier (catalogues, manuals, instruction booklets, diplomas, etc.) – Nothing gets skipped. Every TU must be translated in some way. That makes it good even for fiction!

23. Discussion (3) Since we were teaching them to use the old, Trados 7, is it possible that their choppy, disconnected FICTION translations were being caused by the TU focus? Experiment to find out!

24. 3. METHODOLOGY

26. 3.1. Corpus: general Source text: Three unpublished short stories written in Serbian by our, at the time, undergraduate student (now MA track)

27. 3.1. Corpus: general Source text: Three unpublished short stories written in Serbian by our, at the time, undergraduate student (now MA track) No copy-paste “translating” using online, previously made translations was possible → authentic student translations.

28. LF Aligner -

29. Olifant: 3 column “TM”

30. 3.1. Corpus: general Number of TU (~sentences):1924 Number of WORDS in SOURCE:29116 Number of WORDS in TARGET:32190 Number of WORDS in BACK TRANSLATION:29057

31. 3.1. Corpus: tagging scheme Tag Set Relevant source words/phrasesTranslation ErrorsBack translation Errors

32. 3.1. Corpus: tagging scheme TRANSLATION ERROS Missing Element Word Order/Focus Error: unmarked > marked Word Order/Focus Error: marked > unmarked Punctuation Error Word Choice Error (word or phrase) BACK TRANSLATION ERRORS Missing Element Word Order/Focus Error: unmarked > marked Word Order/Focus Error: marked > unmarked Punctuation Error Word Choice Error (word or phrase)

33. 3.1. Corpus: tagging scheme

34. INTER-ANNOTATOR AGREEMENT Annotation was conducted by TWO ANNOTATORS: – Randy, a native speaker of (American) English – Saša (i.e. Aleksandar), a native speaker of Serbian – this enabled us to have a 360° view of the process Inter-annotator agreement was achieved by joint annotation of the first section of the corpus: 1 disagreement in 20 tags (~95%)

35. 3.1. Corpus: tagging scheme

36. 3.2. Corpus: tagged version TAG RAW FREQUENCY TE: Missing Element154 BTE: Missing Element183 TE: WO/FE: unmarked > marked24 BTE: WO/FE: unmarked > marked72 TE: WO/FE: marked > unmarked38 BTE: WO/FE: marked > unmarked72 TE: Punctuation Error19 BTE: Punctuation Error91 TE: Word Choice (word or phrase)558 BTE: Word Choice (word or phrase)635 Remember, the corpus contained 1924 translation units (TUs).

37. 3.2. Corpus: tagged version TAG FREQUENCY (per 100 TUs) TE: Missing Element8 BTE: Missing Element9.51 TE: WO/FE: unmarked > marked1.25 BTE: WO/FE: unmarked > marked3.74 TE: WO/FE: marked > unmarked1.98 BTE: WO/FE: marked > unmarked3.74 TE: Punctuation Error0.99 BTE: Punctuation Error4.73 TE: Word Choice (word or phrase)29 BTE: Word Choice (word or phrase)33

38. 4. FINDINGS

39. TAG FREQUENCY (per 100 TUs) TE: Missing Element8 BTE: Missing Element9.51 TE: WO/FE: unmarked > marked1.25 BTE: WO/FE: unmarked > marked3.74 TE: WO/FE: marked > unmarked1.98 BTE: WO/FE: marked > unmarked3.74 TE: Punctuation Error0.99 BTE: Punctuation Error4.73 TE: Word Choice (word or phrase)29 BTE: Word Choice (word or phrase)33 4.1. General trends: (un)markedness

40. Markedness in translation? In this research we defined markedness as: – Any deviation from the manner of presenting information in a sentence which results in a change of the natural information focus in a language, or in amplification and/or down-toning of the naturally expected meaning – This generally includes changes of word order, as well as omissions and additions in translation that can have this effect

41. 4.1. General trends:  unmarkedness Konačno je ona velika količina besa prema društvu i Ijudima koju je osećala bila dovedena na neki razuman nivo. ↓ Finally that great amount of anger that she felt towards the society and people was brought down to a sensible level. ↓ Bes koji je osećala prema društvu i Ijudima je konačno sveden na podnošljiv nivo.

42. 4.1. General trends:  unmarkedness Da nema poeziju, mislila je, ne bi imala smisla da živi. ↓ If she hadn’t had poetry, she thought, her life would make no sense. ↓ Mislila je da za nju život ne bi imao nikakvog smisla da joj nije bilo poezije.

43. 4.1. General trends:  markedness Konobar je pogledao sa ču đ enjem kada je sišla niz stepenice, prvi put primetivši koliko je ona u stvari privlačna. ↓ When she was coming down the stairs the waiter looked at her amazed, noticing for the first time how attractive she really was. ↓ Dok je silazila niz stepenice, konobar je pogleda u čudu, videći po prvi put koliko je ustvari privlačna.

44. 4.2. General trends: =equivalence General consensus in literature on back translation: back translation can never fully reflect the original 8% of back translations were exact reproductions of the original, though most are simple sentences, but a number of them are complex

45. 4.2. General trends: =equivalence Bila je potpuno prazna. ↓ She was completely empty. ↓ Bila je potpuno prazna.

46. 4.2. General trends: =equivalence Irena se plašila da je drži u naručju. ↓ Irena was scared to hold her in her arms. ↓ Irena se plašila da je drži u naručju.

47. 4.2. General trends: =equivalence Upisao je elektrotehniku, jer je oduvek voleo prirodne pojave, fizičke procese i elektroniku. ↓ He enrolled on the Faculty of Electrical Engineering, because he always liked natural occurrences, physical processes and electronics. ↓ Upisao je elektrotehniku, jer je oduvek voleo prirodne pojave, fizičke procese i elektroniku.

48. TAG FREQUENCY (per 100 TUs) TE: Missing Element8 BTE: Missing Element9.51 TE: WO/FE: unmarked > marked1.25 BTE: WO/FE: unmarked > marked3.74 TE: WO/FE: marked > unmarked1.98 BTE: WO/FE: marked > unmarked3.74 TE: Punctuation Error0.99 BTE: Punctuation Error4.73 TE: Word Choice (word or phrase)29 BTE: Word Choice (word or phrase)33 4.3. The old axiom is wrong?

49. Carry-over effect? or The old axiom of translation is wrong?

50. Actually, our data supports claims made by Nike Pokorn…

51. 4.3. The old axiom seems to be wrong English translations are, on the whole, good – not extremely good – nor excellent – but better than the back translation As Pokorn says: the quality of the translation, its fluency and acceptability in the target language environment depend primarily on the as yet undetermined individual abilities of the particular translator, his/her translation strategy and knowledge of the source and target cultures, and not on his/her mother tongue or the direction in which s/he is translating.

52. 4.4. Punctuation (1) Run-on sentences Sentence fragments Interesting: Dialogue! – Lack of knowledge of English conventions causes problems in the back-translation “Reality is ruthless, the outside world crashes with no mercy.” I thought. Stvarnost je nemilosrdna, spoljanji svet pada bez milosti. Mislila sam. or “It must have been his brother’s bag.” I thought. -Mora da je to bila torba njegovog brata. Mislila sam.

53. 4.4. Punctuation (2) Serbian style manuals allow two kinds of punctuation in dialogue: – „text ‟ – or a dash (-) at the beginning of a quotation. [problem with the second type: where does the quotation end?] Here’s an example of what happens:

54. 4.4. Punctuation (3) -Šta bi bilo, moj Dragane. U ovom mestu se ništa ne dešava, a i ako se desi, svi odmah saznaju jer rekla-kazala funkcioniše brže od bilo koje vrste transfera informacija. (...) “What could be new, my Dragan?” Nothing happens in this place. However, if something happens, everyone gets to know it immediately. Hearsay functions faster than any other means of transferring information. (...) „Šta da ima novo, moj Dragane?“ Ništa se ovde ne dešava. A i da se desi, svi će odmah saznati. Ovde rekla-kazala radi brže od drugih sredstava obaveštavanja. (…)

55. 4.4. Punctuation (4) This is a case where CONTEXT is really important for deciding when to end a quotation. This is a case where the old Trados SURELY contributes to errors in translation

56. 4.5. General trends:  word choice The focus of this research was on issues in back translation caused by changes in word order, information organization, etc. possibly influenced by the use of Trados. however some word choice errors may have been caused by Trados, as well focus on a single TU may cause errors in deducing the correct meaning of polysemeous words

57. 4.5. General trends:  word choice Smatrala je da su svi ljudi (=human beings) slepi. ↓ She considered all men (=human beings) blind. ↓ Smatrala je sve muškarce (=males) slepima.

58. 4.5. General trends:  word choice Svako veče bi je slušali bez reči, kao omamljeni (=dazed) (by her poetry). ↓ Each night they would listen to her, not uttering a word as if they were drugged. ↓ Svake večeri bi je slušali, ne prozboreći ni reč, kao da su drogirani.

59. 4.5. General trends:  word choice Imala je dvadeset i tri godine i imala veoma lepo lice i crnu kosu koja joj je nestašno padala u loknama.(=dark hair, cascading in unruly locks.) ↓ She was twenty-three years old and she had a very beautiful face and restless curly dark hair. ↓ Imala je 33 godine, prelepo lice i nemirnu, kovrdžavu, crnu kosu.

60. 4.5. General trends:  word choice Upotpunjavali su se, uzdizali (=they uplifted one another). ↓ They were complementing each other – they were flying. ↓ Jedno drugo su upotpunjavali – letili su.

61. 5. CONCLUSIONS

62. Conclusions - General The number of errors we looked at did not rise to our (wild?) expectation of 50% caused by forced TU focus. – closer to 30% However, our hypothesis proved to be at least partially true.

63. TE: 41.22 / BTE: 54.72 TAG FREQUENCY (per 100 TUs) TE: Missing Element8 BTE: Missing Element9.51 TE: WO/FE: unmarked > marked1.25 BTE: WO/FE: unmarked > marked3.74 TE: WO/FE: marked > unmarked1.98 BTE: WO/FE: marked > unmarked3.74 TE: Punctuation Error0.99 BTE: Punctuation Error4.73 TE: Word Choice (word or phrase)29 BTE: Word Choice (word or phrase)33

64. Conclusions - General In our further teaching of students at the department, in terms of TU focus, we need to INSIST on Newmark’s “every word in the ST must be accounted for, though not necessarily translated”. (“no man left behind translating”)

65. Conclusion - Trends Our students tend to “neutralize” the ST in their TTs (back translation): – changing the markedness of the ST sentence – not paying careful attention to lexical choice of polysemous words: choosing the most common meaning regardless of the context – omit, misuse and/or misinterpret punctuation marks – pay much more attention to translations into L2 than into L1

67. Conclusions The observed trends may be: – due to Trados tunneling their vision to a single TU – due to something else

69. Conclusions for translator training

70. Pay attention to CONTEXT