Presentation is loading. Please wait.

Presentation is loading. Please wait.

QUALETRA “QUALITY IN LEGAL TRANSLATION” WS 4 Testing, Evaluation & Assessment KU Leuven Antwerpen 16-17 October 2014 QUALETRA JUST/2011/JPEN/AG/2975 With.

Similar presentations


Presentation on theme: "QUALETRA “QUALITY IN LEGAL TRANSLATION” WS 4 Testing, Evaluation & Assessment KU Leuven Antwerpen 16-17 October 2014 QUALETRA JUST/2011/JPEN/AG/2975 With."— Presentation transcript:

1 QUALETRA “QUALITY IN LEGAL TRANSLATION” WS 4 Testing, Evaluation & Assessment KU Leuven Antwerpen 16-17 October 2014 QUALETRA JUST/2011/JPEN/AG/2975 With financial support from the Criminal Justice Programme of the European Commission Directorate General Justice

2 Translation Evaluation Translation product (target text, the result of the translation process) Translation process (think aloud protocol, eye tracking, key logging) Translation service (contact with the client, offering the quotation, invoicing, compliance agreements, complaints, etc.) © Hendrik J. Kockaert & Winibert Segers

3 Translation Evaluation Translation product: methods Holistic method Analytical method CDI method (Calibration of Dichotomous Items) PIE method (Preselected Items Evaluation) © Hendrik J. Kockaert & Winibert Segers

4 Translation Evaluation Translation product: methods Research Reports: How do evaluators carry out the following evaluation methods? (1) Holistic (2) Analytical (3) PIE

5 Research Reports Translation Evaluation Holistic & Analytical Methods Research Method A sample translation from English to Spanish of a UK robbery judgment was allocated to ten evaluators who received a translation brief (structured translation specifications) Shortly after submitting their assessment, the evaluators were asked to complete a questionnaire These case studies were carried out by UAH (Holistic method) and DCU (Analytical method)

6 Research Report Translation Evaluation PIE Method Preselected Items Evaluation © Hendrik J. Kockaert & Winibert Segers

7 Case study KU Leuven carried out a case study where 46 level A professional translators from federal ministries in Belgium did a translation from French to Dutch The text contained 56 elements, including punctuation. Seven preselected items were used An intern corrected the translations using the analytical evaluation method, referring to the error categories of the ATA evaluation grid, without implementing the actual ATA format and criteria The standard error of difference between the scores obtained on the basis of the analytical method and the PIE method was 0.177, which is not statistically significant Research Report Translation Evaluation PIE Method

8 Sample Test JUGEMENT CORRECTIONNEL DEMANDE DE MISE EN LIBERTÉ DEBATS Avant l’audition de [PI 1] XXX, le président a constaté que celui-ci ne parlait pas suffisamment la langue française ; Il a désigné YYY, interprète inscrit sur la liste du tribunal ; l’interprète a ensuite prêté son ministère [PI 2] chaque fois qu’il a été utile. A l’appel de la cause [PI 3], le président a donné connaissance de [PI 4] l’acte qui a saisi le tribunal [PI 5] et constaté la présence et l’identité de XXX, dont [PI 6] il a reçu les déclarations [PI 7]. Maître ZZZ, conseil [PI 8] du prévenu, a été entendu [PI 9] en sa plaidoirie. Le ministère public a été entendu en ses [PI 10] réquisitions. Le prévenu a eu la parole en dernier. Le greffier a tenu note du déroulement des débats.

9 PIE: Case Study Statistics: Analytical Score vs. PIE Score © Hendrik J. Kockaert & Winibert Segers

10 PIE: Case Study Statistics: T-Test P value and statistical significance: The two-tailed P value equals 0.1376 By conventional criteria, this difference is considered to be Not statistically significant Confidence interval: The mean of Group One minus Group Two equals 0.268 95% confidence interval of this difference: From -0.090 to 0.626 Intermediate values used in calculations: t = 1.5151 df = 40 standard error of difference = 0.177 © Hendrik J. Kockaert & Winibert Segers

11 Conclusion: No statistically significant difference between analytical scores and PIE scores PIE proves a justifiable evaluation method objectivity transparency equality © Hendrik J. Kockaert & Winibert Segers Research Proposals Translation Evaluation PIE Method

12 Activities Translation Evaluation ECQA Test Design ECQA Test on three successive stages: (1) MC questions on legal knowledge (specifically EDs and EAWs) (2) MC and open questions on legal language (phraseologies, terminology, style, register), possibly including a recognition test (to recognise whether a proposed translation is correct or not); after passing each of the two previous stages (3) A translation test corrected with PIE

13 Survey disseminated, analysed and survey report Legal Translation Product Quality Assurance (1) Three research reports drafted (holistic, analytical and PIE method) (2) Strategy and methods have been decided for reporting on the evaluation methods (3) Submission of “Objective Translation Evaluation through PIE” (KU Leuven), 1 st International Young Researchers’ Conference on Translation and Interpreting (UAH, 7-8/11/2013) Deliverables

14 No.Deliverable name/type (a)Format (b)Target group (d) 1 Survey on the testing, assessment and evaluation on the current legal translation practices in criminal proceedings in the EU (product, translator and service/process quality assurance) Websites and manuals (updatable and online information on product, translator and service/process quality assurance) Translator trainers Legal practitioners Professional associations Deliverables

15 When you evaluate legal translations, how important do you find the following?

16 Which evaluation method do you find most suitable for legal translations?

17 How important do you find the following?

18 Which are the most important issues in evaluating legal translations?

19 How can the evaluator justify the evaluation?

20 Test Formats Three different testing formats: Translation test Revision test Recognition test

21 Certification Procedure Prerequisite Only European Master’s in Translation Network (EMT) graduates may apply to take the ECQA certification examination.

22 Examination The examination consists of 3 stages A candidate must pass each element of each stage before passing to the next stage A candidate who fails one or more elements of a stage only has to repeat the failed element(s)

23 Examination The 3 stages of the examination are: Multiple choice questions: LTiCP.U1 Legal Knowledge U1.E1 Criminal law and procedure U1.E2 Monolingual legal terminology (in both languages) Internet access is not permitted at this stage Open questions: LTiCP.U2 Professional aspects & LTiCP.U3 Instrumental competence U2.E1 Professional practice U2.E2 Professional conduct U3.E1 Information acquisition U3.E2 Legal terminology management Internet access is not permitted at this stage Translation: 2x250-word texts, one for each element of LTiCP.U4 Translation U4.E1 Translation of essential documents (Directive 2010/64/EU) U4.E2 Translation of European Arrest Warrants Internet access is permitted at this stage

24 ECQA Skills Card Skill Card Type of Examination LTiCP.U1Legal Knowledge U1.E1Criminal law and procedureMCQ U1.E2Monolingual legal terminology (in both languages)MCQ LTiCP.U2Professional aspects U2.E1Professional practiceOpen Questions U2.E2Professional conductOpen Questions LTiCP.U3Instrumental competence U3.E1Information acquisitionOpen Questions U3.E2Legal terminology managementOpen Questions LTiCP.U4Translation U4.E1Translation of essential documents (Directive 2010/64/EU)Translation U4.E2Translation of European Arrest WarrantsTranslation

25 Translation Test maximum 2x250-words PIE method

26 Test Format PIE Items Report a lost or stolen passport [1] You must report the loss or theft of your passport as soon as possible, even if you don’t want to replace it immediately. You’ll need to complete a Lost or Stolen Notification form [2]. Find out how to get the form and what to do if your passport is lost abroad. If your passport is lost or stolen in the UK [3] You must report the loss or theft of your passport to the Identity and Passport Service [4] (IPS) [5]. This will reduce the risk of anyone else using your passport or your identity. To report your passport lost or stolen, fill in and sign a Lost or Stolen (LS01) [6] Notification form [7] and return it to IPS [8]. The address is on the form. You should [9] report all passport thefts to the police. You’ll need the crime reference details for the LS01 form [10].

27 Revision Test maximum 10 items maximum score = 10

28 Test Format Candidates have to revise the English translation; have to recommend corrective measures for elements in the English translation that are not correct. Vérifiez la date du document. Check the data of the document. A candidate can make three types of erroneous revision: 1)The candidate revises an element in the translation that was correct, and proposes, for instance, to replace ‘Check’ by ‘Control’ or by ‘Verify’; 2)The candidate does not revise an element in the translation that was not correct: in our example the word ‘data’, which has another meaning then the French word ‘date’ (‘date’); 3)The candidate revises an element in the translation that was not correct, but her/his revision is not correct, and proposes, for instance, to replace ‘data’ by ‘information’.

29 Recognition Test maximum 10 items maximum score = 10

30 Example Candidates have to indicate if the proposed English translation is correct or incorrect. Le témoignage de l’épouse de l’accusé n’est pas crédible. The testimony of the wife of the accused is not credible. correctincorrect

31 Test Format PIE Translation, Revision, Recognition After passing the three tests (translation, revision and recognition), the candidate will obtain a score with a maximum of thirty The proposed testing approach is a better guarantee for objectivity than the traditional translation test The score of the candidate does not depend on the personal appreciation and interpretation of the evaluator Another advantage of this testing approach is the time-saving aspect: The time you spend to develop the three tests is largely compensated by the short evaluation time

32 Validity of Tests

33

34 ECQA Overview Question Types Sample Learning Elements to be covered consistent with skill card Type of ExaminationDeveloper CLTiCP.U1 Professional competence U1.E1 Professional practiceOpen Questions U1.E2 Professional conductOpen Questions

35 ECQA Questions Samples LTICP.U1.E1.PC2: The candidate has a sound understanding of criminal procedure in the legal systems involved (e.g. levels of jurisdiction, legal structures, institutions, settings, parties). 1.A person who __________________ a crime may be sentenced to imprisonment. Fill in the blank with one of the four choices: Adoes Bis convicted Crenders Dperforms

36 ECQA Questions Samples 1.1.LTICP.U1: Legal knowledge 1.1.1.LTICP.U1.E1: Criminal law and procedure CLTICP.U5.E1.PC5: The candidate masters the main domains and sub-domains of criminal law, especially the most frequent offences in essential documents and European Arrest Warrants, e.g. drugs, fraud and theft. 1.From the following list, choose one example of an offence against the person: Atheft Bsmuggling Cassault Dpossession

37 ECQA Questions Samples LTICP.U1.E1.PC3The candidate has a general awareness of current legal issues and their development in the relevant countries. 1.From the following list, choose one example of a current legal issue of special concern in the EU cross-border criminal context: AIllegal sale and distribution of food products BTrafficking in human beings CElectronic surveillance by state security services DNon-payment of fines

38 ECQA Questions Samples LTICP.U1.E1.PC4: The candidate is familiar with the EU directives on legal translation. 1.A criminal suspect has the right to translation of (choose one of the following): AQuestions from the judge BProsecution statements CTranscripts of evidence given in court by witnesses DOther


Download ppt "QUALETRA “QUALITY IN LEGAL TRANSLATION” WS 4 Testing, Evaluation & Assessment KU Leuven Antwerpen 16-17 October 2014 QUALETRA JUST/2011/JPEN/AG/2975 With."

Similar presentations


Ads by Google