1. Applying Machine Translation Metrics to Student-Written Translations Lisa N. Michaud Computer Science Department Merrimack College North Andover, Massachusetts, USA Patricia Ann McCoy Language Department Universidad de las Americas Puebla Puebla, Mexico

2. Evaluating learner translation Michaud and McCoy2 Translating ancient texts and doing what MT cannot do yet

3. Criteria for judging translations fluency (is it well-formed?) fidelity (does it convey original meaning?) (Hovy et al., 2002) 3Michaud and McCoy

4. Multiplicity of translations  In each one of these jobs the professor could have agreed to work 6 hours a day and therefore would not be surpassing the working day hour limit.  In each one of these jobs the teacher could have agreed to work 6 hours per day and therefore he wouldn't be bound by the limits of the working day.  In each of these examples the teaching could have been arranged so that he/she works six hours a day and would not be affected by any workday limitations.  In both of these jobs the professor could have agreed to work six hours daily and therefore he wouldn't be affecting his work shift limit. 4Michaud and McCoy

5. Multiplicity of translations  In each one of these jobs the professor could have agreed to work 6 hours a day and therefore would not be surpassing the working day hour limit.  In each one of these jobs the teacher could have agreed to work 6 hours per day and therefore he wouldn't be bound by the limits of the working day.  In each of these examples the teaching could have been arranged so that he/she works six hours a day and would not be affected by any workday limitations.  In both of these jobs the professor could have agreed to work six hours daily and therefore he wouldn't be affecting his work shift limit. 5Michaud and McCoy

6. BLEU 6Michaud and McCoy Hypothesis Multiple References

7. TERp 7Michaud and McCoy Hypothesis Single Reference PHRASAL EQUIVALENCE or SYNONYM SHIFTSUBSTITUTION INSERTION SAME STEM

8. TERp alignment and tags 8Michaud and McCoy

9. Student translation corpus 9Michaud and McCoy

10. Sentence boundaries 10Michaud and McCoy EXPERT: Due to the fact that previous studies have shown that POMC neurons are linked to obesity, the scientists believe the findings might also lead to developing treatments to control obesity and other metabolic disorders. STUDENT: Past studies have shown that the POMC neurons are involved with obesity. The scientists believe this finding could also help develop treatments for the obese and other metabolic disorders. ORIGINAL: Estudios en el pasado han demostrado que las neuronas POMC están involucradas en la obesidad, por lo tanto los científicos creen que el hallazgo también ofrece la posibilidad de desarrollar tratamientos para controlar la obesidad y otros trastornos metabólicos.

11. Does TERp agree with an expert? Michaud and McCoy11 Instructor Scores vs Inverted TERp 650 sentences (22%) Pearson Correlation r = 0.232236

12. Score distribution 12Michaud and McCoy

13. Instructor rubric (original) Michaud and McCoy13 10Excellent 9Good 8Satisfactory 0-7 Deficient

14. TERp scores over the semester 14Michaud and McCoy

15. Evaluating TERp tags (pilot) Michaud and McCoy15

16. Future work Michaud and McCoy16

17. Instructor rubric (revised) Michaud and McCoy17 100 Excellent 90Good 80Satisfactory 0-70 Deficient

18. Modifying the TERp Score 18Michaud and McCoy Hypothesis Single Reference PHRASAL EQUIVALENCE or SYNONYM SHIFTSUBSTITUTION INSERTION SAME STEM

19. Recognizing false cognates 19Michaud and McCoy Hypothesis Single Reference SUBSTITUTION cynical brazen Source cínico

20. Extracting mistranslation pairs Michaud and McCoy20 SPANISH DICTIONARY ENGLISH DICTIONARY cynical cínico cynical brazen zonazone