1. A Study of the Decision-making Behavior of Markers in E-C Sentence Translation Assessment
Wen Hui
Nie Jianzhong

2. Abstract
Second language testing increasingly used rated tasks in place of objectively scored items. An understanding of the raters’ decision-making behavior is of particular importance for both reliability and validity. This paper is intended to investigate the decision-making behavior of experienced raters with similar background in their assessment of an English-Chinese sentence translation task on the basis of quantitative and qualitative analysis. rating scores and raters’ verbal protocol analysis indicated that raters in this study differed significantly in their decision making processes.
Key words:
English-Chinese sentence translation; Think-aloud verbal protocol; Decision-making behavior; Inter-rater difference; Translation assessment
(1-11)

3. (2-11)

4. Current researches on rater behavior
Bachman et al.
1995
speaking
Brown
writing
Cumming et al
2002
Lumley & McNamara
Lumely
Milanovic et al.
1996
Pollitt & Murray
Weigle
1994
Weigle et al.
2003
张文霞
2004
(3-11)

5. Objective To explore decision-making behavior of experienced raters during the process of translation assessment.

6. Research Design and Methodology
(5-11)

9. Results and discussion
RQ1: How is the students’ E-C sentence translation assessed by the raters
in terms of scores?
Source
Sum of Squares df
Mean Square F
Sig.
Between Groups
1.35
5.00
0.27
0.21
0.95
Accuracy
Within Groups
67.90
54.00
1.26
Total
69.25
59.00
3.40
0.68
0.67
0.65
Expression
55.00
1.02
58.40
Figure 4.1 a Frequency analysis on the category of accuracy
Table 4.3 ANOVA table for the data of table 4.1
Figure 4.1 b Frequency analyses on the category of expression
(8-11)

10. Qualitative judgments
最后评分
RQ2 Are there any remarkable differences among raters in their qualitative judgments to the same translation? RQ3 What elements do these raters focus on while marking these two translation tasks and how do they focus on these elements?
(9-11)
A model of the decision– making process in translation marking
Decide final
mark
Tentative marking
Good
First Impression
Bad
Macro-assessment
Micro-assessment
1.Task completion
2.Legibility
Qualitative judgments
Elements
1.Locating errors Judging gravities of errors
1 In addition to the extremely good or bad performance, raters usually hold different perceptions to the same performance.
2. In E-C sentence translation assessment, rater showed great difference in deciding on the gravities of errors located.

11. E-C translation elements raters focused on in marking
Sentence translation
Discourse translation
criterion-related elements
Comprehension of the source text
Mistranslation
Omission
Incorrect addition
Uncontrolled translation
Lexical choice
Chinese conventions
Fluency in expression
Stylistic inappropriateness
Task completion
Error types
Error gravity
Error frequency
non-criterion elements
Layout
Spelling mistakes
Rater’s affective responses
Comparison with other scripts
criterion-related elements
Comprehension of the source text
Mistranslation
Omission
Incorrect addition
Obscurity of language
Uncontrolled translation
Lexical choice
Sentence structure
Pauses of sentences
Coherence of sentences
Literary grace
Chinese conventions
Fluency in expression
Succinctness of language
Stylistic inappropriateness
Global communicative effect
Task completion
Error types
Error gravity
Error frequency
non-criterion elements
Layout
Spelling mistakes
Rater’s affective responses
Comparison with other scripts
(10-11)

12. incomplete investigation of protocols

13. THANKS ！