1. Testing Writing
Rio Darmasetiawan

2. Testing problem
We have to set writing tasks that are properly representative of the population of tasks
The tasks shoudl elicit valid samples of writing
It is essential that the samples of writing can and will be scored validly and reliably

3. Representative tasks Specify all possible content
In order to judge whether the tasks we set are representative of the tasks that we expect students to be able to perform, we have to be clear at the outset just what these tasks are that they should be able to perform.

4. Representative tasks
Include a representative sample of the specified content
The more tasks (within reason) that we set, the more representative of a candidate's ability (the more valid) will be the totality of the samples (of the candidate's ability) we obtain.

5. Elicit a valid sample of writing ability
Set as many separate tasks as is feasible
We have to offer candidates as many 'fresh starts' as possible, and each task can represent a fresh start. By doing this, we will achieve greater reliability and so greater validity

6. Elicit a valid sample of writing ability
Test only writing ability and nothing else
This advice assumes that we do not want to test anything other than the ability to write. Therefore, for the sake of validity, we should not set tasks which measure other abilities such as creative, imaginative, etc.

7. Elicit a valid sample of writing ability
Restrict candidates
Writing tasks should be well defined: candidates should know what is required of them, and they should not be allowed to go too far astray. One last thing to say about tasks is that they should not only fit well with the specifications, but they should also be made as authentic as possible

8. Ensure valid and reliable scoring
Set tasks which can be reliably scored
Set as many tasks as possible
The more scores for each candidate, the more reliable should be the total score.
Restrict candidates
The greater the restrictions imposed on the candidates, the more directly comparable will be the performances of different candidates
Give no choice of tasks
Making the candidates perform all tasks also makes comparisons between candidates easier
Ensure long enough samples
The samples of writing that are elicited have to be long enough for judgments to be made reliably

9. Ensure valid and reliable scoring
Create appropriate scales for scoring
Holistic scoring
Holistic scoring involves the assignment of a single score to a piece of writing on the basis of an overall impression of it. This kind of scoring has the advantage of being rapid.
Analytic scoring
Methods of scoring which require a separate score for each of a number of aspects of a task are said to be analytic.

11. Ensure valid and reliable scoring
Calibrate the scale to be used
Any scales which is to be used should first be calibrated, meaning collecting samples of performance collected under test conditions, and covering the full range of the scales.
Select and train scorers
They should be sensitive to language, have had experience of teaching writing and marking written work. It is also helpful if they have had training in testing.

12. Ensure valid and reliable scoring
Follow acceptable scoring procedures
Each task of each student should be scored independently by two or more scorers (as many scorers as possible should be involved in the assessment of each student's work. Senior member of the team should collate scores and identify discrepancies in scores awarded to the same piece of writing. Nevertheless, once scoring is completed, It is useful to carry out simple statistical analyses to discover if anyone's scoring is unacceptably aberrant