1. Using Common Testing Techniques Effectively
Writing Multiple-choice, Gap-filling, and Short-answer Items
Hanh Thi Nguyen, Ph.D.
TESL Programs
Learning Assessment Committee member

2. What is a ‘good’ test? An effective test
measures what it is supposed to measure (has validity)
measures consistently (has reliability)
is practical to make, administer, and score (has practicality)
has positive effects on learning (has positive washback)

3. How to meet those criteria?
Validity:
Write test items that correspond to course objectives
Make sure only these objectives are tested
Reliability:
Include not too few items, but not too long a test either
Eliminate distracting factors in the environment
Make sure scoring is done consistently
Make sure candidates are familiar with testing format
Make sure instructions are non-ambiguous
Practicality
Consider time and cost
Sometimes practicality conflicts with validity and reliability, so test what needs to be tested, do not just test what is easy to test
Positive washback
Make sure test can improve teaching
Make sure test has positive impact on learning

4. Common testing techniques
Multiple-choice items
Gap-filling items
Short-answer items
Extended response (essay) items – not covered today

5. Multiple-choice items
1. What is the shape of the earth? Stem Options
A. round key
B. square distractor
C. oval distractor
D. triangular distractor

6. Advantages of MC items Objective testing Scoring is easy
Less dependent on students’ writing skills
Can cover a broad range of contents
Very effective when eliciting a best answer

7. Disadvantages of MC items
Testing only recognition knowledge, no production knowledge
Candidates have a 25% chance to guess right (if 4 options) – hence, “multiple-guess” test
Not always possible to have 3-4 plausible alternatives to correct answer

8. Examples
If I ___ him last year, I ___ him to help me. A. know, ask B. knew, asked C. have known, asked D. had known, would have asked
Key is the longest string – facilitates guessing
Option A not likely to distract

9. Examples
She started to ____, thinking about his death. A. giggle B. complain C. shout D. weep
All 4 options are possible answers.
Need further context to restrict answer.

10. Suggestions for constructing MC items
For the stem:
Use direct questions rather than incomplete sentences.
Present a single question or problem in stem
Make sure to include in the stem the words that are repeated in all options (avoid redundancy)

11. Suggestions for constructing MC items
For the options
Use errors commonly made by students in class as basis for distractors
Use options that say “none of the above”, “all of the above”, “A & B only” sparingly
Avoid having the key being the longest or shortest item
When possible, present options in some logical order (chronological, least to most, alphabetical…)
Use similar format for all options (all are phrases, all are single words, etc.)

12. True/False items
Essentially, T/F items are MC items with only 2 options
Advantages:
Good for testing misconceptions, comprehension
Can be scored quickly
Disadvantages:
Students have a chance to guess the right answer

13. Suggestions for constructing T/F items
Make sure there is a single idea in the stem
Make sure the statement is either absolutely true or false, no exceptions
Avoid using extreme qualifiers (never, always, etc.) whenever possible
Avoid using negative statements
Avoid recycling the exact statement from the text
Avoid using unfamiliar vocabulary

14. Matching items Directions: On the line next to each children’s book in
Column A print the letter of the animal or insect in Column
B that is a main character in that book. Each animal or insect
in Column B can be used only once.
Column A Column B
____1. Charlotte’s Web A. Bear
____2. Winnie the Pooh B. Chimpanzee
____3. Black Beauty C. Cricket
____4. Tarzan D. Deer
____5. Pinocchio E. Horse
____6. Bambi F. Pig

15. Matching items: advantages & disadvantages
Essentially are MC items, in which the answer of one item also depends on the answers of the other items in the matching set
Advantages: Good for testing
Definitions
Classifications
Cause and effects
Problems and solutions
Relationship between 2 entities (e.g., event-date, book-character)
Disadvantages
Items are inter-related
Time consuming for test-takers
Not suitable for all knowledge and learning skills

16. Suggestions for constructing matching items
Make sure all items are of the same topic Directions: Match the following.
1. Water A. NaCl
2. Discovered Radium B. Fermi
3. Salt C. NH3
4. Ammonia D. 1942
5. Year of the first Nuclear Fission E. H F. Curie
G. 1957
Do not mix matching with sentence completion, e.g., __ 1. A ____ orbits a planet. A. sun etc.

17. Gap-filling items
The colors of the United States flag are ____, ____, and ____.
Gap-filling items are open-ended, requiring students to produce language.

18. Gap-filling items: advantages & disadvantages
Reducing guessing
Students must know answers, no hint given
More or less objective
Disadvantages:
Answers limited to a few restricted words, not assessing higher level learning
Difficult to restrict answers, it is easy to have more than one correct answer
Scoring takes more time than MC items

19. Examples
The smallest phonetic ____ in a language that is capable of conveying a distinction in meaning is called a phoneme.
The ______ were to Egypt as the ______were to Persia and as ______ were to the early tribes of Israel.
Some men don’t have ____ on their face.
An ____ is a part or particle considered to be an irreducible constituent of a specified system.

20. Suggestions for constructing gap-filling items
The gap should contain the key information being tested.
Make sure the meaning of the statement is not lost after words have been taken out
Make sure only one correct answer is possible.
Avoid grammatical clue that gives away the answer

21. Short-answer items What are the colors of the United States flag?
Essay-test items:
Extended response: paragraphs
Restricted-response: short-answer items
How short is short?
A word, a phrase, a sentence (not a paragraph)

22. Short-answer items Advantages: Disadvantages:
student needs to produce language, less guessing
can test some higher learning level
easier to write: no need for distractors or strict answer restriction
avoid the problems of extended response items
Disadvantages:
scoring is less objective
more time consuming for test-takers and scorers

23. References
Hughes, A. (2003). Testing for language teachers (2nd ed). Cambridge University Press.
Clay, Ben. (2001). Is this a trick question? A short guide to writing effective test questions. Kansas Curriculum Center.