Presentation on theme: "Using Common Testing Techniques Effectively"— Presentation transcript:
1Using Common Testing Techniques Effectively Writing Multiple-choice, Gap-filling, and Short-answer ItemsHanh Thi Nguyen, Ph.D.TESL ProgramsLearning Assessment Committee member
2What is a ‘good’ test? An effective test measures what it is supposed to measure (has validity)measures consistently (has reliability)is practical to make, administer, and score (has practicality)has positive effects on learning (has positive washback)
3How to meet those criteria? Validity:Write test items that correspond to course objectivesMake sure only these objectives are testedReliability:Include not too few items, but not too long a test eitherEliminate distracting factors in the environmentMake sure scoring is done consistentlyMake sure candidates are familiar with testing formatMake sure instructions are non-ambiguousPracticalityConsider time and costSometimes practicality conflicts with validity and reliability, so test what needs to be tested, do not just test what is easy to testPositive washbackMake sure test can improve teachingMake sure test has positive impact on learning
5Multiple-choice items 1. What is the shape of the earth? Stem OptionsA. round keyB. square distractorC. oval distractorD. triangular distractor
6Advantages of MC items Objective testing Scoring is easy Less dependent on students’ writing skillsCan cover a broad range of contentsVery effective when eliciting a best answer
7Disadvantages of MC items Testing only recognition knowledge, no production knowledgeCandidates have a 25% chance to guess right (if 4 options) – hence, “multiple-guess” testNot always possible to have 3-4 plausible alternatives to correct answer
8ExamplesIf I ___ him last year, I ___ him to help me. A. know, ask B. knew, asked C. have known, asked D. had known, would have askedKey is the longest string – facilitates guessingOption A not likely to distract
9ExamplesShe started to ____, thinking about his death. A. giggle B. complain C. shout D. weepAll 4 options are possible answers.Need further context to restrict answer.
10Suggestions for constructing MC items For the stem:Use direct questions rather than incomplete sentences.Present a single question or problem in stemMake sure to include in the stem the words that are repeated in all options (avoid redundancy)
11Suggestions for constructing MC items For the optionsUse errors commonly made by students in class as basis for distractorsUse options that say “none of the above”, “all of the above”, “A & B only” sparinglyAvoid having the key being the longest or shortest itemWhen possible, present options in some logical order (chronological, least to most, alphabetical…)Use similar format for all options (all are phrases, all are single words, etc.)
12True/False itemsEssentially, T/F items are MC items with only 2 optionsAdvantages:Good for testing misconceptions, comprehensionCan be scored quicklyDisadvantages:Students have a chance to guess the right answer
13Suggestions for constructing T/F items Make sure there is a single idea in the stemMake sure the statement is either absolutely true or false, no exceptionsAvoid using extreme qualifiers (never, always, etc.) whenever possibleAvoid using negative statementsAvoid recycling the exact statement from the textAvoid using unfamiliar vocabulary
14Matching items Directions: On the line next to each children’s book in Column A print the letter of the animal or insect in ColumnB that is a main character in that book. Each animal or insectin Column B can be used only once.Column A Column B____1. Charlotte’s Web A. Bear____2. Winnie the Pooh B. Chimpanzee____3. Black Beauty C. Cricket____4. Tarzan D. Deer____5. Pinocchio E. Horse____6. Bambi F. Pig
15Matching items: advantages & disadvantages Essentially are MC items, in which the answer of one item also depends on the answers of the other items in the matching setAdvantages: Good for testingDefinitionsClassificationsCause and effectsProblems and solutionsRelationship between 2 entities (e.g., event-date, book-character)DisadvantagesItems are inter-relatedTime consuming for test-takersNot suitable for all knowledge and learning skills
16Suggestions for constructing matching items Make sure all items are of the same topic Directions: Match the following.1. Water A. NaCl2. Discovered Radium B. Fermi3. Salt C. NH34. Ammonia D. 19425. Year of the first Nuclear Fission E. H F. CurieG. 1957Do not mix matching with sentence completion, e.g., __ 1. A ____ orbits a planet. A. sun etc.
17Gap-filling itemsThe colors of the United States flag are ____, ____, and ____.Gap-filling items are open-ended, requiring students to produce language.
18Gap-filling items: advantages & disadvantages Reducing guessingStudents must know answers, no hint givenMore or less objectiveDisadvantages:Answers limited to a few restricted words, not assessing higher level learningDifficult to restrict answers, it is easy to have more than one correct answerScoring takes more time than MC items
19ExamplesThe smallest phonetic ____ in a language that is capable of conveying a distinction in meaning is called a phoneme.The ______ were to Egypt as the ______were to Persia and as ______ were to the early tribes of Israel.Some men don’t have ____ on their face.An ____ is a part or particle considered to be an irreducible constituent of a specified system.
20Suggestions for constructing gap-filling items The gap should contain the key information being tested.Make sure the meaning of the statement is not lost after words have been taken outMake sure only one correct answer is possible.Avoid grammatical clue that gives away the answer
21Short-answer items What are the colors of the United States flag? Essay-test items:Extended response: paragraphsRestricted-response: short-answer itemsHow short is short?A word, a phrase, a sentence (not a paragraph)
22Short-answer items Advantages: Disadvantages: student needs to produce language, less guessingcan test some higher learning leveleasier to write: no need for distractors or strict answer restrictionavoid the problems of extended response itemsDisadvantages:scoring is less objectivemore time consuming for test-takers and scorers
23ReferencesHughes, A. (2003). Testing for language teachers (2nd ed). Cambridge University Press.Clay, Ben. (2001). Is this a trick question? A short guide to writing effective test questions. Kansas Curriculum Center.