WRITING QUALITY MULTIPLE CHOICE QUESTIONS Barbara Allen, Vernon-Verona-Sherrill Barbara Patterson, Liverpool

MATCH OBJECTIVE Assessment measures if students mastered desired objectives

QUALITY MULTIPLE CHOICE QUESTIONS ARE A complete concept Clear and concise Reflect the main points or significant details of the passage Longer than the answer choices

QUALITY MULTIPLE CHOICE QUESTIONS ARE NOT Answerable without listening to or reading the passage Dependent on one word Written in the negative Grammar-based

QUALITY MULTIPLE CHOICE ANSWERS ARE Phrased positively Approximately equal in length o Similar in construction, vocabulary or context o Internally consistent Mutually exclusive Distractors should be plausible and unquestionably the wrong answer

QUALITY MULTIPLE CHOICE ANSWERS ARE NOT Identical in wording from the passage Contradictory Irrelevant Dependent on prior knowledge Written in the negative Written using all of the above or none of the above

What is the topic of this news item? 1. The environment 2. Pro sports 3. International politics 4. Domestic travel

VALID OR INVALID? WHY?

\ Qual è il tempo nel nord-ovest d'Italia a domenica? 1. nevica 2. piove 3. è nuvoloso 4. c’è il sole

VALID OR INVALID? WHY?

At what time do the doors open for the game? 1. 7:00 pm 2. 7:00 am 3. 17: :00

VALID OR INVALID? WHY?

Welches Problem hat er nicht Donald Duck? 1. Er muss seine Rechnungen zu bezahlen. 2. Er hat nicht genug Geld. 3. Er hat kein Papier mehr. 4. Seine Miete ist fallig.

VALID OR INVALID? WHY?

POSSIBLE READING COMPREHENSION ITEMS: Forms, signs, labels, tickets, programs, timetables, maps, menus, celebration invitations and announcements, advertisements, short articles, informal notes, brief correspondence, hotel literature, telephone books

LISTENING COMPREHENSION SETTING Provides students with the background information they would have in an authentic listening/speaking/responding situation in the target language. Usually includes where they are and to whom they are listening.

LISTENING STIMULUS SHOULD BE Spoken in the target language One consistent message, not “tricky” Appropriate to the age level and interest of students Length and complexity appropriate to comprehension level of students

ACCEPTABLE SOURCES FOR TEST ITEMS Authentic cultural material Articles of general interest to students Appropriate for students’ age level From current magazines, newspapers, Internet, notes, texts, radio and television broadcasts

UNACCEPTABLE SOURCES FOR TEST ITEMS Textbooks and ancillary materials Exam review books Commercially published tests Materials frequently used in class work Teachers’ imaginations Controversial or objectionable Sensitive subjects (e.g. disasters) Humor

PERCENTAGE OF DIFFICULTY Include type of questioning used throughout the unit Easiest question first Within each section, place questions easiest to the most difficult End with an easy question Arrange questions so there isn’t a pattern of answers Vary the ratio of answers

LEVEL OF DIFFICULTY Using Bloom’s Taxonomy Focus of the stem: Is it about the main idea? (easier) Is it regarding a significant detail? (more difficult) Is it inferential? (very challenging) Options with a high degree of similarity require greater understanding in order to identify a correct response

CALCULATING ITEM DIFFICULTY The percentage of students that correctly answered the item. Calculation: Divide number of students who got an item correct by total number of students who answered it. Higher the value, easier the item 90+ answered correctly - very easy, shouldn’t be reused Below 20 answered correctly - very difficult & should be reviewed for validity or students didn’t understand concept Ideal value is around 60%

QUESTIONS FOR REMEMBERING What happened after...? How many...? What is...? Who was it that...? Can you name...? Describe what happened after… Who spoke to...? Which is true/false …? Select Choose

QUESTIONS FOR UNDERSTANDING Can you explain why…? How would you explain…? Who do you think...? What was the main idea...? What do you think could have happened next...? Does everyone act in the way that …….. does? Describe …

QUESTIONS FOR APPLYING Do you know of another instance where…? Group by characteristics such as… Which factors would you change if…? What questions would you ask of…? Compute/calculate …

QUESTIONS FOR ANALYZING How is...similar to...? Which are other possible outcomes? Why did...changes occur? An explanation for what must have happened when...? What are some or the problems of...? What were some of the motives behind..? What was the turning point? What was the problem with...? Which events could not have happened? If...happened, what might the ending have been? Sequence/order/arrange …

QUESTIONS FOR EVALUATING What is the better solution to/for...? Why is … a good/bad thing? How would an ?/American have handled...? What changes to..? What are the consequences..? What influence will....have on our lives? What are the pros/cons of....? Why is....of value? What are the alternatives of..? Who will gain/loose ….?

QUESTIONS FOR CREATING A possible solution to… How would … deal with...? What would happen if...? What could improve …?

QUESTIONS