1. Writing Selection Items
EDU 385 Session 8
Writing Selection Items
Multiple Choice

2. Bell Work
How to not to prepare a Selection Item Test

3. Content Objectives
Understand the characteristics of multiple- choice items
Understand the strengths and limitations of multiple-choice items

4. Language Objectives
Distinguish between well-stated and poorly stated multiple-choice items
Identify and correct faults in poorly state multiple-choice
Match multiple-choice items to intended learning outcomes
Construct multiple-choice items that are well stated, relevant to important learning outcomes, and free of defects

5. Vocabulary
Stem of the Item
Distractor Items

6. Multiple-Choice Items
Strengths
1. Learning outcomes from simple to complex can be measured
2. Highly structured and clear tasks are provided
3. A broad sample of achievement can be measured
4. Incorrect alternatives provide diagnostic information
5. Scores are less influenced by guessing than true-false items
6. Scoring is easy, objective, and reliable
Limitations
1. Constructing good items is time consuming
2. It is frequently difficult to find plausible distractors
3. This item in ineffective for measuring some types of problem solving and the ability to organize and express items
4. Score can be influenced by reading ability

7. Nature of Multiple-Choice Items
Consists of
A Stem, which represents a problem situation
Several Alternatives (Options or Choices), which provide possible solutions to the problem
The Stem may be a question or incomplete statement
Alternatives include the correct answer and several plausible wrong answers called Distractors (for students uncertain of correct answer)

8. Example (Question Form & Incomplete Sentence Form)
Which one of the following item types is an example of a supply-type test item?
A. multiple-choice item
B. true-false item
C. matching item
*D. short answer item
2. An example of a supply-type item is the
{Though stated differently both stems pose the same problem.}
NOTE: The incomplete sentence is more concise

9. Best Answer Multiple Choice
Best answer form all answers are partially correct but one is clearly the best answer
Example
Which item is best for measuring computational skill
A. Multiple-choice item
B. True-false item
C. Matching item
*D. Short answer item

10. Rules for Writing Multiple-Choice Items
Effective multiple-choice questions present students with tasks that are important and clearly understood and can be answered by anyone who has achieved the intended learning outcome
Nothing in the content should prevent the informed student from responding correctly
Nothing in the content should give clues to enable the uninformed student to select the correct answer ˜

11. 1. Design each item to measure an important learning outcome
The problem situation should be important and related to the intended learning outcome to be measured
Avoid obscure, irrelevant, and unimportant material
Remember the purpose is help determine the extent to which the intended learner outcomes have been achieved

12. Rules for Writing Multiple-Choice Items
2. Present a single clearly formulated problem in the STEM of the item
Example
Poor”
A table of specifications
A. indicates how a test will be used to improve learning
*B. Provides more balanced sampling
C. Arranges the instructional objectives in order of their importance
D. Specifies the method of scoring to be used on a test
Better
What is the main advantage of using a table of specification when preparing for an achievement test
A. It reduces the amount of time required
*B. It improves the sampling of content
C. It make the construction of test items easier
D. It increases the objectivity of the test

13. Rules for Writing Multiple-Choice Items
3. State the stem of the item in simple, clear language
Example
Poor:
The paucity of plausible, but incorrect, statements that can be relate to a central idea poses a problem when constructing which one of the following items?
A. Short answer
B. True-false
*C. Multiple choice
D. Essay
Better
The lack of plausible, but incorrect, alternatives will cause the greatest difficulty when constructing
A. short-answer items
B. true-false items
*C. multiple-choice items
D. essay items

14. Rules for Writing Multiple-Choice Items
4. Put as much of the wording as possible in the stem of the item
Example
Poor
In objective testing, the term objective
A. refers to the method of identifying the learning outcomes
B. refers to the method of selecting the test content
C. refers to the method of presenting the problem
*D. refers to the method of scoring answers
Better
In objective testing, the term objective refers to the method of
A. identifying the learning outcomes
B. selecting the test content
C. presenting the problem
*D. scoring the answers

15. Some times it requires more than moving common words to the STEM
Poor Example
Instructional objectives are more apt to be useful for test-construction purposes when they are stated in such a way that they show
A. the course content to be covered during the instructional period
*B. the kinds of performance students should demonstrate reaching the goal
C. the things the teacher will do to obtain maximum student learning
D. the types of activities to be participated in during the course
Good Example
Instructional objectives are most useful for test-construction when they are stated in terms of
A. course content
*B. student performance
C. teacher behavior
D. learning activities

16. Rules for Writing Multiple-Choice Items
5. State the stem of the item in positive form, whenever possible
*A positively stated item tends to measure more important learning outcomes than negatively stated items
*Knowing the most relevant argument has greater education value than knowing least relevant arguments.
*Positively stated are harder to write because the item needs more distractors, than negatively stated items

17. Rules for Writing Multiple-Choice Items
5. State the stem of the item in positive form, whenever possible
Example
Item one:
Which one of the following is a category in the revised taxonomy of cognitive domain?
*A. Understand
B. (distractor needed)
C. (distractor needed)
D. (distractor needed)
Item two:
Which one of the following is not a category in the revised taxonomy of cognitive domain?
A. Understand
B. Apply
C. Analyze
*D. (answer needed)

18. When it is difficult to find enough distractors, a more desirable procedure is a modified true-false form
Example
Directions: Indicate which of the following are categories in the taxonomy of the cognitive domain by circling Y for yes and N for no
*Y N Understand
Y N* Critical Thinking
Y N* Reasoning
*Y N Create

19. Rules for Writing Multiple-Choice Items
6. Emphasize NEGATIVE wording whenever it is used in the stem of an item
Example
Poor:
Which one of the following is not desirable practice when preparing for multiple-choice items?
A. Stating the stem in positive form
B. Using the stem that could function as a short-answer item
C. Underlining certain words in the stem for emphasis
*D. Shortening the stem by lengthening the alternatives
Better:
All of the following are desirable practices when preparing multiple-choice items EXCEPT

20. 7. Make certain the intended answer is correct or clearly the best
Example
Poor:
What is the best method of selecting course content for test items?
Better:
Which one of the following is the best method of selecting course content for test items?
Example of proper phrasing of the stem
What is the purpose of classroom testing?
One purpose of classroom testing is
(or)
The main purpose of classroom testing is

21. 8. Make all alternatives grammatically consistent with the stem of the item and parallel in form
Example
Poor
The recall of factual information can be measured best with a
A. matching item
B. multiple-choice item
*C. short-answer item
D. essay question
Better
The recall of factual information can be measured best with
A. matching items
B. multiple-choice items
*C. short-answer items
D. essay questions
The indefinite article “a” in the first version makes the last distractor obviously wrong (correct form would be “an” essay

22. a) similarity of wording in both the stem and the correct answer
9. Avoid verbal clues that might enable students to select the correct answer or to eliminate an incorrect alternative
a) similarity of wording in both the stem and the correct answer
Which one of the following would you consult first to locate research articles on achievement testing
A. Journal of Educational Psychology
B. Journal of Educational Measurement
C. Journal of Consulting Psychology
*D. Review of Educational Research
b) Stating the correct answer in textbook language or stereotyped phraseology
Learning outcomes are most useful in preparing test when the are
*A. clearly stated in performance terms
B. developed cooperatively by teachers and students
C. prepared after the instruction has ended
D. stated in general terms

23. c) Stating the correct answer in greater detail
Lack of attention to learning outcomes during test preparation
A. will lower the technical quality of the items
B. will make the construction of test items more difficult
C. will result in greater use of essay questions.
*D. may result in a test that is less relevant to the instructional program
d) Including absolute terms in the distracter
Achievement tests help students improve their learning by
A. encouraging them all to study hard
*B. informing them of their progress
C. giving them all a feeling of success
D. preventing any of them from neglecting their assignments

24. e) Including two response that are all inclusive
Which of the following types of test items measures learning outcomes at the recall level?
*A. Supply-type items
B. Selection-type items
D. Matching items
E. Multiple-choice items
f) Including two responses that have the same meaning
Which one of the following is the most important characteristic of achievement-test results?
A. Consistency
B. Reliability
*C. Relevance
D. Objectivity

25. 10. Make the distracters plausible and attractive to the uninformed
Ways to increase plausibility and attractiveness of distracters
a. Use the common misconceptions of errors of students as distracters
b. State the alternative in the language of the students
c. Use “good-sounding” words (“accurate,” “important,”) in the distracters
d. Make the distractors similar to the correct answer in both length and complexity of wording
e. Use extraneous clues in the distracters, such as stereotyped phrasing, scientific-sounding answers, and verbal associations with the stem of the item. But don’t overuse these clues to the point where the become ineffective
f. Make the alternatives homogeneous, but in doing so beware of fine discriminations that are educationally insignificant

26. 11. Vary the relative length of the correct answer to eliminate length as a clue
There is a tendency for the correct answer to be longer than alternatives because of the need to qualify statements to make them unequivocally correct
The relative length of the correct answer can be removed as a clue by varying it in such a manner that no apparent pattern is provided

27. 12. Avoid using the alternative “all of the above,” and use “none of the above” with extreme caution
*Lessens the effectiveness of the test
*On “all the above” questions - even a student with a partial knowledge of the topic knows one of the responses is false they know the correct answer
*Same with “none of the above” - if they know one response is true then know the correct answer
*They also have a 50 percent chance of guessing the right answer

28. 13. Vary the position of the correct answer in a random manner
Avoid following a pattern
Randomly select where to put the correct answer
Page # strategy - open any page in a book look at the last digit on the right hand page (since the right page alway ends in an “odd number”
A. = 3
B. = 5
C. = 7
D. = 9

29. To increase the difficulty of a test item
14. Control the difficulty of the item by varying the problem in the stem or changing the alternatives
To increase the difficulty of a test item
- Increase the level of knowledge called for to by making the item more complex
- Make the alternatives more homogeneous

30. 15. Make certain each item is independent of the other items in the test
- Occasionally information given in the stem of another question help students answer another one
- Avoid by carefully reviewing all the question before administering the test
- Avoid interlocking questions where the answer to a question relies on correctly knowing the answer to the previous question

31. 16. Use an efficient item format
- Alternatives (distracters) listed below each other
- Place answers on the left side of test (makes scoring easier
- Use letters not numbers on items (less confusing for he students

32. 17. Follow the normal rules of grammar

33. 18. Break (or bend) any of these rules if it will improve the effectiveness of the item
The previous rules are set forth to help design multiple- choice tests
As teachers gain more experience they may want to vary ignore or modify some of these suggestions

34. Checklist for Evaluating Multiple-Choice Items
1. Is the type of item appropriate for measuring the intended learner outcome?
2. Does the item task match the learning task to be measured?
3. Does the stem of the item present a single, clearly formulated problem?
4. Is the stem stated in simple, clear language?
5. Is the stem worded so that there is no repetition of material in the alternative?
6. Is the stem stated in positive form whenever possible?
7. If negative wording is used in the stem, is it emphasized (by underlining or caps?
8. Is the intended answer correct or clearly best?

35. Checklist for Evaluating Multiple-Choice Items
9. Are all alternatives grammatically consistent with the stem and parallel in form?
10. Are the alternatives free from verbal clues to the correct answer?
11. Are the distracters plausible and attractive to the uniformed?
12. To eliminate length as a clue, is the relative length of the correct answer varied?
13. Has the alternative “all the above” been avoided and “none of the above” used only when appropriate?
14. Is the position of the correct answer varied so that there is no detectable pattern?
15. Does the item format and grammar usage provide for efficient test taking?

36. Summary
1. The multiple-choice item is the most highly regarded and useful selection-type item
2. The multiple-choice item consists of a stem and a set of alternative answers (Options or Choices)
3. The multiple-choice item can be designed to measure various intended learning outcomes, ranging from simple to complex
4. Items designed to measure achievement beyond the knowledge level must contain some novelty

37. Summary 5. The stem of a multiple-choice item should present
a single, clearly formulated problem that is related
to an important learning outcome
6. The intended answer should be correct or clearly best, as agreed upon by authorities
7. The distractors (incorrect alternatives) should be plausible enough to lead the uninformed away from
the correct answer
8. The items should be written in simple, clear language that is free of nonfunctioning content

38. Summary
9. The items should be free from irrelevant sources of difficulty (e.g., ambiguity) that might prevent an informed examinee from answering correctly
10. The items should be free of irrelevant clues (e.g., verbal associations) that might enable an uniformed examinee to answer correctly
11. The item format should provide for efficient responding and follow the normal rules of grammar
12. The item format should provide a framework for preparing effective multiple-choice items, but experience in item writing may result in modifications to fit particular situations

39. Next Class Writing Selection Type Items True-False Matching
Interpretive Exercise

40. Now Go Forth
and
Do Good Things