Designing Objective (Multiple Choice) Exams Mike DeBisschop, Dawn Rager, and Theresa Westbay

Tools & Strategies for Writing Questions  Types of Questions  Factual  Conceptual  Applied  See examples corresponding to Bloom’s Taxonomy at the University of Texas Instructional Assessment Resources website See examples corresponding to Bloom’s Taxonomy at the University of Texas Instructional Assessment Resources website  Content of Questions and the Exam as a Whole  Work backwards from learning objectives  Does the distribution of items make sense given the learning objectives?  Construct an exam blueprint ■See example blueprint at the University of Texas Instructional Assessment Resources websiteSee example blueprint at the University of Texas Instructional Assessment Resources website

Another Example of an Exam Blueprint Levels of Understanding Number of Questions % of Test Devoted to Each Topic Topics Factual Questions Conceptual Questions Applied Questions Number of Questions % of Test Items at Each Level

Tools & Strategies for Writing Questions  Design of Questions  Simplicity and clarity are usually best  Avoid "tricky" or confusing questions  Take extra care when using negative language  Avoid obvious distracter items  Time Considerations  Carefully consider the length of the exam and time required to complete it ■Influenced by the nature of the items ■Try to keep questions brief ■Consider individual student variability

Minimizing Test Anxiety  Consider giving practice questions  Consider permitting students to ask for clarification  Consider using humor in small doses

Evaluating Test Items & the Exam as a Whole  Item analysis  Peer review  Consider allowing students to challenge questions in a controlled, but fair manner  Consider marking and quickly reviewing scantrons by hand to detect errors and discourage cheating  What to do when you discover that an item is faulty  Evaluate the overall grade distribution  Compare across sections and/or over time if possible  Use the information gathered reflectively in designing future exams

Activity: Evaluating Items  Activity on evaluating multiple choice items at the University of Texas Instructional Assessment Resources website Activity on evaluating multiple choice items at the University of Texas Instructional Assessment Resources website

Additional Resources  PETAL website PETAL website  Grading  Writing Multiple-Choice Questions that Demand Critical Thinking Writing Multiple-Choice Questions that Demand Critical Thinking  Designing and Managing MCQs: Designing and Managing MCQs: