Guidelines for preparing Multiple Choice Questions (MCQ’s)

How? References available: Case and Swanson, NBME “Constructing Questions for the Basic and Clinical Sciences” - publications, item writing manual

Traditional MCQ’s Multiple types Do not use: True / False Negative Stems R type K type

Good MCQ’s * Assess application of knowledge Assess reasoning skills (how do you think) DO NOT assess simple recall of isolated facts

Good MCQ’s * “One Best Answer” type

The Basic Rules for One-Best-Answer Items 1. Each item should focus on an important concept Common, serious or potentially catastrophic clinical problem Assesses an Objective or Competency of the curriculum Avoid : Trivial, Overly complex, or Tricky Case and Swanson, 2001

The Basic Rules for One-Best-Answer Items 2. Each item should assess the application of knowledge NOT recall of an isolated fact. Case and Swanson, 2001

How? Present a Stem: Present a clinical case scenario as a good basis for a question Present a question Provide Options for answers (generally 5) “One Best” correct answer several “distractors”

How? An old example Acute intermittent porphyria is the result of a defect in the biosynthetic pathway for: A. collagen B. corticosteroid C. fatty acid D. glucose E. heme

How to do it better? A better question is one that uses a clinical scenario in the stem: An otherwise healthy 33 year-old man has mild weakness and occasional episodes of steady, severe abdominal pain with some cramping and no diarrhea. One aunt and a cousin have had similar episodes. During an episode, his abdomen is distended, and bowel sounds are decreased. Neurological examination shows mild weakness in the upper arms. These findings suggest a defect in the biosynthetic pathway for A. collagen B. corticosteroid C. fatty acid D. glucose E. heme

How? Another example What area is supplied with blood by the posterior inferior cerebellar artery? or A 63-year old man develops left-sided limb ataxia, Horner’s syndrome, nystagmus, and loss of appreciation of facial pain and temperature sensations. What artery is most likely to be occluded? Case and Swanson, 2001

Basic Principle Application of knowledge rather than Isolated Fact

Basic Principle Not only the Memory (memorization) Avoid questions of simple fact that can be found on a single page in a textbook Engage the Mind (cognitive – reasoning, problem solve)

The Basic Rules for One-Best-Answer Items 3. The stem of the item must pose a clear question Based on the scenario, pose the question. It should be possible to arrive at an answer without seeing the options Case and Swanson, 2001

The Basic Rules for One-Best-Answer Items 4. Write the answer and distractors All options should be homogeneous Correct and incorrect options (distractors) should fall into the same category (all diagnoses, tests, treatments, etc) Case and Swanson, 2001

Distractors Should be: Plausible Grammatically consistent Logically compatible Same relative length as the correct answer Place the options in logical order (numeric or alphabetical) Case and Swanson, 2001

The Basic Rules for One-Best-Answer Items 5. Avoid “technical item flaws” that provide special benefit to test-wise examinees or that pose irrelevant or unnecessary difficulty (please refer to manual pg 26 and 33) Case and Swanson, 2001

Example Technical Item Flaws In patients with advance dementia, Alzheimer’s type, the memory defect A. can be treated adequately with phosphatidylcholine (lecithin) B. could be a sequela of early parkinsonism C. is never seen in patients with neurofibrillary tangles at autopsy D. is never severe E. possibly involves the cholinergic system

Possible changes A 87 yrs female has advanced dementia of the Alzheimer’s type. Her memory and communications with her family are improved following a new medication. The drug type most likely responsible for her improvement is: A. cholinergic agonists B. dopaminergic agonists C. GABA agonists D. nicotinic agonists E. serotonergic agonists

Advanced Question Writing K1 Questions – designed to test Competency of Application of Knowledge K2 Questions – Designed to Assess Competency of Reasoning and Clinical Problem Solving

Advanced Question Writing K2 Questions – Designed to Assess reasoning and Clinical Problem Solving - the question goes beyond application - the question skips the step of application and requires the candidate to engage in Clinical Reasoning - the answer usually involves a diagnosis, differential diagnosis or a management option

Advanced Question Writing K2 Questions Ref: Ware, J. Torstein V. Quality Assurance of Item Writing. Medical Teacher, 2009, Vol. 31, No. 3 : Pages

Review the Basic Rules Subject each question you write to the above 5 “tests” If a question passes all 5, it is probably well-written and a good question Does your question assess application of Knowledge or Clinical Reasoning Case and Swanson, 2001

Final Check? Please remember to confirm: The question is testing a learning objective or competency The case scenario is appropriate The question and the answer are both clear

What to do now? Each lecturer or tutor is asked to construct at least two questions for the block. Each question is based on one of the weekly case objectives Your questions can be reviewed with your writing team providing feedback to you.