2 QUESTION – ANSWER RELATIONSHIP LITERALINFERENTIALRight ThereThink and SearchWriter and MeOn My OwnAnswer is in the text, usually in one sentenceAnswer is in text, but not all in one placeAnswer is in text and in reader’s headAnswer is totally in reader’s head
3 Types of Questions What Are You Assessing? Detail/recallReading ComprehensionVocabularyChart/graph analysisProblem-solvingInferencePredictAuthor’s purposeCause & effect
4 Summative & Formative Assessment Assessment of Learning(Summative Assessment)How much have students learned as of a particular point in time?Assessment for Learning(Formative Assessment)How can we use assessments to help students learn more?There are two different assessments that we are going to talk about—Assessment of Learning and Assessment for Learning. It is important to understand the difference in the two. They are sometimes referred to as summative and formative assessments. This is partially true, but we are going to be using the terms Assessment of Learning and Assessment for Learning.
5 Assessment of Learning Purpose isto measure student achievement at a point in time for reporting and accountabilityto sort students in rank orderto maximize student learning through standardized testsResults used primarily bypolicy makers & program plannerseducatorsParents & studentsAssessments of Learning are those assessments that happen after learning is supposed to have occurred to determine if it did. They are used to make a statement of student learning status at a point in time to those outside the classroom.
6 Assessment for Learning The purpose isTo promote further improvement of student learning during the learning processTo involve students in ongoing assessment of their own achievementIdentify student needsHelp students see and hit a targetResults are used primarily byEducators – to adjust teachingParents & studentsAssessments for Learning happen while learning is still underway. These are the assessment that we conduct throughout teaching and learning to diagnose student needs, plan our next steps in instruction, provide students with feedback they can use to improve the quality of their work, and help students see and feel in control of their success.
8 Disadvantages of Multiple Choice Questions Quality items are difficult and time consuming to developTendency for items to focus on low level learning objectivesNo credit for partial knowledgeAssessment results may be biased by students’ reading ability and test-wisenessMay overestimate learning due to the ability to utilize an elimination process for answer selectionDoes not measure the ability to organize and express ideas
9 With that in mind . . .Allows for assessment of a wide range of learning objectives from factual to evaluative understandingAmenable to item analysisPermits manipulation of difficulty level by adjusting the degree of similarity among response optionsObjective nature limits bias in scoringEasily administered to large numbers of studentsLimits assessment bias caused by poor writing skillsAND it is the format used on high-stakes tests
10 General Guidelines for Writing Questions Keep wording simple and focusedFollow rules of grammarEliminate superfluous materialTest mastery of material, not ability to figure out what you’re askingAvoid clues within itemsGrammatical cluesDeterminers such as “always” and “never”Don’t lift wording directly from materialswrite in your own wordsHighlight critical words such as MOST, LEAST, EXCEPTthey are easily overlooked
11 Parts of the MC Question StemProvides introductory informationDirect question or incomplete statementCorrect AnswerOne and ONLY ONE is preferableDistractorsIncorrect response options
12 General Guidelines for Multiple Choice Questions State whole question in the stemKeep responses brief and parallel in:LengthUse of specific determinersGrammatical constructionGeneralityAvoid repetition in response optionsLimit use of “all” or “none of the above”Be sure there is only one correct or best answerBest answer items should be worded to ask for BESTCORRECT – best measures factual knowledgeBEST – well suited for items dealing with interpretation, understanding, or inference12
13 More Guidelines Negative premises are undesirable may especially disadvantage ESL (English Second Language) students.The answer to one question should not be obtainable from another question.Avoid potential sources of bias & distortionWordingstimuli (graphics, pictures, charts)Use at least 3 distractors to reduce the chance of guessing the correct answer.Avoid using a pattern for the position of the correct response.
14 School Bus — Multiple Choice A school is planning a field trip and will be using school buses to transport the students. A school bus holds 36 students. If 1,128 students are being transported, how many buses are needed?a c. 32b d. 36This mathematical question focuses on a real-life situation. The students are being asked to solve a problem. There is actually only one correct answer (c), and that is how most teachers would use this item. Upon further review, this item does provide more information about student learning. For example, a student that responds with the answer of (a) might have actually done the work. They simply did not know how to deal with those students that couldn’t fit into those 31 buses. From a pure mathematical sense, (b) is correct, but it does not relate to a real-life solution. An answer of (d) might indicate that the student read the question and saw a number in the stem that they thought was correct. It must be noted, that with a multiple choice question, an answer that is correct might simply be a guess.
15 The Good, The Bad, and The Ugly Sample QuestionsThe Good, The Bad, and The Ugly
16 Two cyclists start biking from a trail’s start 3 hours apart Two cyclists start biking from a trail’s start 3 hours apart. The second cyclist travels at 10 miles per hour and starts 3 hours after the first cyclist who is traveling at 6 miles per hour. How much time will pass before the second cyclist catches up with the first from the time the second cyclist started biking?2 hours7 ½ hours4 hours, 30 minutes6 hours5 ¾ hoursFIX – put answers into ascending or descending orderFIX – put answers into similar formatPractice question from ACT Prep booklet
17 If a circle has the diameter of 8, what is the circumference? 6.2812.5625.1350.24100.48Answers are parallel in form - all done with same number of decimal points, so no ONE answer stands out.Also answers are in ascending order.Practice question from ACT Prep booklet
18 The wound exhibited signs of copious drainage requiring medical intervention. Copious means MinimalClarityMaximalFoulFIX – put answers into parallel format (all similar parts of speech, forms of words)If a student chose #4 – what do we know about that child’s reading strategy?? (context)Practice question from ACT Prep booklet
19 When New York freed its slaves, Isabella had no childrenfive childrenan educationthree childrenDistractor #3 really IS A DISTRACTOR because it doesn’t fit in with the other answers. Is this a trick question?
20 The concept of the plasma membrane as a simple sieve-like structure is not inadequate to explain the passage of gases involved in respiration into and out of the cellability of the cell to admit selectively some inorganic ions while excluding othersFailure of protein molecules to pass through the membraneInability of the cell to use starch without prior digestionMisleading Language – not adequate + inabilityVery tough vocabulary!
21 Example of testing vocabulary “connotes” - do students know this term? Have you used it in the classroom?Released test questions from the 2003 and 2004 California Standards Tests (CSTs).
22 The details about the princess’s life before her eighteenth birthday imply that she was - Raised to value only material goodsIndifferent to the feelings of othersEasily boredBoth A and CBoth B and C, but not AUse sparingly!Sample question from Standardized Test Practice
23 What are you asking the student to know? Purpose of QuestionsWhat are you asking thestudent to know?
24 Detail Where was Susan’s dad stationed overseas? In Albany, New York In Adu Jahm, near BaghdadAmong ancient ruins in southern IraqEast of Iraq, in IranSample question from Scholastic Scope
25 Vocabulary – Multiple Meanings The subject of today’s lesson is words with multiple meanings. In which sentence does the word subject mean the same thing as in the sentence above?Susan’s mother is subject to fainting spells.In Kenya, high school students take 13 subjects at a time.If you disobey traffic laws, you may be subject to arrest.Feel free to change the subject of your report.Sample question from SAT9 Activities (McDougall-Littell).
26 Inference SampleReleased test questions from the 2003 and 2004 California Standards Tests (CSTs).
27 Infer/Predict SampleReleased test questions from the 2005 Massachusetts Standards Tests (CSTs).
28 InferenceThe details about the princess’s life before her eighteenth birthday imply that she was -Raised to value only material goodsIndifferent to the feelings of othersEasily boredNot interested in getting marriedSample question from Standardized Test Practice
29 Comprehension SampleReleased test questions from the 2003 and 2004 California Standards Tests (CSTs).
30 Author’s PurposeReleased test questions from the 2003 and 2004 California Standards Tests (CSTs).
31 Author’s PurposeReleased test questions from the 2003 and 2004 California Standards Tests (CSTs).
32 Application SampleReleased test questions from the 2005 Massachusetts Standards Tests (CSTs).
33 Critical Analysis You can conclude from the passage that Epimetheus did not want to know the contents of the box.already knew what the box contained.wanted to know what was in the box.never did learn the contents of the box.Sample question from SAT9 Activities (McDougall-Littell).
34 Critical Analysis The poem suggests that John Henry dies from shock. a broken heart.disappointment.pushing himself too hard.Sample question from SAT9 Activities (McDougall-Littell).
35 Reading Comprehension Which of the following questions is answered in the article?What causes heart damage during prolonged space travel?Why might space travelers develop kidney stones?What can space travelers do to avoid becoming depressed?What will the interior of a passenger space plane look like?Sample question from Standardized Test Practice