Completion, Short-Answer, and True-False Items Educational Assessment of Students Chapter 7 Anthony J. Nitko (Brenda Stewart)

Three Fundamental Principles for Crafting Assessments Focus each assessment task entirely on important learning targets Craft each assessment task to elicit from students only the knowledge and performance that are relevant to the learning targets you are assessing Craft each assessment task to neither prevent nor inhibit a student’s ability to demonstrate attainment of the leaning targets you are assessing First principle is a strong one. Learning trivial performance or minor points of content is a waste of time. Focus on important learning targets. Second principle. Instructors need a very clear idea of what the learning target is. Students will either have the knowledge or they may have to guess, which shows re-teaching is necessary. Third principle recognizes that imprecise wording in a question, may make an item so ambiguous that a student may answer wrong.

Short-Answer Items Require a word, short phrase, number, or symbol response. Three types of short-answer items: Question – What is the capital of Texas? Completion – The capital of Texas is _________. Association – On the blank next to the state name, write the abbreviation for that state. Texas TX California CA The question variety asks a direct question and a short answer is required. The completion variety requires a student to add one or more words to complete it. The association variety consists of a list of terms or a picture for which students have to recall numbers, labels, symbols, or other terms. It is also called identification variety.

Usefulness of Short-Answer Items Abilities Assessed Lower-order thinking skills Recall Comprehension of information Higher-level abilities Make simple interpretations of data and applications of rules Ability to solve numerical problems in science and mathematics Ability to manipulate mathematical symbols and balance mathematical and chemical equations

Usefulness of Short-Answer Items Strengths Relatively easy to construct Can be scored objectively Lowers the probability of getting the answer correct by random guessing Shortcomings Not free of subjectivity Cannot anticipate all possible responses Tends to lower the reliability of the obtained score A teacher may have to make judgments as to the correctness of what the students write. (Spelling, grammar, legibility) The capital of Texas is ________? Students can answer this in many different ways.

Crafting Short-Answer Items Importance of what is assessed How does it match the test blueprint Question format is the preferred format for a short-answer item. Focus the item on the specific knowledge sought Word the items specifically and clearly Put the blank near the end of the sentence If possible do not copy statements verbatim The author of Alice in Wonderland is _______. What is the pen name of the author of Alice in Wonderland? ________. Word your questions so that they require a single correct answer. Putting the blank at the end of the sentence will prevent the student from having to mentally rearrange the question. Verbatim encourages rote memorization.

Crafting Short-Answer Items A completion item should omit important words and not trivial words Limit blanks to one or two Keep all blanks the same length Specify the precision you expect in your answer Avoid irrelevant clues

True-False Items Consists of a statement or proposition the student may judge as true or false Six varieties: True-False Yes-No Right-Wrong Correction Multiple True-False Yes-No with explanation

Usefulness of True-False Items Advantages Certain aspects of the subject matter readily lend themselves to verbal propositions that can be judged true or false Relatively easy to write Scored easily and objectively Can cover a wide range of content within a relatively short period Criticisms Poorly constructed assess only specific, frequently trivial facts Ambiguously worded Answered by random guessing Encourage students to study and accept only oversimplified statements of truth and factual details

Usefulness of True-False Items Generalizations in a subject area Comparisons among concepts Casual or conditional propositions Relationships between two events, concepts, facts, or principles Explanations for why events or phenomena occurred Instances or examples of a concept or principle

Usefulness of True-False Items Evidential statements Predictions about phenomena or events Steps in a procedure of process Computations (or other kinds of results obtained from applying a procedure) Evaluations of events of phenomena

Validity of True-False Item Format Essence of educational achievement is the command of useful verbal knowledge All verbal knowledge can be expressed in propositions Proposition is any sentence that can be said to be true or false Particular area of knowledge is indicated by success in judging truth or falsity of propositions

Guessing on True-False Items: An Ill Wind? Guessing – getting 50% correct Blind (complete random guessing) quite unlike informed guessing (based on partial knowledge) Well-motivated guess blindly only on a small percentage of questions on a test Difficult to obtain a good score on a test by blind guessing only A given true-false test has a high reliability coefficient – evidence blind guessing does not seriously affect the scores

Guessing on True-False Items: An Ill Wind? Random guessing – sometimes called blind guessing Partial knowledge – make an informed guess Higher than 50-50 chance of success Measures of partial knowledge Full knowledge – 100% chance of answering correctly Percentages of correct answers go down with the number of blind guesses made

Suggestions for Getting Started Properly Create pairs of items, one true and one false Rephrase evaluative judgments into a comparative statement Write false statements that reflect the actual misconceptions held by students Convert a multiple-choice item into two or more true-false items

Suggestions for Improving True-False Items Importance of what is assessed How does it match the test blueprint Assess important ideas, rather than trivia, general knowledge, or common sense Make sure it is either definitely true or definitely false Use short statements when possible Use exact language Use positive statements and avoid double negatives

Suggestions for Improving True-False Items Avoid copying sentence verbatim True and false statements should have approximately the same number of words Don’t present in a repetitive or easily learned pattern Do not use verbal clues that give away the answer Attribute the opinion in a statement to an appropriate source Focus on one idea

Crafting Multiple True-False Items Looks like a multiple-choice item – followed by several alternatives No single correct or best answer Response to each alternative is true or false Each alternative is scored correct or incorrect

Crafting Multiple True-False Items Advantages Two or three multiple true-false responses in the time it takes to make one multiple-choice response Multiple true-false tests created from multiple-choice items have higher reliability than original multiple-choice Can assess same abilities as straight multiple-choice items that are crafted to assess parallel content

Crafting Multiple True-False Items Advantages Students believe multiple true-false items do a better job of assessing knowledge than multiple-choice items Students perceive them to be harder May be easier to write – not limited to one correct answer Limitations Shares some of same limitations as multiple-choice

Works Cited Nitko, A. J., (2004). Educational Assessment of Students, 4th ed. Upper Saddle River, NJ: Pearson Education, Inc.