1. Completion, Short-Answer, and True-False Items
Educational Assessment of Students
Chapter 7
Anthony J. Nitko
(Brenda Stewart)

2. Three Fundamental Principles for Crafting Assessments
Focus each assessment task entirely on important learning targets
Craft each assessment task to elicit from students only the knowledge and performance that are relevant to the learning targets you are assessing
Craft each assessment task to neither prevent nor inhibit a student’s ability to demonstrate attainment of the leaning targets you are assessing
First principle is a strong one. Learning trivial performance or minor points of content is a waste of time. Focus on important learning targets.
Second principle. Instructors need a very clear idea of what the learning target is. Students will either have the knowledge or they may have to guess, which shows re-teaching is necessary.
Third principle recognizes that imprecise wording in a question, may make an item so ambiguous that a student may answer wrong.

3. Short-Answer Items
Require a word, short phrase, number, or symbol response.
Three types of short-answer items:
Question – What is the capital of Texas?
Completion – The capital of Texas is _________.
Association – On the blank next to the state name, write the abbreviation for that state.
Texas TX
California CA
The question variety asks a direct question and a short answer is required.
The completion variety requires a student to add one or more words to complete it.
The association variety consists of a list of terms or a picture for which students have to recall numbers, labels, symbols, or other terms. It is also called identification variety.

4. Usefulness of Short-Answer Items
Abilities Assessed
Lower-order thinking skills
Recall
Comprehension of information
Higher-level abilities
Make simple interpretations of data and applications of rules
Ability to solve numerical problems in science and mathematics
Ability to manipulate mathematical symbols and balance mathematical and chemical equations

5. Usefulness of Short-Answer Items
Strengths
Relatively easy to construct
Can be scored objectively
Lowers the probability of getting the answer correct by random guessing
Shortcomings
Not free of subjectivity
Cannot anticipate all possible responses
Tends to lower the reliability of the obtained score
A teacher may have to make judgments as to the correctness of what the students write. (Spelling, grammar, legibility)
The capital of Texas is ________? Students can answer this in many different ways.

6. Crafting Short-Answer Items
Importance of what is assessed
How does it match the test blueprint
Question format is the preferred format for a short-answer item. Focus the item on the specific knowledge sought
Word the items specifically and clearly
Put the blank near the end of the sentence
If possible do not copy statements verbatim
The author of Alice in Wonderland is _______.
What is the pen name of the author of Alice in Wonderland? ________.
Word your questions so that they require a single correct answer.
Putting the blank at the end of the sentence will prevent the student from having to mentally rearrange the question.
Verbatim encourages rote memorization.

7. Crafting Short-Answer Items
A completion item should omit important words and not trivial words
Limit blanks to one or two
Keep all blanks the same length
Specify the precision you expect in your answer
Avoid irrelevant clues

8. True-False Items
Consists of a statement or proposition the student may judge as true or false
Six varieties:
True-False
Yes-No
Right-Wrong
Correction
Multiple True-False
Yes-No with explanation

9. Usefulness of True-False Items
Advantages
Certain aspects of the subject matter readily lend themselves to verbal propositions that can be judged true or false
Relatively easy to write
Scored easily and objectively
Can cover a wide range of content within a relatively short period
Criticisms
Poorly constructed assess only specific, frequently trivial facts
Ambiguously worded
Answered by random guessing
Encourage students to study and accept only oversimplified statements of truth and factual details

10. Usefulness of True-False Items
Generalizations in a subject area
Comparisons among concepts
Casual or conditional propositions
Relationships between two events, concepts, facts, or principles
Explanations for why events or phenomena occurred
Instances or examples of a concept or principle

11. Usefulness of True-False Items
Evidential statements
Predictions about phenomena or events
Steps in a procedure of process
Computations (or other kinds of results obtained from applying a procedure)
Evaluations of events of phenomena

12. Validity of True-False Item Format
Essence of educational achievement is the command of useful verbal knowledge
All verbal knowledge can be expressed in propositions
Proposition is any sentence that can be said to be true or false
Particular area of knowledge is indicated by success in judging truth or falsity of propositions

13. Guessing on True-False Items: An Ill Wind?
Guessing – getting 50% correct
Blind (complete random guessing) quite unlike informed guessing (based on partial knowledge)
Well-motivated guess blindly only on a small percentage of questions on a test
Difficult to obtain a good score on a test by blind guessing only
A given true-false test has a high reliability coefficient – evidence blind guessing does not seriously affect the scores

14. Guessing on True-False Items: An Ill Wind?
Random guessing – sometimes called blind guessing
Partial knowledge – make an informed guess
Higher than chance of success
Measures of partial knowledge
Full knowledge – 100% chance of answering correctly
Percentages of correct answers go down with the number of blind guesses made

15. Suggestions for Getting Started Properly
Create pairs of items, one true and one false
Rephrase evaluative judgments into a comparative statement
Write false statements that reflect the actual misconceptions held by students
Convert a multiple-choice item into two or more true-false items

16. Suggestions for Improving True-False Items
Importance of what is assessed
How does it match the test blueprint
Assess important ideas, rather than trivia, general knowledge, or common sense
Make sure it is either definitely true or definitely false
Use short statements when possible
Use exact language
Use positive statements and avoid double negatives

17. Suggestions for Improving True-False Items
Avoid copying sentence verbatim
True and false statements should have approximately the same number of words
Don’t present in a repetitive or easily learned pattern
Do not use verbal clues that give away the answer
Attribute the opinion in a statement to an appropriate source
Focus on one idea

18. Crafting Multiple True-False Items
Looks like a multiple-choice item – followed by several alternatives
No single correct or best answer
Response to each alternative is true or false
Each alternative is scored correct or incorrect

19. Crafting Multiple True-False Items
Advantages
Two or three multiple true-false responses in the time it takes to make one multiple-choice response
Multiple true-false tests created from multiple-choice items have higher reliability than original multiple-choice
Can assess same abilities as straight multiple-choice items that are crafted to assess parallel content

20. Crafting Multiple True-False Items
Advantages
Students believe multiple true-false items do a better job of assessing knowledge than multiple-choice items
Students perceive them to be harder
May be easier to write – not limited to one correct answer
Limitations
Shares some of same limitations as multiple-choice

21. Works Cited
Nitko, A. J., (2004). Educational Assessment of Students, 4th ed. Upper Saddle River, NJ: Pearson Education, Inc.