Week 4 11/23/07

Five General Item-Writing Commandments 1. Thou shall not provide opaque directions to students regarding how to respond to your assessment instruments.

Five General Item-Writing Commandments 2. Thou shall not employ ambiguous statements in your assessment items. Sentences in which pronouns are used can frequently be unclear about the individual or individuals to whom the pronouns refers.

Five General Item-Writing Commandments True or False: Leaders of developing nations have tended to distrust leaders of developed nations due to their imperialistic tendencies. The word their could refer to the leaders of developing or developed nations.

Five General Item-Writing Commandments 3. Thou shall not provide students with unintentional clues regarding appropriate responses. Thus, we want you to intentionally avoid unintentional clues.

Five General Item-Writing Commandments Inexperienced item writers often tend to make the correct answer to a multiple-choice items twice as long as the incorrect answers. Absolute qualifiers such as never and always are sometimes used for the false items in a true false test. Even uninformed students know that there are few absolutes in this world.

Five General Item-Writing Commandments Here is an unintended clue: The bird in the story was an… A. Falcon B. Hawk C. Robin D. Owl Can you guess the correct answer, even though you did not read the story?

Five General Item-Writing Commandments 4. Thou shall not employ complex syntax in your assessment items. Avoid complicated sentence constructions and, instead, use very simple sentences.

Five General Item-Writing Commandments 5. Thou shall not use vocabulary that is more advanced than required. You should eschew obfuscative verbiage.

Binary-Choice Items Two options from which to select. (True-False, Yes-No, Correct-Incorrect, Fact-Opinion) Good things…so terse, that students can answer many items in a short time. You can cover a lot of content. Bad thing…50-50 chance.

Item-Writing Guidelines for Binary-Choice Items 1. Phrase items so that a superficial analysis by the student suggests a wrong answer. (Phrase the items to elicit thoughtfulness)

Item-Writing Guidelines for Binary-Choice Items 2. Rarely use negative statements, and never use double negatives. Example: The League of Nations was not formed immediately after the conclusion of World Ware II. Or The League of Nations was formed immediately after the conclusion of World War II.

Item-Writing Guidelines for Binary-Choice Items Example: The League of Nations was not formed immediately after the conclusion of World Ware II. True Or The League of Nations was formed immediately after the conclusion of World War II. False

Item-Writing Guidelines for Binary-Choice Items 3. Include only one concept in each statement. This is common sense…one concept at a time please!

Item-Writing Guidelines for Binary-Choice Items 4. Have an approximate equal number of of items representing the two categories being tested.

Item-Writing Guidelines for Binary-Choice Items 5. Keep item length similar for both categories being tested. Avoid systematic patterns wherein long statements are true and short statements are false.

Multiple Binary-Choice Items A multiple binary-choice item is one in which a cluster of items is presented to students requiring a binary response to each of the items in the cluster.

Multiple Binary-Choice Items Rules: 1. Separate item clusters vividly from one cluster to another. 2. Make certain that each item meshes well with the cluster’s stimulus material.

Mode, Median and Mean Mode: The mode is the most frequent piece of data in a frequency distribution.

Mode, Median and Mean Median The median measures the middle position of a frequency distribution for a group of data. In order to calculate the median we need to order our group of data from the lowest (the 1 st score in our order) to the highest (the last score in our order). How we calculate the median depends on whether there is an even or odd number in your group of data.

Mode, Median and Mean Median: Even numbers: If the group of data contains an even number of scores; in other words, if your group of data contains 100 scores, for example, then we calculate the median by adding the two middle scores together and dividing them by two.

Mode, Median, Mean Median: As a result, we must take the middle two values, namely the 50th and 51st scores, add them together (for example…58 + 59 = 117) and divide them by 2 (117 ÷ 2 = 58.5). Hence, the median for our group of data is 58.5 out of 100.

Mode, Median, Mean Median: Odd numbers: If the group of data contains an odd number of scores; in other words, if your group of data contains 9 scores, for example, then the median will simply be the middle score, the score of the 5th student.

Mode, Median, Mean The median is considered to be a good measure of central tendency because it describes the middle position of a frequency distribution for a group of data. However, it has one main weakness. Like the mode, the median fails to take into consideration all the values in the group of data.

Mode, Median, Mean Mean The mean, otherwise called the arithmetic mean or average, is the most commonly used measure of central tendency. In order to calculate the mean, we simply add up all the values in the group of data and divide them by the number of values.

Mode, Median, Mean As a result, the mean is much more effective at measuring the central position in a frequency distribution because it takes into account not only the total number of scores but also their distance from the central position.

Range In descriptive statistics, the range is the length of the smallest interval which contains all the data. It is calculated by subtracting the smallest observations from the greatest and provides an indication of statistical dispersion. It is measured in the same units as the data. Since it only depends on two of the observations, it is a poor and weak measure of dispersion except when the sample size is large.

Multiple Binary-Choice Items … Suppose that a dozen of your students completed a 10 item multiple-choice test that earned the following number of correct scores: 5,6,7,7,7,7,8,8,8,8,9,10 9. The median of your students’ scores is 7.5 True or False The mode for the set of scores is 8.0 True or False The range of the students’ scores is 5.0 True or False The median is different than the mean. True or False

Multiple-Choice Items The strength of multiple-choice questions is they can contain several answers that differ in their relative correctness, thus, the student can be called upon to make subtle distinctions among answer options. A weakness is that students need only recognize the correct answer not generate the correct answer.

Multiple-Choice Items The first-part of a multiple-choice item is referred to as the item’s stem. The potential answer options are described as item alternatives. Incorrect alternatives are typically referred to as the item’s distracters. Item stems can be a direct question or an incomplete statement. With younger students the direct question approach is preferable. Using these stems, you can ask the student to select the best answer or the best answer.

Multiple-Choice Items 1. The stem should consist of a self-contained question or problem. It should be loaded with as much content as possible so that the student knows exactly what they are to consider.

Multiple-Choice Items 2. Avoid negatively stated stems. If you use the word not in a multiple-choice stem, be sure to highlight it so that the student sees and pays attention to this negative.

Multiple-Choice Items Which one of the the following cities is located west of the Mississippi? San Diego, Pittsburgh, Boston or Atlanta Which one of the following cities is not located in a state east of the Mississippi? San Diego, Pittsburgh, Boston or Atlanta

Multiple-Choice Items Do not let the length of alternatives supply unintended clues. Keep it short and sweet or make all lengthy…you make the call.

Multiple-Choice Items Randomly assign correct answers to alternate positions. As a rule, if you have four choices, make sure the correct answer is assigned to each position (a,b,c,or d,) 25% of the time.

Multiple-Choice Items 5. Never use all-of-the-above alternative but do use none of the above alternatives to increase item difficulty.

Matching Items A matching item consists of two parallel lists of words or phrases that require the student to match entries on one list with appropriate entries on the second list. Entries in the list for which a match is sought are referred to as premises. Entries in the list from which selections are made are referred to as responses.

Matching Items Advantage: Easily scored Takes up little space and taps a good deal of information efficiently Disadvantage: Encourages memorization of low level factual information.

Matching Items Matching Item-Writing Guidelines 1. Employ homogeneous lists 2. Use relatively brief lists, placing the shorter words or phrases at the right. 3. Employ more responses than premises. 4. Order the responses logically 5. Describe the basis for matching and the number of times responses may be used

Constructed-Response Items Short answer items and essays

Constructed-Response Items Short-Answer Items: Students are asked to supply a word, a phrase, or a sentence in response to either a direct question or an incomplete statement. The major advantage is that the student needs to produce a correct response answer, not merely recognize it from a set of selected-response options. The major disadvantage is that student responses are difficult to score. The longer the responses sought, the tougher it is to score them.

Five Guidelines for Writing Short Answer Items 1. Usually employ direct questions rather than incomplete statements, particularly for younger students. 2. Structure the item so that a response should be concise. 3. Place blanks in the margin for direct questions or near the end of incomplete statements. 4. For incomplete statements, use only one or, at most, two blanks. 5. Make sure blanks for all items are equal in length.

Essay Items: Development Essay items are particularly useful in gauging a student’s ability to synthesize, evaluate and compose. For assessing certain types of complex learning outcomes, the essay item is hands-down the winner. Disadvantages: Essay items are more difficult to write properly--than is generally thought. Scoring essays can be difficult also.

Essay Items: writing guidelines 1. Convey to students a clear idea regarding the extensiveness of the response desired 2. Construct items so that the student’s task is explicitly described. 3. Provide students with the approximate time to be expended on each item as well as each item's value 4. Do not employ optional items 5. Precursively judge an item’s quality by composing, mentally or in writing, a possible response.