1. McGraw-Hill/Irwin © 2012 The McGraw-Hill Companies, Inc. All rights reserved. Chapter 8: Summative Assessment Confirming Student Achievement

2. 8-2 The Function of Summative Assessment Summative assessment is recognized for its primary role in confirming student learning and accomplishment in the classroom. This is a necessary and important function that must be met and cannot be disregarded. Summative assessment should not overshadow other assessment forms, but unfortunately it can and does at times. Often happens due to inflated importance that is placed on it by the user (e.g., educators, administrators, legislators, general public, etc.).

3. 8-3 Summative Measures in the Classroom If done well, all assessment avenues enable students to perform at the highest levels possible which are hopefully demonstrated on formal summative measures. Summative measures used in the classroom usually take the form of tests and comprehensive exams. Due to time restrictions, ease of administration, and other related factors, tests are the preferred measure in the formal assessment of learning.

4. 8-4 Classroom Test A classroom test consists of a collection of items that are designed to measure and evaluate a learner’s skills, knowledge, general performance, or other capabilities as they relate to the content and lesson objectives that are covered as part of the instructional process. A classroom test serves as a formal assessment of the students’ accomplishment of the intended learning objectives. This work is usually graded and evaluated according to set procedures and/or scales within the school district.

5. 8-5 Teachers Must Construct Effective Summative Assessments Summative assessments, whether they are end-of- lesson tests, performance checklists, or classroom projects are used to measure and evaluate student learning progress in the classroom. They serve a critical role in the “assessment of learning” where every student’s accumulated progress can be measured and documented. Generating effective summative assessments is essential as teachers bear responsibility for the construction and administration of the assessments used to directly measure learning gains.

6. 8-6 Planning Your Test When putting together a summative measure, usually a test, it is essential to plan out and organize the test before it is administered. As you construct the test framework and decide on the items that make the test, it is important to confirm and check-off that basic assessment actions have taken place during the test-construction process.

7. 8-7 Summative Assessment Construction Rules 1. As an instructor, you need to be clear about what you want to accomplish with your summative assessment. Make sure students are aware of what they are supposed to know and demonstrate as part of their testing performance. Ensure that the items or tasks match up with the objectives and outcomes of the lessons. Whether test items are generated by you or through a commercial testing bank, you must ensure that the selected or constructed response items align with the instructional objectives of the lesson and the material that has been covered.

8. 8-8 Summative Assessment Construction Rules 2. The skill and performance levels measured with the summative assessment need to be identified before it is administered. Both knowledge levels and cognitive processes need to be identified with the test items and/or tasks. Make sure the material has been taught and practiced at these levels as part of the daily classroom instruction.

9. 8-9 Summative Assessment Construction Rules For example, if a test over introductory material is constructed, it is likely that more basic skill levels (e.g., remembering, understanding, application) will be emphasized in the test items as compared to higher order cognitive processing skills. However, higher level skills can and should be measured so that an appropriate skill performance balance is demanded from the students. Also, students need to be familiar with and have practiced using the measurement forms that appear in the summative measure(s).

10. 8-10 Summative Assessment Construction Rules 3. A variety of selected and constructed response items should be considered if a summative test is going to be used. This should be done for a number of reasons. First, lesson objectives and desired student outcomes will likely necessitate several different response formats. For instance, if higher processing skills are directly connected to lesson outcomes, then student-generated responses should connect with questions that require those skills. In addition, if recognition and comprehension learning targets are identified, then selected response items such as multiple-choice or matching can be very useful in measuring those skills across a broad range of content and information.

11. 8-11 Summative Assessment Construction Rules Also, a variety of test items helps to create an unbiased testing field in regard to assessment practice. If a test is constructed that only has multiple-choice items, then the possibility of success is dramatically lessened for those students who have little confidence with this test item type or just don’t do well with reviewing and selecting from multiple responses. With a variety of items, difficulty with a certain item type is minimized by the opportunity to respond to the other item formats in the test.

12. 8-12 Summative Assessment Construction Rules 4. You need to make sure that the directions along with each test question are clear and understandable to any reader. Along with appropriate readability and clarity is the need to provide adequate time to complete the test. If time limitations do exist, then you need to make sure to include only the number of items that can be reasonably finished within those time limits. You don’t want to create a speed test where your focus is on how many items can be completed within a specified time period. Instead you want to create a power test where you obtain the strongest and most complete performance effort from a student where time restriction does not limit that performance.

13. 8-13 Summative Assessment Construction Rules 5. Accept the reality that time and effort are always necessary in developing any comprehensive test or other performance measure(s) and that is not going to change. Every summative evaluation can be improved upon so seize every opportunity. Need to review each and every test administration, so each one can be strengthened and be more effective in the future.

14. 8-14 Summative Assessment Construction Rules 6. As a pre-service teacher, you will continue to learn about the effective use of assessment in your classroom and in examining and monitoring the learning progress of your students. Even after years of teaching, you will still be learning new information and more effective ways of enhancing the instruction-assessment process. Be comfortable with the fact (as well as humbled) that you will always be learning and improving your assessment skills.

15. General Test Construction Rules Principles to Follow

16. 8-16 General Test Construction Rules Good test construction practices reflect the use of sound measurement practices along with good common sense. A test should involve the careful consideration and selection of various items and questions.

17. 8-17 Test Development Regardless of what items are used in a test, the following rules should be considered: Ensure proper item coverage based on specified goals and outcomes. Complete a Table of Specifications to determine the knowledge and cognitive processing levels and skills covered with the test. Include a listing of what test items are covering specific knowledge and cognitive levels.

18. 8-18 Test Development Do not use unfamiliar item formats that the learners have not seen before. Learners should be familiar with every type of item used in the test. To gain test-item familiarity, use test items in formative quizzes or other activities prior to the summative test. After the instruction is finished, generate a test review providing an accurate and complete review of the material the test will cover. This will ensure that all students are fully informed and aware of what will be covered with the test. The value of the test, along with the point values of the various test items, should be provided.

19. 8-19 Test Construction All test items should be clear and concise, but complete and fully understandable. A variety of test item types should be considered and incorporated within a test. This helps eliminate any construction bias favoring a specific response type. For example, if all or most of the test are essay questions, that would favor students who have confidence with this type of question or possess strong writing skills. Clear and understandable directions need to be provided at the beginning of the test.

20. 8-20 Test Construction Questions should be constructed so they reflect a singular, clear statement or question. Should not be designed to mislead the reader. Test questions need to accurately determine if someone knows specific information or can perform a certain skill set. Test questions should not be constructed to see if students can avoid being fooled or misdirected by the question. If selected response items are used, make sure to provide appropriate and reasonable alternative responses for each question. For example, if a question requires the students to identify the largest planet in the solar system, then provide names of real planets and do not provide answer responses that are not related to the subject matter.

21. 8-21 Test Construction Only provide the number of questions that can be completed based on the amount of time that is available to the student. If a majority of selected response questions are used, then less time would be required as compared to a test that possesses a considerable number of essay questions. Typically, test items require at least one to two minutes just to be read, understood, and for answers to be considered. If constructed response items have higher point values relative to other test items, then indicate that value next to each of the questions.

22. 8-22 Test Implementation Administer the test within one session as this will help to ensure fair and uniform examination conditions. If more time is required, it is best if additional time is provided immediately after class and not on a different day. Always provide adequate time for a students to complete the test or examination. Always be available to provide clarification or explanation of an test item, if necessary. Maintain a quiet classroom environment that is free of distractions, so all students can focus their attention on the demands of the test.

23. 8-23 Test Review As the teacher, it is imperative to examine the collective testing performance of your students. Determine what areas or content was easy, as well as what the students had difficulty understanding Complete an item analysis of the test where each test item is examined relative to: The number of students that passed the item The scoring or rating frequencies that were evidenced for essay questions Recording your impression of the student responses and their general performance

24. 8-24 Test Review Learn from and improve upon test construction after each administration. Identify what worked well and what did not. This feedback and review is invaluable in the test construction process and for future examinations. Remember that well-stated test questions connected with clear directions and instructions contribute greatly to the reliability and validity of the test results and consequently the decisions that are made based on those results.

25. Selected and Constructed Response Items Basic Information and Principles to Follow

26. 8-26 Selected and Constructed Response Items Two kinds of test items, selected and constructed response, are typically used when making up a test. With selected response items, students are required to identify the correct answer from a set of provided choices or options. The most common types of selected response test items include multiple-choice, True-False, and matching. The use of one or more of these test-item forms is common in classroom tests across all grade levels. In the constructed response area, the most common types of test questions are short answer and essay. With these items, no alternative response items are provided as the respondent must generate the answer material to a question(s).

27. 8-27 Selected Response: Multiple Choice As reported in Popham (2008), there are basically two types or forms of multiple-choice questions. 1) The direct-question form or question that is designed to have the respondent select the best answer from the provided responses. 2) The incomplete-statement form that is constructed so that the learner needs to identify the correct answer. Every multiple-choice item is made of a stem which serves as the primary question or statement. The stem should be constructed so that it can be completely understood by the reader. The stem is then followed by a set of provided alternatives or responses, usually a set of four, that the learner selects from, one of which is the correct answer to the question. The others are wrong answers called distracters.

28. 8-28 Selected Response: Multiple Choice Multiple-choice items are popular and effective. They are relatively fast and easy to score. A moderate number of items can potentially cover a large amount of information and content. One of the greatest advantages is that these items can be constructed to evaluate a wide range of cognitive processes (e.g., retention and knowledge-based responses to questions that involve application and analysis skills).

29. 8-29 Multiple-Choice Test Item Do’s and Don’ts When constructing test items, it is best to generate the constructed question or statement in a positive form. This is to enhance the comprehension and general reading flow of the item. If a negative is used in a test item, it should be clearly recognized (e.g., capitalized and bolded) and placed at the end of the question if at all possible.

30. 8-30 Multiple-Choice Test Item Do’s and Don’ts Make sure that the answer responses make grammatical sense with the question stem so that all alternatives present as complete and logical responses. Avoid the use of articles “a” or “an” at the end of the stem, and if used list as a(n) so that both articles must be considered in the test item. Can provide unwarranted clues as to the correct answer based solely on grammatical alignment. When constructing responses or alternatives, make sure all the responses are possible and are connected to the content. They need to be viewed as viable and legitimate answers to the question. Varying the word length of the responses is necessary since in general longer answers tend to be correct and test-wise students often identify this trend. Providing correct answer responses that are short, moderate, and long in regards to sentence length can help minimize this tendency.

31. 8-31 Multiple-Choice Test Item Do’s and Don’ts General test construction wisdom dictates that certain alternative responses are to be avoided or used on a restricted basis. For example, the response “all of the above” is not an effective response option due to the fact that if only two of the presented responses can be determined to be true then this response becomes the automatic choice and answer. If present, this response option can artificially inflate the passage rate of an item and it has little to do with genuine knowledge on the student’s part. The “none of the above” response can and should be used when it is desirable to increase the difficulty level of the item. This response requires the learner to examine and eliminate all remaining item responses as not being viable answers and that may be necessary for the design of some test items.

32. 8-32 Multiple-Choice Test Item Do’s and Don’ts It is important as the test constructor to make sure that you randomly alternate the position (e.g., A, B, C, or D) of the correct responses across test items. Each letter used in the response should demonstrate about the same percentage usage across the entire test. Also, never intentionally embed a specific pattern or try to spell out words with the letters of the responses. Make sure no item is dependent upon the answer of another item. Each item should remain independent of all other items.

33. 8-33 Selected Response: True- False Binary choice items are test items that have only two possible response options This format is often used in classroom tests. Different response sets may exist, but they are essentially the same type of item. The true-false response set is common and most widely recognized of the different variations.

34. 8-34 True-False Test Item Do’s and Don’ts Consistent with multiple-choice items, true-false items should be centered on a singular main point within the item statement. The decision on whether the statement is true or false should be contingent on that statement alone. Use only the words that are necessary and avoid any needless detail within the statement. The statement should be verifiable and the decision response should be clear to a knowledgeable respondent.

35. 8-35 True-False Test Item Do’s and Don’ts Be very selective with the use of negative statements. Respondents don’t always recognize the “not” in a statement and are likely to misinterpret the statement because of it. Also, double negatives should be avoided entirely due to the inevitable confusion that is created when they are used in a test item. Generate an approximately equal number of true and false answered items within a test. Make sure the items are of comparable length regardless of being true or not. As a general pattern, true responses tend have longer statements and false responses are often associated with shorter statements.

36. 8-36 Selected Response: Matching Matching items involve combining a word or statement from one column recognized as premises, with its correct and corresponding answer from a second column known as responses. A matching question format can be used to determine whether students can align basic terms, vocabulary, or other item sets with their corresponding definitions, functions, or connections based on some identified relationship.

37. 8-37 Matching Test Item Do’s and Don’ts Provide clear directions so that the respondent clearly understands what he or she must do with the matching items. Homogeneous lists need to be used with all matching items. Make sure that both columns contain items that are clearly related and similar in regards to the attribute that is being examined. For example, if matching chemical compounds with their molecular formulas, make sure all listed items are indeed chemical compounds and appropriate formulas. Do not provide a list of inconsistent items. Concise item lists are desirable and provide for greater student understanding of the matching exercise.

38. 8-38 Matching Test Item Do’s and Don’ts Provide a limit of no more than ten premises for each matching test exercise. Beyond that number, the visual connections and alignments become very confusing and increase extraneous errors and misunderstandings. It is recommended to list more responses than premises in all matching items. The respondent must be able to accurately identify each match as opposed to the easier task of aligning an equal number of premises and responses. When listing items within a column, use some standard ordering method so a consistent listing method is provided. Both columns must fit onto one single page of a test. Actually, for all test items, it is essential that no question be “split” so that part of the item is on one page and the other section is on the next page.

39. 8-39 Student Constructed Response: Fill- in-Blank/Listing Questions, Short Answer & Essay Questions Within the standard classroom test, the most commonly used constructed responses are fill-in- blank, listing questions, short answer and essay questions. Fill-in-blank (FIB) items involve having the learner complete a test statement by providing a correct missing word intentionally left out. The test question, usually listed as a statement, is provided in its entirety except for the blank that requires the missing word.

40. 8-40 Student Constructed Response: Fill- in-Blank/Listing Questions, Short Answer & Essay Questions Since short-answer items require a minimal written response, they are particularly helpful in evaluating and measuring knowledge retention and the general comprehension of a student. The major advantage with this item type is that the respondent is required to actually produce the correct response as opposed to “discovering” the correct answer from a set of provided responses.

41. 8-41 Short Answer/Completion Test Item Do’s and Don’ts The teacher must generate precisely worded and concise questions or statements that are understandable to every respondent. That determination should be made based on the age and general reading proficiency of the learners. With short-answer items, writing precision is important since these items require very specific responses for answers. Construct the test item, so a clear and obvious answer response exists. Avoid constructing items that can potentially have multiple correct responses. Make sure to identify the correct responses for all short- answer items before the test is administered. If additional responses are subsequently found that are also technically correct, then the item should be rewritten so that only one answer exists.

42. 8-42 Short Answer/Completion Test Item Do’s and Don’ts Avoid generating completion items that possess more than one blank in the item statement as this ends to create confusion and misunderstanding. It is better to have a blank at the end of the statement. This allows the respondent to better understand the item and determine what answer is needed. The actual length of the blanks in all the items should be uniform. This is done to eliminate providing an obvious clue that blank length relates to word length.

43. 8-43 Student Constructed Response: Fill- in-Blank/Listing Questions, Short Answer & Essay Questions The most common and effective constructed response item utilized for the examination of higher-level thinking and processing is the essay question. The essential high standard for student-generated work on written tests. This type of item can be designed and crafted so that specific and detailed student responses are required that can be aligned with identified instructional goals and objectives. Yet, with this format, there exists an extensive range of student information that can be gathered and reviewed. For example, with an essay item, students can provide the teacher with a sample of their understanding, analysis, or evaluation of the principles or theories associated with a specific content or knowledge base.

44. 8-44 Student Constructed Response: Fill- in-Blank/Listing Questions, Short Answer & Essay Questions When constructing essay questions, two basic forms or types are recognized: 1) Restricted 2) Extended response Both are useful and effective for their intended purposes. Restricted-response items are designed to set particular perimeters on a student’s response to a question. Particularly as it relates to the focus and potential length of response. Response restrictions can be generated through the use of specific guiding words within the question or statement, as well as setting limits to the actual length of the written response. If an objective and uniform scoring method is in place, these items can be scored with a high level of reliability and consistency.

45. 8-45 Student Constructed Response: Fill- in-Blank/Listing Questions, Short Answer & Essay Questions The extended response allows a respondent maximum flexibility in organizing and constructing a written response to a question. Typically, few restrictions are included and the respondent can execute “free thinking” in the generation of the response. The benefit of this question format is the opportunity to examine each student’s highest levels of thinking and processing. If constructed well, extended response items can examine the highest analysis, evaluation, and creation skills that are rarely tapped with other test-item formats. Due to the large time and effort required for this type of item, only a limited number of these items are usually included in a test. These questions must be carefully selected and placed within assessments.

46. 8-46 Essay Test Item Do’s and Don’ts Provide a clear and readable question or statement that includes the demands of the question and what should be provided in response to the question. Provide direction as to specific requirements that may exist with the essay question. For assessment purposes, an essay question shouldn't be designed to be a nebulous, free floating treatise with no response boundaries or measurement direction. Prior to any exam, and as part of the scoring key, a model answer or response needs to be constructed along with the essential components that the answer must possess in order to receive scoring credit. For scoring accuracy, if a question is worth several points, the value for each of the elements of the answer should be determined.

47. 8-47 Essay Test Item Do’s and Don’ts Essay questions require more testing time, and that need must be factored into the administration of the exam. The extensiveness of the question will determine the time- response requirements. Allocate more time than is considered necessary to ensure an adequate latency exists for all respondents. Given the extensive response for essay questions, it is important to list the point value of each question so that each respondent identifies the relative importance of the question being answered. Essay items place a premium on writing ability. Those who demonstrate comfort and/or skill with generating on- demand writing responses are at a distinct advantage over those who do not.

48. 8-48 Essay Test Item Do’s and Don’ts Ensure that your students are familiar with responding to essay questions, and the higher level responses that can go with them, by providing in- class opportunities where essay questions are given as part of the instructional process. When scoring essay questions, review all student responses to the same question before moving on to other questions. When scoring the answer, do not attempt to identify the respondent, as this will help to control for instructor bias and enhance objectivity in the scoring process.

49. 8-49 Constructed Responses Beyond the Traditional Test Within a test, constructed response items require students to generate their own answers in response to pre-selected questions or other teacher-based requests. However, in the classroom or school setting, student generated work can involve and encompass productions that are not included as part of a traditional test. These productions can center on written work, visual productions, verbal performances, and fine/gross motor-based events. Given this variety of evidence products, teachers can utilize a wide range of assessment options to measure the learning progress of their students.

50. 8-50 Written Construction Performances Common sources of written evidence that are used in the classroom include logs and journals. A documentation log involves a written record of specific events or conditions over a certain amount of time. Entries can incorporate verification of behavior with brief descriptions or summaries, as well as personal impressions of what was experienced and what was particularly important at that moment of documentation. Logs are often kept during scientific studies, experiments, or other long-term projects that require information and evidence collection.

51. 8-51 Written Construction Performances Information is also written and recorded in journals. The information collected is likely to be more general as opposed to the more specific or restricted collection channels that may be specified with logs. Personal, anecdotal information complete with opinions, perceptions, feelings, and other subjective elements is more likely present in journal entries. Students can also record daily journal entries into an audiocassette recorder, so journal information does not have to be written. However, audio entries are sometimes translated into some kind of written record.

52. 8-52 Portfolios An assessment method that involves the collection of written evidence is the portfolio. In a portfolio, student-generated work is systematically collected over a certain period of time and reviewed and evaluated throughout that instructional period. Different subjects and/or skills areas can be emphasized or integrated together. However, teachers often follow a singular content as evidenced in a writing portfolio. Here students periodically generate pieces of work that are reviewed and placed in the portfolio.

53. 8-53 Portfolios There are different purposes for collecting and reviewing portfolios. For example, in a developmental or growth portfolio the progress of the writing process of the student is highlighted and emphasized. In a best pieces portfolio, the emphasis is on the best constructions of the various assignments that are required over the instructional period. Professional portfolios found in art, photography, music, as well as other professions typically represent best pieces portfolios. Other types of portfolios that have been identified for use in the classroom include memorabilia, skills, and proficiency or promotion portfolios (Butler & McMunn 2006).

54. 8-54 Visual Construction Performances Written productions can also be evidenced through the construction of visual symbols and other visual designs including: Illustrations, diagrams, graphs, matrices, concept maps, and flowcharts. These pictorial constructions can serve as important measures and demonstrations of acquired learning. Concept maps can serve as visual connectors used to compare and contrast various concepts or ideas, as well as provide visual links to subordinate or supportive details and facts.

55. 8-55 Visual Construction Performances Flowcharts typically involve the use of geometric shapes and directional arrows to demonstrate the direction or sequence to an event (e.g., chemical reactions, physiological processes, historical events, steps in solving a problem, etc.). Tables, matrices and graphs can list and present information in meaningful visual ways, and that is why they are so often present in books and other learning materials. Graphs are particularly helpful in representing and clarifying complex relationships or organizing large amounts of information into visually meaningful units. Matrices also help to organize large amounts of information into focused and meaningful data laid out across selected rows and columns.

56. 8-56 Verbal Construction Performances Verbal presentations and demonstrations can take various forms, such as providing responses to teacher-directed questions in class, speeches, organized debates or panel discussions, dramatic readings, or singing performances as part of choir or vocal ensemble events. Beyond classroom questions and answers, refined and focused oral demonstrations as evidenced in speeches or debates can provide specific information on effective communication skills and impact. Ratings and checklists connected to measure these events can provide useful evidence and guidance regarding the effectiveness of student communication.

57. 8-57 Fine/Gross Motor Construction Performances Fine and/or gross motor performances that can serve as evidence events include: Drawing or painting, dance performances and recitals, as well as athletic demonstrations and performances. In the worlds of art, music, and athletics, motor performances and products are considered essential and basic to the progress review. Knowledge and skill, along with competition issues, are all wrapped up in performances that are often provided in front of an audience or that involve public review.

58. 8-58 Reviewing Performances with Students The results from summative assessments are typically used for formal and official documentation of students’ progress and performance levels. Beyond the official function of summative assessments, there is an equally important purpose of providing useful information and feedback to students in regards to their performance.

59. 8-59 Reviewing Performances with Students Some teachers may be reluctant to share results or take time out of class to examine test results either for individuals or for the class as a collective group. Recognizing that instructional time is always limited, reviewing assessment performances, especially summative assessments, is not a valueless activity. On the contrary, it can be invaluable in recognizing performance issues that will likely be utilized and required for the next formal assessment.