Presentation is loading. Please wait.

Presentation is loading. Please wait.

Teacher Evaluation Institute Assessment Literacy for Student Achievement Goal Setting 1 Division of Teacher Education and Licensure September 2013.

Similar presentations


Presentation on theme: "Teacher Evaluation Institute Assessment Literacy for Student Achievement Goal Setting 1 Division of Teacher Education and Licensure September 2013."— Presentation transcript:

1 Teacher Evaluation Institute Assessment Literacy for Student Achievement Goal Setting 1 Division of Teacher Education and Licensure September 2013

2 Standard 7: The work of the teacher results in acceptable, measurable, and appropriate student academic progress. Virginia’s Uniform Performance Standards for Teachers 2

3 7.1 Sets acceptable, measurable and appropriate achievement goals for student academic progress based on baseline data. 7.2 Documents the progress of each student throughout the year. 7.3 Provides evidence that achievement goals have been met, including the state provided growth measure when available as well as other multiple measures of student growth. 7.4 Uses available performance outcome data to continually document and communicate student academic progress and develop interim learning goals. Standard 7: Performance Indicators 3

4 Assessment Data Sources Student Growth Percentiles (SGPs) based on Standards of Learning (SOL) assessments Other standardized assessments Common assessments (developed for the division or school level) Teacher-developed assessments 4

5 Guiding Questions 1.What role can teacher-developed assessments play in demonstrating student learning? 2.How can the validity and reliability of teacher-developed assessments be strengthened? 3.How can teachers analyze results from students’ performance on teacher-developed assessments and demonstrate student progress? 4.How can a teacher and/or principal determine what currently available assessments may be appropriate for establishing baseline data for student achievement goal setting? 5

6 Guiding Question 1 What role can teacher-developed assessments play in demonstrating student learning? 6

7 Standard 4: The teacher systematically gathers, analyzes, and uses all relevant data to measure student academic progress, guide instructional content and delivery methods, and provide timely feedback to both students and parents throughout the school year. 7 Virginia’s Uniform Performance Standards for Teachers

8 Common Unit Test Key Skill Performance Assessment Demonstrated Student Progress Standard 7 Standard 4 Other Teacher- Developed Assessments 8

9 Guiding Question 2 How can the validity and reliability of these assessments be strengthened? 9

10 Assessment Literacy The ability to create and use valid and reliable assessments as a classroom teacher to facilitate and communicate student learning. 10

11 Desired Characteristics of Teacher-Developed Assessments Reliable Practical to Employ Instructionally Useful Valid 11

12 Overview of the Process for Creating a Valid and Reliable Teacher-Developed Assessment 1.Identify objectives for a unit of instruction. 2.Unpack the objectives. 3.Map the objectives onto a “table of specifications.” 4.Use the table of specifications to guide the construction of the assessment. 5.Use the table of specifications to analyze student learning. 12

13 Definition: Curriculum A set of objectives for student learning. 13

14 4.16The student will a)recognize and demonstrate the meaning of equality in an equation; and b)investigate and describe the associative property for addition and multiplication. UNDERSTANDING THE STANDARD (Background Information for Instructor Use Only) ESSENTIAL UNDERSTANDINGSESSENTIAL KNOWLEDGE AND SKILLS  Investigating arithmetic operations with whole numbers helps students learn about several different properties of arithmetic relationships. These relationships remain true regardless of the numbers.  The commutative property for addition states that changing the order of the addends does not affect the sum (e.g., 4 + 3 = 3 + 4). Similarly, the commutative property for multiplication states that changing the order of the factors does not affect the product (e.g., 2  3 = 3  2).  The associative property for addition states that the sum stays the same when the grouping of addends is changed [e.g., 15 + (35 + 16) = (15 + 35) + 16]. The associative property for multiplication states that the product stays the same when the grouping of factors is changed [e.g., 6  (3  5) = (6  3)  5]. All students should  Understand that mathematical relationships can be expressed using equations.  Understand that quantities on both sides of an equation must be equal.  Understand that the associative property for addition means you can change the groupings of three or more addends without changing the sum.  Understand that the associative property for multiplication means you can change the groupings of three or more factors without changing the product. The student will use problem solving, mathematical communication, mathematical reasoning, connections, and representations to  Recognize and demonstrate that the equals sign (=) relates equivalent quantities in an equation.  Write an equation to represent equivalent mathematical relationships (e.g., 4  3 = 2  6).  Recognize and demonstrate appropriate use of the equals sign in an equation.  Investigate and describe the associative property for addition as (6 + 2) + 3= 6 + (2 + 3).  Investigate and describe the associative property for multiplication as (3 x 2) x 4 = 3 x (2 x 4). 14

15 Sources of Objectives for Instructional Units Objectives for an instructional unit VA SOL Local curriculum (including text series) Teacher (as subject- area expert) What gets taught 15

16 Sample Objectives for a Unit on “Equivalency and the Associative Property” (SOL 4.16) The student will: a. Recognize and demonstrate the meaning of equality in an equation (i.e., an equals sign [=] relates equivalent quantities in an equation).  Write equations to represent equivalent mathematical relationships (e.g., 4 × 3 = 2 × 6). b. Investigate and describe the associative property for addition, such as (6 + 2) + 3 = 6 + (2 + 3).  Use mathematical communication to demonstrate understanding that the associative property for addition means you can change the groupings of three or more addends without changing the sum. c. Investigate and describe the associative property for multiplication, such as (3 x 2) x 4 = 3 x (2 x 4).  Use mathematical communication to demonstrate understanding that the associative property for multiplication means you can change the groupings of three or more factors without changing the product. d. Infer how the associative property does not operate for subtraction and division by exploring and proposing explanations of non-examples of the associative property. Source: VA SOL Source: The teacher 16

17 Unpacking Objectives Content Cognitive Level Objective 17

18 Unpacking Objectives The student will describe the planets and their relative positions from the sun. 18

19 Bloom’s Taxonomy of Cognitive Behaviors Evaluation Synthesis Analysis Application Comprehension Knowledge 19

20 Bloom’s Taxonomy Verbs Taxonomy LevelAssociated Verbs Knowledgecount, define, describe, draw, find, identify, label, list, locate, name, state, recall, recite, record, tell Comprehension clarify, classify, compare, contrast, describe, exemplify, explain, extrapolate, give examples, identify, illustrate, infer, interpret, outline, paraphrase, report, represent, restate, summarize, tell, translate Application apply, carry out, compute, demonstrate, dramatize, draw, execute, illustrate, implement, prepare, select, show, solve, transfer, use Analysis analyze, break down, characterize, deconstruct, deduce, differentiate, discriminate, distinguish, examine, infer, investigate, relate, separate Synthesis adapt, create, design, develop, formulate, integrate, invent, imagine, make, modify, perform, plan, predict, produce, propose Evaluation argue, assess, choose, conclude, critique, decide, evaluate, judge, justify, predict, prioritize, prove, rank, rate, select 20

21 Unpacking Objectives The student will describe the planets and their relative positions from the sun. comprehension 21

22 Unpacking Objectives for Content and Cognitive Level The student will: a.Recognize and demonstrate the meaning of equality in an equation (i.e., an equals sign [=] relates equivalent quantities in an equation). b.Investigate and describe the associative property for addition, such as (6 + 2) + 3= 6 + (2 + 3). c.Investigate and describe the associative property for multiplication, such as (3 x 2) x 4 = 3 x (2 x 4). d.Infer how the associative property does not operate for subtraction and division by exploring and proposing explanations of non-examples of the associative property. CO AP AN SY CO: comprehensionAN: analysis AP: applicationSY: synthesis 22

23 Unpacking Objectives The student will identify the narrator of a short story. The student will find the surface area of a rectangular prism. The student will list the qualifications necessary to vote in Virginia and describe the process for registering to vote in Virginia. The student will create a diagram that demonstrates the steps in the cell cycle, including the phases of mitosis. 23

24 Unpacking Objectives The student will identify the narrator of a short story. The student will find the surface area of a rectangular prism. The student will list the qualifications necessary to vote in Virginia and describe the process for registering to vote in Virginia. The student will create a diagram that demonstrates the steps in the cell cycle, including the phases of mitosis. CO AP KN CO KN: knowledge CO: comprehension AP: application 24

25 Unpacking Objectives The student will write a persuasive essay on a school- wide issue. The student will verify the properties of circles. By reviewing slogans in post-World War II America, the student will describe changes in economic opportunities for women. 25

26 Unpacking Objectives The student will write a persuasive essay on a school- wide issue. The student will verify the properties of circles. By reviewing slogans in post-World War II America, the student will describe changes in economic opportunities for women. EV AP AN AP: application AN: analysis EV: evaluation 26

27 Cautions About Unpacking 1.Bloom’s taxonomy may be incorrectly viewed as hierarchical in nature. 2.Not all learning objectives fit neatly into one category of the taxonomy. 3.Most words associated with the taxonomy have different meanings in different contexts. 4.There are other taxonomies of cognitive behaviors besides Bloom’s. 5.Bloom’s taxonomy addresses the cognitive domain only, not the psychomotor or affective domains. 6.Subject-matter expertise is critical in unpacking objectives. 27

28 Table of Specifications A blueprint for what should be included (and should not be included) on an assessment. 28

29 29

30 Content Cognitive Level KnowledgeCompre- hension ApplicationAnalysisSynthesisEvaluation 30

31 Steps to Create a Table of Specifications Step #1 – Unpack objectives for… Content Cognitive level. Step #2 – Plot the intersection between the content and level of cognitive demand for each learning objective on a matrix. Step #3 – If helpful, indicate the relative emphasis of each intersection on the chart. 31

32 Content Cognitive Level Know- ledge Compre- hension Appli- cation AnalysisSyn- thesis Evalu- ation the meaning of equality in an equation the associative property for addition the associative property for multiplication how the associative property does not operate for subtraction and division (teacher- developed objective) recognize demonstrate describe investigate describe investigate explore infer propose explana- tions Math 4.16 32

33 Using a Table of Specifications 1.To create an assessment. 2.To critique and improve an existing assessment. 3.To create a unit assessment plan. 33

34 Sixth-Grade Unit: “Resource Use and Conservation” (Science SOL 6.9) Science SOL 6.9 Overview “The strand focuses on student understanding of the role of resources in the natural world and how people can utilize those resources in a sustainable way. An important idea represented in this strand is the importance of managing resources. This begins with basic ideas of conservation and proceeds to more abstract consideration of costs and benefits. The topics developed include conservation of materials, soil and plants as resources, energy use, water, Virginia’s resources, and how public policy impacts the environment.” 34

35 Objectives for the “Resource Use and Conservation” Unit (Science 6.9) The student will: a.Differentiate between renewable and nonrenewable resources. b.Describe the role of local and state conservation professionals in managing natural resources. These include wildlife protection; forestry and waste management; and air, water, and soil conservation. c.Analyze reports, media articles, and other narrative materials related to waste management and resource use to determine various perspectives concerning the costs/benefits in real-life situations. d.Analyze how renewable and nonrenewable resources are used and managed within the home, school, and community. e.Analyze resource-use options in everyday activities and determine how personal choices have costs and benefits related to the generation of waste. f.Evaluate the impact of resource use, waste management, and pollution prevention in the school and home environment. 35

36 Content Cognitive Level Know- ledge Compre- hension Applica- tion Analy- sis Synthe- sis Evalua -tion Renewable and nonrenewable resources Differentiate between Role of local and state conservation professionals in managing natural resources Describe Reports, media articles, etc., that give various perspectives on costs/benefits in real-life situations Determine Analyze Resource use and management in the home, school, and community Analyze Resource use options in everyday activities and cost/benefits of personal choices in relation to waste generation Analyze Deter- mine personal choices Impact of resource use, waste management, and pollution prevention in school and at home Evaluate What would you expect to see on a test that assesses these objectives? 36

37 Unit Test Question #1 Which of these is a nonrenewable resource? A.fossil fuels B.sunlight C.trees D.wind What content does this item assess and at what cognitive level? 37

38 Content Cognitive Level Know- ledge Compre- hension Applica- tion Analy- sis Synthe -sis Evalua- tion Renewable & nonrenewable resources Differentiate between Role of local and state conservation professionals in managing natural resources Describe Reports, media articles, etc., that give various perspectives on costs/benefits in real-life situations Determine Analyze Resource use and management in the home, school, and community Analyze Resource use options in everyday activities and cost/benefits of personal choices in relation to waste generation Analyze Deter- mine personal choices Impact of resource use, waste management, and pollution prevention in school and at home Evaluate 1 38

39 Unit Test Question #9 What content does this item assess and at what cognitive level? 39

40 Content Cognitive Level Know- ledge Compre- hension Applica- tion Analy- sis Synthe- sis Evalua -tion Renewable & nonrenewable resources Differentiate between Role of local and state conservation professionals in managing natural resources Describe Reports, media articles, etc., that give various perspectives on costs/benefits in real-life situations Determine Analyze Resource use and management in the home, school, and community Analyze Resource use options in everyday activities and cost/benefits of personal choices in relation to waste generation Analyze Deter- mine personal choices Impact of resource use, waste management, and pollution prevention in school and at home Evaluate 1 9 40

41 Unit Test Question #14 What content does this item assess and at what cognitive level? 41

42 Content Cognitive Level Know- ledge Compre- hension Applica- tion Analy- sis Synthe- sis Evalua -tion Renewable & nonrenewable resources Differentiate between Role of local and state conservation professionals in managing natural resources Describe Reports, media articles, etc., that give various perspectives on costs/benefits in real-life situations Determine Analyze Resource use and management in the home, school, and community Analyze Resource use options in everyday activities and cost/benefits of personal choices in relation to waste generation Analyze Deter- mine personal choices Impact of resource use, waste management, and pollution prevention in school and at home Evaluate 1 14 9 42

43 Content Cognitive Level Know- ledge Compre- hension Applica- tion Analy-sisSynth e-sis Evalua -tion Renewable & nonrenewable resources 1, 2, 16 Differentiate Role of local and state conservation professionals in managing natural resources 11 Describe 20 5, 10 Reports, media articles, etc., that give various perspectives on costs/benefits in real-life situations Determine 9 7, 12, 15 Analyze 4, 8 Resource use and management in the home, school, and community 146 Analyze Resource use options in everyday activities and cost/benefits of personal choices in relation to waste generation 17 Analyze Deter- mine personal choices Impact of resource use, waste management, and pollution prevention in school and at home 3, 13, 18, 19 Evaluate How VALID is this test? 43

44 Content Cognitive Level Know- ledge Compre- hension Applica -tion AnalysisSyn- thesis Evalua- tion Renewable & nonrenewable resources 1, 2 Differentiate 5, 10, 16 Role of local and state conservation professionals in managing natural resources 11 Describe 7, 20, 24 Reports, media articles, etc., that give various perspectives on costs/benefits in real-life situations Determine 9, 12 Analyze 4, 8, 21 Resource use and management in the home, school, and community 146 Analyze 18, 19, 25 Resource use options in everyday activities and cost/benefits of personal choices in relation to waste generation 17 Analyze 15, 22, 23 Deter- mine personal choices Impact of resource use, waste management, and pollution prevention in school and at home 3, 13 Evaluate Is this a more valid test? 44

45 Content Cognitive Level KNCOAPANSYEV Renewable & nonrenewable resources Differentiate Role of local and state conservation professionals in managing natural resources Describe Reports, media articles, etc., that give various perspectives on costs/benefits in real-life situations Determine Analyze Resource use and management in the home, school, and community Analyze Resource use options in everyday activities and cost/benefits of personal choices in relation to waste generation Analyze Determine personal choices Impact of resource use, waste management, and pollution prevention in school and at home Evaluate Using a TOS as a “unit assessment plan” MC Test Personal Resource Use Project

46 Three Practical Uses of a Table of Specifications 1.To create an assessment. 2.To critique and improve an existing assessment. 3.To create a unit assessment plan. 46

47 Strengthening the Validity and Reliability of Assessments by Using a “Table of Specifications” Does the assessment align to the objectives not only in content but also in the targeted cognitive levels? Does the assessment adequately and proportionately cover the objectives? Does the assessment account for chance, error, and bias by providing more than one opportunity for the student to demonstrate learning on key objectives? Construct Validity Sampling Validity Repeated Trials Reliability 47

48 Guiding Question 3 How can teachers analyze results from students’ performance on teacher- developed assessments and demonstrate student progress? 48

49 Four Examples Performance Task Paper-Pencil Unit Test with Subsample Pre-assessment Paper-Pencil Unit Test Cumulative Test 49

50 Performance Task for Fourth-Grade Art 50

51 Fourth-Grade Art Unit: Perspective Drawing The student will: Use perspective drawing techniques to create a work of art that depicts a three-dimensional object on a two-dimensional surface. (SOL 4.9) Use a variety of lines in the one point perspective drawing. (SOL 4.6) Use characteristics of color, including hue, tint, shade, and intensity in the one point perspective drawing. (SOL 4.4) AP SY AP 51 AP: application SY: synthesis

52 Content Cognitive Level Know- ledge Compre- hension Applica- tion AnalysisSyn- thesis Evalua- tion Perspective drawing techniques to create a work of art that depicts a three- dimensional object on a two- dimensional surface Use Create Use a variety of lines in the one point perspective drawing Use Characteristics of color in the one point perspective drawing Use Fourth-Grade Art Unit: Perspective Drawing 52

53 Three Elements of a Performance Task 1.Prompt 2.Response Format 3.Grading Criteria 53

54 Fourth-Grade Art Pre-Assessment Prompt and Response Format Teacher Directions: Have students look around the classroom from their perspectives and describe what they see. Ask them to pick a point of focus (e.g., the whiteboard, the classroom door) and describe what they see. Student Directions: Picking a point of focus, draw a picture showing your perspective on what you see in the classroom. Be sure to include as much detail as possible. Use appropriate lines, points, and color to provide a realistic perspective. Use colored pencils and the 11” X 14” paper provided for your drawing. 54

55 Grading Criteria Advanced (3)Proficient (2)Developing (1)Unacceptable (0) Element #1: One Point Perspective Drawing is in complete one-point perspective indicating an excellent level of craftsmanship in drawing and shading the room. Drawing includes most objects in the room drawn in accurate one point perspective, indicating a high level of craftsmanship in drawing and shading the room. The majority of the objects in the room are not drawn in one point perspective, indicating a low level of craftsmanship in drawing and shading the room. Unable to discern objects in room, indicating a low level of craftsmanship. Element #2: Design Principles Drawing indicates a complete understanding of line and how it is used to draw objects in one point perspective. Drawing indicates a mostly accurate understanding of line and how it is used to draw objects in one point perspective. Drawing indicates an unclear understanding of line and how it is used to draw objects in one point perspective. Drawing indicates no concept of line and how it is used to draw objects in one point perspective. Element #3: Use of color Excellent use of color, used multiple colors and layering to achieve extreme depth. Good use of color, mixing and layering achieves some depth. Basic use of color. Layers are thin, used few colors, little depth. Poor use of color. Did not layer or mix multiple colors, flat. 55

56 Content Cognitive Level Know- ledge Compre- hension Applica- tion AnalysisSynthesisEvalua- tion Perspective drawing techniques to create a work of art that depicts a three- dimensional object on a two- dimensional surface Use Element #1 Create Element #1 Use a variety of lines in the one point perspective drawing Use Element #2 Characteristics of color in the one point perspective drawing Use Element #3 Art Unit: Perspective Drawing 56

57 Fourth-Grade Art Post-Assessment Prompt and Response Format Teacher Directions: Take students into the library instructing students to sit down at tables. Using the digital cameras provided, have students take a picture of the library from their seats. Student Directions: Draw a picture showing your perspective on what you see in the library. Be sure to include as much detail as possible. Use appropriate lines, points, and color to provide a realistic perspective. Use the colored pencils and 11” X 14” paper provided for your drawing. 57

58 Grading Criteria Advanced (3)Proficient (2)Developing (1)Unacceptable (0) Element #1: One Point Perspective Drawing is in complete one-point perspective indicating an excellent level of craftsmanship in drawing and shading the room. Drawing includes most objects in the room drawn in accurate one point perspective, indicating a high level of craftsmanship in drawing and shading the room. The majority of the objects in the room are not drawn in one point perspective, indicating a low level of craftsmanship in drawing and shading the room. Unable to discern objects in room, indicating a low level of craftsmanship. Element #2: Design Principles Drawing indicates a complete understanding of line and how it is used to draw objects in one point perspective. Drawing indicates a mostly accurate understanding of line and how it is used to draw objects in one point perspective. Drawing indicates an unclear understanding of line and how it is used to draw objects in one point perspective. Drawing indicates no concept of line and how it is used to draw objects in one point perspective. Element #3: Use of color Excellent use of color, used multiple colors and layering to achieve extreme depth. Good use of color, mixing and layering achieves some depth. Basic use of color. Layers are thin, used few colors, little depth. Poor use of color. Did not layer or mix multiple colors, flat. 58

59 Teacher P Goal Statement During the course of this school year, all students will make measurable progress in U.S. History II as determined by the multiple choice and essay common assessment. Students will improve as follows: Students scoring between 0 and 30 percent on the pre-assessment will improve by 55 percentage points on the post-assessment. Students scoring 55 percent and higher on the pre-assessment will improve their scores by at least 30 percentage points on the post-assessment. Three Approaches to Pre- and Post-Assessment 1)A 10-question unit pre- assessment and 40-question unit post-assessment. 2)A 40-question unit pre- assessment and 40-question unit post-assessment. 3)A 40-question cumulative pre-assessment and 40- question cumulative post- assessment. 59

60 Goal Setting Sample for a Unit Assessment: 10-Question Pre-Assessment 40-Question Post-Assessment 60

61 U.S. History II.8b-e The student will demonstrate knowledge of the economic, social, and political transformation of the United States and the world between the end of World War II and the present by: b)describing the conversion from a wartime to a peacetime economy; c)identifying the role of America’s military and veterans in defending freedom during the Cold War, including the wars in Korea and Vietnam, the Cuban missile crisis, the collapse of communism in Europe, and the rise of new challenges; d)describing the changing patterns of society, including expanded educational and economic opportunities for military veterans, women, and minorities; e)describing how international trade and globalization have impacted American life. 61

62 U.S. History II.1 Make connections between past and present (USII.1b) Analyze and interpret primary and secondary source documents (USII.1a) Interpret slogans and documents (USII.1h) Identify the costs and benefits of specific choices made in trade and globalization (USII.1i) Interpret charts and graphs (school division addition) 62

63 Content Cognitive Level Know- ledge Compre- hension Appli- cation Analy- sis Syn- thesis Evaluation Percent of Questions Correct Conversion from a wartime to a peacetime (USII.8b) Describe 1 Make connections between past and present (USII.1b) 2 27.5 Role of America’s military and veterans in defending freedom during the Cold War, including the wars in Korea and Vietnam, the Cuban missile crisis, the collapse of communism in Europe, and the rise of new challenges (USII.8c) Identify 8, 9 Analyze and interpret primary and secondary source documents (USII.1a) 10 23.3 Changing patterns of society, including expanded educational and economic opportunities for military veterans, women, and minorities (USII.8d) Describe 3 Interpret slogans and documents (USII.1h) 4 20 The impact of international trade and globalization on American life (USII.8e) Describe 5 Interpret charts and graphs 6 Identify the costs and benefits of specific choices made in trade and globalization (USII.1i) 7 8 Table of Specifications for Pre-Assessment With Assessment Results

64 Teacher P: U.S. History II.8b-e Pre-Assessment Data USII.8bUSII.8cUSII.8dUS118.e TOTAL CORRECT Percent Correct Number of Questions 232310 100% Student 1 000000.0% Student 2 000000.0% Student 3 000000.0% Student 4 0010110.0% Student 5 1000110.0% Student 6 1000110.0% Student 7 0100110.0% Student 8 0100110.0% Student 9 1100220.0% Student 10 1100220.0% Student 11 0110220.0% Student 12 1010220.0% Student 13 0110220.0% Student 14 1001220.0% Student 15 1100220.0% Student 16 1100220.0% Student 17 0111330.0% Student 18 1110330.0% Student 19 1211550.0% Student 20 1212660.0% AVERAGE.55.7.4.2519.9% 64

65 Content Cognitive Level KNCOAPANSYEV Percent of Questions Correct Conversion from a wartime to a peacetime (USII.8b) 1, 28 Describe 3, 4, 13, 38 Make connections between past and present (USII.1b) 33, 34, 35 27.5 (Pre) 82 (Post) +54.5 Role of America’s military and veterans in defending freedom during the Cold War, including the wars in Korea and Vietnam, the Cuban missile crisis, the collapse of communism in Europe, and the rise of new challenges (USII.8c) 2, 27 Identify 5, 6, 9, 36, 39 Analyze and interpret primary and secondary source documents (USII.1a) 10, 11, 12 23.3 (Pre) 84 (Post) +60.7 Changing patterns of society, including expanded educational and economic opportunities for military veterans, women, and minorities (USII.8d) 14, 29 Describe 7, 15, 16, 37, 40 Interpret slogans and documents (USII.1h) 8, 24, 25 20 (Pre) 80 (Post) +60 The impact of international trade and globalization on American life (USII.8e) 17, 18 Describe 19, 20, 26 Interpret charts and graphs 30, 31, 32 Identify the costs and benefits of specific choices made in trade and globalization (USII.1i) 21, 22, 23 8 (Pre) 67 (Post) +59 Using the Table of Specifications to Connect Student Results to the Curriculum 65

66 Teacher P: U.S. History II.8b-e Post-Assessment Data USII.8bUSII.8cUSII.8dUS118.e TOTAL CORRECT Percent Correct Number of Questions910 1140 100% Student 1 86352255.0% Student 2 67682767.5% Student 3 88572870.0% Student 4 87852870.0% Student 5 87762870.0% Student 6 610762870.0% Student 7 781073280.0% Student 8 79762972.5% Student 9 56652255.0% Student 10 89993587.5% Student 11 69983280.0% Student 12 810 73587.5% Student 13 78983280.0% Student 14 89883382.5% Student 15 79762972.5% Student 16 910 3997.5% Student 17 891073485.0% Student 18 89983485.0% Student 19 881083485.0% Student 20 910 1140100.0% AVERAGE 7.458.408.07.2531.0577.6%

67 Teacher P: Pre-Assessment and Post-Assessment Data PRE-ASSESSMENT DATA Percent Correct POST-ASSESSMENT DATA Percent Correct TOTAL 100% Student 1 0.0%55.0% Student 2 0.0%67.5% Student 3 0.0%70.0% Student 4 10.0%70.0% Student 5 10.0%70.0% Student 6 10.0%70.0% Student 7 10.0%80.0% Student 8 10.0%72.5% Student 9 20.0%55.0% Student 10 20.0%87.5% Student 11 20.0%80.0% Student 12 20.0%87.5% Student 13 20.0%80.0% Student 14 20.0%82.5% Student 15 20.0%72.5% Student 16 20.0%97.5% Student 17 30.0%85.0% Student 18 30.0%85.0% Student 19 50.0%85.0% Student 20 60.0%100.0% 67

68 Goal Setting Sample for a Unit Assessment: 40-Question Pre-Assessment 40-Question Post-Assessment 68

69 Content Cognitive Level Know- ledge Compre- hension Appli- cation AnalysisSyn- thesis Evaluation Conversion from a wartime to a peacetime (USII.8b) Describe Make connections between past and present (USII.1b) Role of America’s military and veterans in defending freedom during the Cold War, including the wars in Korea and Vietnam, the Cuban missile crisis, the collapse of communism in Europe, and the rise of new challenges (USII.8c) Identify Analyze and interpret primary and secondary source documents (USII.1a) Changing patterns of society, including expanded educational and economic opportunities for military veterans, women, and minorities (USII.8d) Describe Interpret slogans and documents (USII.1h) The impact of international trade and globalization on American life (USII.8e) Describe Interpret charts and graphs Identify the costs and benefits of specific choices made in trade and globalization (USII.1i) 69

70 Content Cognitive Level Know- ledge Compre- hension Appli- cation AnalysisSyn- thesis Evalu-ation Conversion from a wartime to a peacetime (USII.8b) 1, 28 Describe 3, 4, 13, 38 Make connections between past and present (USII.1b) 33, 34, 35 Role of America’s military and veterans in defending freedom during the Cold War, including the wars in Korea and Vietnam, the Cuban missile crisis, the collapse of communism in Europe, and the rise of new challenges (USII.8c) 2, 27 Identify 5, 6, 9, 36, 39 Analyze and interpret primary and secondary source documents (USII.1a) 10, 11, 12 Changing patterns of society, including expanded educational and economic opportunities for military veterans, women, and minorities (USII.8d) 14, 29 Describe 7, 15, 16, 37, 40 Interpret slogans and documents (USII.1h) 8, 24, 25 The impact of international trade and globalization on American life (USII.8e) 17, 18 Describe 19, 20, 26 Interpret charts and graphs 30, 31, 32 Identify the costs and benefits of specific choices made in trade and globalization (USII.1i) 21, 22, 23 Table of Specifications for Pre-Assessment 70

71 Content Cognitive Level KNCOAPANSYEV Percent of Questions Correct Conversion from a wartime to a peacetime (USII.8b) 1, 28 Describe 3, 4, 13, 38 Make connections between past and present (USII.1b) 33, 34, 35 28 Role of America’s military and veterans in defending freedom during the Cold War, including the wars in Korea and Vietnam, the Cuban missile crisis, the collapse of communism in Europe, and the rise of new challenges (USII.8c) 2, 27 Identify 5, 6, 9, 36, 39 Analyze and interpret primary and secondary source documents (USII.1a) 10, 11, 12 18 Changing patterns of society, including expanded educational and economic opportunities for military veterans, women, and minorities (USII.8d) 14, 29 Describe 7, 15, 16, 37, 40 Interpret slogans and documents (USII.1h) 8, 24, 25 22 The impact of international trade and globalization on American life (USII.8e) 17, 18 Describe 19, 20, 26 Interpret charts and graphs 30, 31, 32 Identify the costs and benefits of specific choices made in trade and globalization (USII.1i) 21, 22, 23 27 Using the Table of Specifications to Connect Student Results to the Curriculum

72 Teacher P: U.S. History II.8b-e Pre-Assessment Data USII.8bUSII.8cUSII.8dUS118.e TOTAL CORRECT Percent Correct Number of Questions 910 1140 100% Student 1 001125.0% Student 2 002137.5% Student 3 1012410.0% Student 4 2012512.5% Student 5 2112615.0% Student 6 2112615.0% Student 7 2013615.0% Student 8 3012615.0% Student 9 2222820.0% Student 10 21251025.0% Student 11 31331025.0% Student 12 42221025.0% Student 13 32231025.0% Student 14 32241127.5% Student 15 23241127.5% Student 16 32331127.5% Student 17 32341230.0% Student 18 41341230.0% Student 19 48462255.0% Student 20 68752665.0% AVERAGE 2.551.82.23.009.5523.9% 72

73 Content Cognitive Level Know- ledge Compre- hension Appli- cation AnalysisSyn- thesis Evalu-ation Conversion from a wartime to a peacetime (USII.8b) 1, 28 Describe 3, 4, 13, 38 Make connections between past and present (USII.1b) 33, 34, 35 Role of America’s military and veterans in defending freedom during the Cold War, including the wars in Korea and Vietnam, the Cuban missile crisis, the collapse of communism in Europe, and the rise of new challenges (USII.8c) 2, 27 Identify 5, 6, 9, 36, 39 Analyze and interpret primary and secondary source documents (USII.1a) 10, 11, 12 Changing patterns of society, including expanded educational and economic opportunities for military veterans, women, and minorities (USII.8d) 14, 29 Describe 7, 15, 16, 37, 40 Interpret slogans and documents (USII.1h) 8, 24, 25 The impact of international trade and globalization on American life (USII.8e) 17, 18 Describe 19, 20, 26 Interpret charts and graphs 30, 31, 32 Identify the costs and benefits of specific choices made in trade and globalization (USII.1i) 21, 22, 23 Table of Specifications for Post-Assessment (Different Questions) 73

74 Content Cognitive Level KNCOAPANSYEV Percent of Questions Correct Conversion from a wartime to a peacetime (USII.8b) 1, 28 Describe 3, 4, 13, 38 Make connections between past and present (USII.1b) 33, 34, 35 28 (Pre) 82 (Post) +54 Role of America’s military and veterans in defending freedom during the Cold War, including the wars in Korea and Vietnam, the Cuban missile crisis, the collapse of communism in Europe, and the rise of new challenges (USII.8c) 2, 27 Identify 5, 6, 9, 36, 39 Analyze and interpret primary and secondary source documents (USII.1a) 10, 11, 12 18 (Pre) 84 (Post) +66 Changing patterns of society, including expanded educational and economic opportunities for military veterans, women, and minorities (USII.8d) 14, 29 Describe 7, 15, 16, 37, 40 Interpret slogans and documents (USII.1h) 8, 24, 25 22 (Pre) 80 (Post) +62 The impact of international trade and globalization on American life (USII.8e) 17, 18 Describe 19, 20, 26 Interpret charts and graphs 30, 31, 32 Identify the costs and benefits of specific choices made in trade and globalization (USII.1i) 21, 22, 23 27 (Pre) 67 (Post) +40 Using the Table of Specifications to Connect Student Results to the Curriculum

75 Teacher P: U.S. History II.8b-e Post-Assessment Data USII.8bUSII.8cUSII.8dUS118.e TOTAL CORRECT Percent Correct Number of Questions910 1140 100% Student 1 86352255.0% Student 2 67682767.5% Student 3 88572870.0% Student 4 87852870.0% Student 5 87762870.0% Student 6 610762870.0% Student 7 781073280.0% Student 8 79762972.5% Student 9 56652255.0% Student 10 89993587.5% Student 11 69983280.0% Student 12 810 73587.5% Student 13 78983280.0% Student 14 89883382.5% Student 15 79762972.5% Student 16 910 3997.5% Student 17 891073485.0% Student 18 89983485.0% Student 19 881083485.0% Student 20 910 1140100.0% AVERAGE 7.458.408.07.2531.0577.6% 75

76 Teacher P: Pre-Assessment and Post-Assessment Data PRE-ASSESSMENT DATA Percent Correct POST-ASSESSMENT DATA Percent Correct TOTAL 100% Student 1 5.0%55.0% Student 2 7.5%67.5% Student 3 10.0%70.0% Student 4 12.5%70.0% Student 5 15.0%70.0% Student 6 15.0%70.0% Student 7 15.0%80.0% Student 8 15.0%72.5% Student 9 20.0%55.0% Student 10 25.0%87.5% Student 11 25.0%80.0% Student 12 25.0%87.5% Student 13 25.0%80.0% Student 14 27.5%82.5% Student 15 27.5%72.5% Student 16 27.5%97.5% Student 17 30.0%85.0% Student 18 30.0%85.0% Student 19 55.0%85.0% Student 20 65.0%100.0% 76

77 Goal Setting Sample for a Cumulative Assessment: 40-Question Cumulative Pre- Assessment 40-Question Cumulative Post- Assessment 77

78 U.S. History II Test Blueprint 78

79 Content Cognitive Level KNCOAPANSYEV Reconstruction to Modern America [7] 1, 2 10, 11, 19 Historical Analysis Skill Questions (U.S. II.1) 20, 21 Turmoil and Change [10] 15 3, 4, 7, 23, 29 Historical Analysis Skill Questions (U.S. II.1) 5, 6, 22 United States through World War II [9] 35, 36 14, 30, 31, 37, 38 Historical Analysis Skill Questions (U.S. II.1) 33, 34 Geography [7] 9 8, 24, 25, 32 Historical Analysis Skill Questions (U.S. II.1) 12, 13 Civics and Economics [7] 26, 27 16, 28, 40 Historical Analysis Skill Questions (U.S. II.1) 17, 18, 39 Table of Specifications for Pre-Assessment 79

80 Content Cognitive Level KNCOAPANSYEV Percent of Questions Correct Reconstruction to Modern America [7] 1, 2 10, 11, 19 Historical Analysis Skill Questions (U.S. II.1) 20, 21 35.0 Turmoil and Change [10] 15 3, 4, 7, 23, 29 Historical Analysis Skill Questions (U.S. II.1) 5, 6, 22 13.5 United States through World War II [9] 35, 36 14, 30, 31, 37, 38 Historical Analysis Skill Questions (U.S. II.1) 33, 34 19.4 Geography [7] 9 8, 24, 25, 32 Historical Analysis Skill Questions (U.S. II.1) 12, 13 40.0 Civics and Economics [7] 26, 27 16, 28, 40 Historical Analysis Skill Questions (U.S. II.1) 17, 18, 39 17.1 Using the Table of Specifications to Connect Student Results to the Curriculum 80

81 Teacher P: U.S. History II Pre-Assessment Data Reconstruction to Modern America Turmoil and Change United States Since WW IIGeography Civics and Economics TOTAL CORRECT Percent Correct Number of Questions 71097740 100% Student 1 0010125.0% Student 2 0021037.5% Student 3 10120410.0% Student 4 20111512.5% Student 5 21120615.0% Student 6 21120615.0% Student 7 20121615.0% Student 8 30120615.0% Student 9 22022820.0% Student 10 212411025.0% Student 11 311321025.0% Student 12 422201025.0% Student 13 302321025.0% Student 14 321411127.5% Student 15 212421127.5% Student 16 322311127.5% Student 17 322411230.0% Student 18 411421230.0% Student 19 454632255.0% Student 20 467542665.0% AVERAGE 2.451.351.752.81.29.5523.9% PERCENTAGE OF QUESTIONS ANSWERED CORRECTLY IN REPORTING CATEGORY 35.0%13.5%19.4%40.0%17.1%23.9%

82 Content Cognitive Level KNCOAPANSYEV Reconstruction to Modern America [7] 1, 2 10, 11, 19 Historical Analysis Skill Questions (U.S. II.1) 20, 21 Turmoil and Change [10] 15 3, 4, 7, 23, 29 Historical Analysis Skill Questions (U.S. II.1) 5, 6, 22 United States through World War II [9] 35, 36 14, 30, 31, 37, 38 Historical Analysis Skill Questions (U.S. II.1) 33, 34 Geography [7] 9 8, 24, 25, 32 Historical Analysis Skill Questions (U.S. II.1) 12, 13 Civics and Economics [7] 26, 27 16, 28, 40 Historical Analysis Skill Questions (U.S. II.1) 17, 18, 39 Table of Specifications for Post-Assessment (Different Questions) 82

83 Content Cognitive Level KNCOAPANSYEV Percent of Questions Correct Reconstruction to Modern America 7 1, 2 10, 11, 19 Historical Analysis Skill Questions (U.S. II.1) 20, 21 35 (Pre) 82.9 (Post) +47.9 Turmoil and Change 10 15 3, 4, 7, 23, 29 Historical Analysis Skill Questions (U.S. II.1) 5, 6, 22 13.5 (Pre) 71.5 (Post) +58 United States through World War II 9 35, 36 14, 30, 31, 37, 38 Historical Analysis Skill Questions (U.S. II.1) 33, 34 19.4 (Pre) 73.3 (Post) +53.9 Geography 7 9 8, 24, 25, 32 Historical Analysis Skill Questions (U.S. II.1) 12, 13 40 (Pre) 85 (Post) +45 Civics and Economics 7 26, 27 16, 28, 40 Historical Analysis Skill Questions (U.S. II.1) 17, 18, 39 17.1 (Pre) 79.3 (Post) +62.2 Using the Table of Specifications to Connect Student Results to the Curriculum

84 Teacher P: U.S. History II Post-Assessment Data Reconstruction to Modern America Turmoil and Change United States Since WWIIGeography Civics and Economics TOTAL CORRECT Percent Correct Number of Questions71097740 100% Student 1 533562255.0% Student 2 646562767.5% Student 3 685452870.0% Student 4 576552870.0% Student 5 574662870.0% Student 6 674652870.0% Student 7 668663280.0% Student 8 576562972.5% Student 9 455442255.0% Student 10 688763587.5% Student 11 477773280.0% Student 12 788753587.5% Student 13 678743280.0% Student 14 686763382.5% Student 15 686542972.5% Student 16 6109773997.5% Student 17 788653485.0% Student 18 688663485.0% Student 19 778753485.0% Student 20 71097740100.0% AVERAGE 5.87.156.65.955.5531.0577.6% PERCENTAGE OF QUESTIONS ANSWERED CORRECTLY IN REPORTING CATEGORY 82.9%71.5%73.3%85.0%79.3%77.6%

85 Teacher P: U.S. History II Pre-Assessment and Post-Assessment Data PRE-ASSESSMENT DATA Percent Correct POST-ASSESSMENT DATA Percent Correct TOTAL 100% Student 1 5.0%55.0% Student 2 7.5%67.5% Student 3 10.0%70.0% Student 4 12.5%70.0% Student 5 15.0%70.0% Student 6 15.0%70.0% Student 7 15.0%80.0% Student 8 15.0%72.5% Student 9 20.0%55.0% Student 10 25.0%87.5% Student 11 25.0%80.0% Student 12 25.0%87.5% Student 13 25.0%80.0% Student 14 27.5%82.5% Student 15 27.5%72.5% Student 16 27.5%97.5% Student 17 30.0%85.0% Student 18 30.0%85.0% Student 19 55.0%85.0% Student 20 65.0%100.0% 85

86 Using a Table of Specifications 1.To create an assessment. 2.To critique and improve an existing assessment. 3.To create a unit assessment plan. 4.To analyze student results. 86

87 Guiding Question 4 How can a teacher and/or principal determine what currently available assessments may be appropriate for establishing baseline data for student achievement goal setting? 87

88 Desired Characteristics of Teacher-Developed Assessments Used to Demonstrate Student Growth Reliable Practical to Employ Instructionally Useful Valid 88

89 Uses of Tables of Specifications To … Create teacher- developed assessments, especially “common assessments” Critique and improve existing assessments, including commercially produced assessments and assessments from test generators Construct Validity Sampling Validity Repeated Trials Reliability 89

90 Choose Assessment Formats Appropriate to the Content and Skills of the Objectives Skills Performance Task Content and Skills Oral Q & A Computer-based Test Performance Task Paper-Pencil Test Computer-based Test Performance Task Paper-Pencil Test Computer-based Test Performance Task Content Oral Q & A Computer-based Test Paper-Pencil Test Computer-based Test Paper-Pencil Test Computer-based Test PrimaryUpper ElementarySecondary Curriculum Considerations Student Developmental Considerations 90

91 Considerations for Selecting a Type of Pre-Assessment Type of Pre-Assessment Sub-Sample or Key Concept Pre-Assessment Cumulative or Comprehensive Pre- Assessment Consideration Student motivation Time away from instruction Instructional appropriateness Construct validity Sampling validity Repeated trials reliability Test-Retest effect 91

92 Virginia’s Uniform Performance Standards for Teachers 92


Download ppt "Teacher Evaluation Institute Assessment Literacy for Student Achievement Goal Setting 1 Division of Teacher Education and Licensure September 2013."

Similar presentations


Ads by Google