1. Accountability, Testing, and Common Exams
Tammy Howard
Jim Kroening
Jennifer Preston
October 25, 2012

2. Overview: 2012-13 Assessments EOGs and EOCs
English Language Arts (3-8, English II)
Mathematics (3-8, Algebra I/Integrated I)
Science (5,8, Biology)
Delayed Scoring (October 2013)
Accountability
Ready Accountability Model
General Assembly’s School Performance Grades
Delayed Reporting (October 2013)
Common Exams
Administration
Addressing Concerns 2

3. READY Accountability Model3

4. Accountability Purpose – Indicators – Key Ideas▲
Provide incentives and supports to
Improve Student Learning Outcomes
Raise Graduation Rates
Close Achievement Gaps 4

5. EVAAS Growth School-wide
Elementary and Middle School Accountability Purpose – Indicators – Key Ideas ▲
Status Indicators
“this year”
Progress Indicators “trend over time for groups of students”
Growth Indicators “combined individual student’s growth”
End of Grade 3-8 Math
Δ End of Grade 3-8 Math
EOG 3-8 Math
EVAAS Growth School-wide
Three column
End of Grade 3-8 ELA
Δ End of Grade 3-8 ELA
End of Grade 5 & 8 Science
Δ End of Grade 5 & 8 Science 5

6. High School School Accountability Purpose – Indicators – Key Ideas▲
Status Indicators
“this year”
Progress Indicators “trend over time for groups of students”
Growth Indicators “combined individual student’s growth”
End of Course
Δ End of Course
Δ End of Grade 3-8 Math
EOG 3-8 Math
ACT
Δ ACT
Three column
EVAAS Growth School-wide
Graduation Rates
Δ Graduation Rates
Math Course Rigor
Δ Math Course Rigor
WorkKeys
Δ WorkKeys
Graduation Project 6

7. How each indicator is defined
Performance Composite (Elementary and High)
Percent of proficient tests in a school
All tests, subjects, and grade levels
Uses the EOG/EOC test data
Algebra II/Integrated III
Percent of graduates who take and pass Alg. II or Int. Math III
Excludes the 1% population
Graduation Rate
Percent of students that graduate within 4 years and within 5 years (4-year cohort graduation rate)
WorkKeys
Percent of seniors who are CTE concentrators who achieve a Silver certificate, or better, on the WorkKeys assessment
ACT
Percent of students who meet college readiness targets
Graduation Project
Did a school implement a graduation project ? 7

8. School Performance Grades8

9. Responding to School Performance Grades (SPG)
The SBE must respond to the General Assembly “…annually by January 15 on recommended adjustments to the school performance grade elements and scales for award of scores and grades.”
Additionally, SECTION 7A.3.(f) indicates: “It is the intent of the General Assembly to add a student growth component to school performance grades.”
Operational in 9

10. + Growth What are the basics of the SPGs? Elementary/Middle Schools
Total Points 0-100
100 points
Performance Composite
Total Points 0-500
High Schools
100 points
Performance Composite
Algebra II/Integrated III
Graduation Rate
WorkKeys
ACT
+ Growth 10

11. How each indicator is defined
Performance Composite (Elementary and High)
Percent of proficient tests in a school
All tests, subjects, and grade levels
Use the EOG/EOC test data
Algebra II/Integrated III
Percent of 4-year cohort graduates who take and pass Alg. II or Int. Math III
Excludes the 1% population
Graduation Rate
Percent of students that graduate within 4 years (4-year cohort graduation rate)
WorkKeys
Percent of seniors who are CTE concentrators who achieve a Silver certificate, or better, on the WorkKeys assessment
ACT
Percent of Grade 11 students who meet a college readiness standard 11

12. Alignment Between Indicators in High School
High School Performance Grades
Performance Composite
Algebra II/Integrated III
Graduation Rate
WorkKeys
ACT
End of Course
Math Course Rigor
Graduation Rates
WorkKeys
Key Point: The set of indicators are shared and set a college and career- ready expectation. The Graduation Project is not part of the school grade.
ACT
Graduation Project 12

13. A Closer Look at the Next Generation of State Assessments13

14. WorkKeys for Early Graduates in Dec
School Calendar - Draft
2012
2013
Aug
Sep
Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
June
July
Sept
New State Tests (3-8 ELA and Math; 5 & 8 Science; 3 High School Exams)
ACT, PLAN, EXPLORE, WORKKEYS
SMARTER Balanced (Development Work)
New Assessments
Last ABCs Report
READY Report to SBE
Compilation of New Data
Standard Setting
PLAN
ACT March 5, 2013 March 19, 2013 (make-up) March 5-19 (accommodations testing)
WorkKeys
EXPLORE
WorkKeys for Early Graduates in Dec
Teacher teams write field test items
Pilot test of summative and interim assessment items and performance tasks
2nd Tech Readiness Tool Collection Windows 14

15. A Change in the Claim of Testing
Claims in the Past: Proficiency
Claims in the Future: Career- and College- Ready 15

16. Changing what we think of as “State Assessments”
Constructed Response
Performance Tasks
Computer Adaptive Testing
This is what we’ve known +
Summative
Interim Tools
“Along the Way”
Formative Processes
“Every Day” 16

17. How do I get READY for the new summative assessments?17

18. Step 1: Dig into the Standards. Getting Ready
Where do I go to do that?
For Common Core
For Essential Standards 18

19. Step 2: Getting Ready Internalize Content Specifications
Where do I go to do that?
Content Specifications 19

20. Step 3: Getting Ready Review New Item Types, Items and Released Forms
Where do I go to do that?
Item Types under Summer Institute 2012
Released Forms Coming in Fall of 2012
Tutorial of New Assessments 20

21. Getting Ready
Step 3:
New Item Types and Items Constructed-Response and Technology-Enhanced Items
Mathematics: gridded response items
Grades 5-8 and Math I (Algebra I/Integrated I)
Calculator Inactive: Grades 3-8 and Math I (Algebra I/Integrated I)
One-third to one-half of grades 3-8
One-third of Math I (Algebra I/Integrated I)
English II: short constructed response, technology-enhanced and multiple choice 21

22. Grade 5 Science TE Item 22 Distance Time Graph
5.P.1.2 Infer the motion of objects in terms of how far they travel in a certain amount of time and the directions in which they travel (RBT = Conceptual/Understand; DOK = 2)
This distance/time graph shows the distance covered by an insect crawling across a table. Using the information on the graph, place (click and drag) the remaining distances to complete the table below.
A more traditional multiple choice item would have most likely been presented as “An ant travels 30 cm in 15 seconds. How far would the ant travel in 25 seconds?” This TE item not only assesses whether a student knows the equation for calculating speed, but also the level of conceptual understanding a student has regarding the correlation between distance and time in determining the speed an object travels. A correct response would involve a student placing the 40 cm in the center box and 50 cm in the box on the right.
Distance traveled in 10 seconds
Distance traveled in 20 seconds
Distance traveled in 25 seconds
20 cm
30 cm
40 cm
50 cm 22

23. Species Likely to Go Extinct
Grade 8 Science TE Item
List of Species and Their Characteristics
Estuarine species inhabit coastal areas; they are adapted to a mix of salt and fresh water
Desert species are adapted to hot, arid areas which have large temperature fluctuations
Very rare tree species inhabit the Amazon basin
Reef-building coral inhabit coastal areas; their health and growth are negatively affected by
pollutants
Salamander species in the southern U.S. depend on flowing water to keep oxygen levels high
enough for survival
If the changing conditions in the chart below occur, which species listed above will likely go extinct? Drag and drop the species that will most likely go extinct for each changing condition.
Changing Conditions
Species Likely to Go Extinct
Sea levels rise rapidly
Estuarine
Prolonged drought
Deforestation in South America
Catastrophic oil spill in a coastal region
8.L.4.1 Summarize the use of evidence drawn from geology, fossils, and comparative anatomy to form the basis for biological classification systems and the theory of evolution (RBT = Conceptual/ Understand; DOK = 2)
This item, dealing with a similar concept in species adaptation, requires a higher level of cognitive demand where the student has to evaluate the characteristics of the species indicated at the top of the page and then determine which species will be most affected by the changing conditions. In this item we have combined potentially, 3 difference multiple choice items, into 1 single item. The student has to select 3 out of 5 possible response options. A correct response would be Box 2 – salamander, box 3 = amazon rare tree, and box 4 = reef-building coral
Amazon rare tree
Desert
Salamander
Reef-building coral 23

24. Grade 6 Constructed Response Item
A company is having a picnic. The expenses will be for music and refreshments.
The music will cost $150.
The refreshments will cost $125. 
Tickets will be sold for $2.50 per employee.
What is the minimum number of tickets that must be sold to pay for the picnic expenses?
Note that students receive instructions and practice for gridding answers.
Calculator: Inactive
DOK: Skill/Concept
Domain: Expressions and Equations
Standard: 6.EE.7 Solve real-world and mathematical problems by writing and solving equations of the form x + p = q and px = q for cases in which p, q and x are all nonnegative rational numbers. 24

25. Math I Constructed Response Item
The daily profit, P, of a business that sells x units of a product each day is given by the function P = –2x x + 3,000. The number of units sold on Tuesday was 10 less than the number needed for maximum possible profit. What is the difference between the actual profit on Tuesday and the maximum possible profit?
Enter your response here:
Only 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ., -, and / are allowed.
Calculator: Inactive
DOK: Skill/Concept
Domain: Interpreting Functions
Standard: F.IF.4 For a function that models a relationship between two quantities, interpret key features of graphs and tables in terms of the quantities, and sketch graphs showing key features given a verbal description of the relationship. Key features include: intercepts; intervals where the function is increasing, decreasing, positive, or negative; relative maximums and minimums; symmetries; end behavior; and periodicity. 25

26. English II Multiple-Choice Item
Moonrise
by Jenette Purcell
City night sky gives itself to me again when I have so little left to receive it. I am dark, crumbling and you are rivers and trees away searching your own night sky for a sign. The strong gates of your heart are wide open to me always, but, if only. So I wait, as seasons before, decades before, fathers and mothers before me still inside watch and listen. Suddenly, bamboo, bones, fiber, fences, water, glistening koi,* all the tiny rooms, paths and places I hold your memories relax in audible, reverent wonder at the fullness forming on this horizon’s edge. *koi: colorful fish that symbolize love and friendship
Which line from the poem describes the speaker’s feelings about loving someone?
“when I have so little left to
receive it”
“are wide open to me always, but”
“paths and places I hold your
memories
“at the fullness forming”
DOK: Skill/Concept
Standard: Reading for Literature (RL.1)
Key Ideas and Details: Cite strong and thorough textual evidence to support analysis of what the text says explicitly as well as inferences drawn from the text. 26

27. English II Constructed-Response Item
Moonrise
by Jenette Purcell
City night sky gives itself to me again when I have so little left to receive it. I am dark, crumbling and you are rivers and trees away searching your own night sky for a sign. The strong gates of your heart are wide open to me always, but, if only. So I wait, as seasons before, decades before, fathers and mothers before me still inside watch and listen. Suddenly, bamboo, bones, fiber, fences, water, glistening koi,* all the tiny rooms, paths and places I hold your memories relax in audible, reverent wonder at the fullness forming on this horizon’s edge. *koi: colorful fish that symbolize love and friendship
In Moonrise, explain how the theme is developed throughout the poem. Use specific details to support your answer.
DOK: Strategic Thinking
Standard: Reading for Literature (RL.2)
Key Ideas and Details: Determine a theme or central idea of a text and analyze in detail its development over the course of the text, including how it emerges and is shaped and refined by specific details; provide an objective summary of the text. 27

28. Scoring Rubric for Constructed- Response Item
In Moonrise, explain how the theme is developed throughout the poem. Use specific details to support your answer.
Score (points)
Rubric 2
Identifies the theme of the poem
Uses at least one example of how the theme is revealed
in the poem
Writes a response that explains how the theme is
developed throughout the poem 1
May or may not use at least one example of how the
theme is revealed in the poem
Writes a response that may or may not explain how the
theme is developed throughout the entire poem
Fails to identify the theme of the poem
Fails to use at least one example of how the theme is
developed in the poem
Fails to write a response that explain show the
theme is developed throughout the poem 28

29. English II Technology-Enhanced Item
Select (by clicking) the synonym that can replace reverent in the poem.
Excerpt from Moonrise
by Jenette Purcell
Suddenly, bamboo, bones, fiber, fences, water, glistening koi,* all the tiny rooms, paths and places I hold your memories relax in audible, reverent wonder at the fullness forming on this horizon’s edge.
DOK: Skill/Concept
Standard: Reading for Literature (RL.4)
Craft and Structure: Determine the meaning of words and phrases as they are used in the text, including figurative and connotative meanings; analyze the cumulative impact of specific word choices on meaning and tone (e.g., how the language evokes a sense of time and place; how it sets a formal or informal tone).
respectful
redundant
amazed
significant 29

30. Common Exams
A Library of Common Exams is being designed for non-tested subjects for district use to populate Standard 6 30

31. So why have statewide Measures of Student Learning/Common Exams?
Focusing on the “Why”
So why have statewide Measures of Student Learning/Common Exams?
North Carolina has a statewide evaluation system to ensure that every teacher receives a fair and consistent evaluation, regardless of his or her employing LEA
Teachers in all content areas should receive a Standard Six rating based on the growth of their own students on their content-specific standards
Most LEAs do not have the capacity to design their own assessments for all non state-tested grades and subjects 31

32. Principles for Administration
1. Every English Language Arts, Science, Mathematics, and Social Studies teacher in grades 4 – 12 has a value-added score
2. Teacher growth values will be calculated based on all students a teacher teaches and, when multiple assessments are required, on all data generated through the assessments 32

33. Decision Tree for Administration33

34. Resources for Districts
Implementation Guide
Administration Timelines
Assessment Specifications
Guide to Measuring Student Growth
Local Planning Template 34

35. Implementation Options - Timing
Administration of the high school MSLs in the fall is optional
If a district chooses to administer, all MSLs must be administered (1st semester only - Geometry & Algebra II optional)
If a district chooses to administer, results will be used to determine the sixth standard rating 35

36. District Flexibility Administration online, paper/pencil or hybrid
Date of administration
Administration during class period or testing week
Use in student grade
Which assessments are administered
How to ensure secure administration 36

37. Addressing Concerns
Who has designed the Common Exams, and how have they been designed?
Same basic process as state assessments with the creation of assessment blueprints, generation of items, review of items, review of forms, and final production
Over 800 teachers from across the State have been involved in the blueprint creation and form review processes
NCDPI psychometricians and test measurement specialists have been involved and will analyze (and remove from results) any poor-performing items before growth is calculated 37

38. Addressing Concerns Why doesn’t anyone know what will be on the MSLs?
Assessment specifications are available at: cifications/
General information on rubrics released to C&I leaders on October 19 (and posted to website)
Online module will provide training on how to use rubrics to score performance tasks
Each item has its own specific rubric 38

39. Addressing Concerns How will the performance items be scored?
There must be at least one scorer who is not the student’s teacher of record AND who has the content knowledge necessary to score the item
With the exception of ELA, performance items can be administered early to allow time for scoring
Scoring of work is necessary for what the 800 teachers deemed to be authentic assessment for new, concept-based standards 39

40. Addressing Concerns MSLs hurt students and teachers
The Common Exam administration process should not affect students any differently than the administration of a teacher-created final exam
MSL scores do not need to be used as final exam grades
Percent correct provided by Winscan is a suggestion for a grade
It is only fair to base SOME part of a teacher’s evaluation on the growth of his or her students 40

41. Contact Information 41