1. State Accountability Overviewz
State Accountability Overview
March 15, 2017
Intro Slide
LD W z

2. Resources and Timeline
Objectives
Introduction to the new state accountability system and California School Dashboard
Dashboard
New System
Resources and Timeline
Providing a brief overview of the metrics in the dashboard and how they are calculated
SLIDE 2
LD W
Presenter’s Notes:
Here are our objectives for today’s session (review slide contents)
Link to Strategic Plan
Today is information only, introduction and familiarity, not for stakes until
We don’t expect you to be experts about this after this presentation, keep in mind areas you want to know more about
This is the first of multiple sessions about new accountability
As we present, think about how this new system is similar to, or different from AYP and API- our former system.
Checking for understanding about the new system
Where to find additional resources
The Timeline for roll out

3. Academic Performance Index (State)
SLIDE 3
LD W
Before we dive into the new system, let’s review the old accountability systems we used to have. The Academic Performance Index or API was the state system since Remember these reports? Each August or September, you would get your API report. Your eyes would gravitate to the section on targets – did we meet our target? What was our rank? But most importantly, you would scan the page to the growth number. Did we make 800? And then you would look to see progress for subgroups.

4. Adequate Yearly Progress (Federal)
SLIDE 4
LD W
Adequate Yearly Progress was the measure for federal accountability and this had consequences for being identified as Program Improvement. When these reports were released, you would look to see if you made AYP. Then you would scan for subgroups to see which subgroups didn’t meet targets.

5. Reviewing the former system
At your tables, discuss:
What were the benefits and challenges of our former federal (AYP) and state (API) accountability systems?
Things to consider –
Communication to all stakeholders
Proficiency vs Growth
Comprehensible
Subgroup Representation
Please be prepared to share out.
SLIDE 5
LD W
Presenter’s Notes:
Before we provide an overview of the new accountability system, let’s review some of the benefits and challenges we experienced in the old system i.e., API and AYP.
[group takes 5-7 minutes to review- facilitator charts benefits and challenges]

6. Accountability System
States establish accountability indicators.
States produce California Dashboard to share results for LEAs and Schools (no stakes).
States establish criteria to identify LEAs and schools for intervention and assistance.
State identifies LEAs and (eventually) schools for support and intervention.
State provides technical assistance to identified LEAs.
LEAs support and ensure that identified schools implement required processes and interventions based on identification.
METRICS
LEA/SCHOOL DESIGNATIONS
INTERVENTION AND SUPPORT
SLIDE 6
LD W
Presenter’s Notes:
Federal and State accountability will be under one system now. No Child Left Behind replaced by ESSA. No more program improvement, highly qualified teachers, proficiency targets.
The new state accountability system is the result of legislation that passed at the California State level in 2013.
California adopted the Local Control Funding Formula and districts were thus required to write a Local Control Accountability Plan (LCAP) to address the state’s priorities.
In 2015, at the federal level, the Every Student Succeeds Act replaced No Child Left Behind, and required states to develop a new multiple-measure system of accountability.
This created the opportunity for California to build on the state system it had been working on through LCFF to create one coherent system that also meets federal requirements.
While we have been in “accountability limbo” between the shift from CST to SBA and the shift away from Program Improvement during the CORE waiver, this new system is likely to be the direction for state and federal accountability for the foreseeable future.
In the past, federal accountability (AYP/Program Improvement) and state accountability (API) were not truly one system, so this is new.
This slide shows us the key processes embedded in the new accountability system to show both an overview and a timeline– it is important to note that this system will be rolled out over time. There will be no stakes this school year as the state rolls out a new data dashboard, which we will review a bit later.
Overall, the system uses multiple metrics to measure school performance. Based on the results, LEAs and Schools will be identified as needing additional support, and those LEAs and schools that are identified will be responsible for carrying out certain requirements with additional support.
This system is UNDER DEVELOPMENT, and the State has only recently adopted the first step of building the accountability system: the indicators.
States across the country are adopting accountability measures, and California is releasing school and district level reports in the coming weeks as the first step of implementing the new system. We will spend some time a bit later reviewing the dashboard so you can begin to be familiar with it.
The next phase of the state’s accountability system design is to determine how the indicators will be used and what criteria will determine which districts and schools are identified for intervention. Unlike AYP where proficiency targets went up every year and more and more schools were identified as program improvement, the new federal law will target fewer schools for intervention than we had under Program Improvement. The hope is that districts and states can focus support and resources for those who need it most.
The state is slated to begin this conversation about identification criteria in March 2017, but we may not know the final decisions until September. This coming fall, LEAs (including potentially affiliated charters, which are considered LEAs under LCFF) will be identified for intervention, but schools will not be identified until the beginning of the school year.
The final aspect of the accountability system is that the identified schools and LEAs would implement required processes and interventions with technical assistance. We also expect to learn more about what that will look like when the state finalizes the plan this coming September.
So, BIG PICTURE: The system is rolling out over time, and it may feel a bit bumpy at first.
Criteria: by 9/2017
LEAs Identified:
Schools Identified:

7. SLIDE 7
Data and Accountability

8. Thumbs Up…Thumbs Down for the next slide
A “thumbs-sideways” indicates somewhat familiar with the metric
Thumbs Up…Thumbs Down for the next slide
A “thumbs-up” indicates familiarity with metric
A “thumbs-down” indicates being unfamiliar with the metric

9. 1 5 2 6 3 4 State Indicators (School and LEA)
Chronic absenteeism
starting , data being collected in 5
College & career indicator
for graduates in 4-yr. cohort, Gr. 11 SBA scores, AP & IB tests, A-G completion with “C” or above, dual enrollment, career pathway completion with “C” or above
in-school and out-of-school suspensions 2
Suspension rate
Grade 3-8 scores on standardized tests, ELA and Math 6
Academic indicator 3
English learner progress
Annual progress on CELDT + reclassified EL’s in prior year
Four-year cohort graduation rate, excludes options schools, five-year rate considered for 18-19 4
Graduation rate
SLIDE 8
Data and Accountability
Rearranged the sequence to match the next slide
Reversed order of slides 7 and 8
Presenter’s Notes: These are all the state indicators, which apply at the school and LEA levels. These are calculated by the state.
1) Chronic Absenteeism:
2) Suspension Rate: this will include both in-school and out-of-school suspensions. As a district, we currently track out-of-school suspensions.
3) English Learner progress will be based on the number of English learners that make annual progress on CELDT and the number of EL’s that reclassified in the previous year. This indicator will be adjusted as we roll out ELPAC.
4) Graduation Rate is a 4 yr. cohort graduation rate. The state is considering a 5 yr. grad rate for future years. It’s also worth noting that options schools will not have graduation rates, and an alternative accountability system will be developed for those schools.
5.) College and Career Indicator: This is for high schools, and it is looking at the graduates in the four-year cohort. Of those graduates, the following indicators are included: gr. 11 Smarter Balanced results, AP/IB test results, A-G completion with C or above, dual enrollment, and career pathway completion (which is the completion of a capstone course). If you look at the back of your packet, you’ll see a handout that describes how they determine if a student is “prepared” or “well-prepared”, based on a combination of the various indicators.
6) The Academic Indicator includes grades 3-8 Smarter Balanced results in ELA and Math. We will spend more time on this later in the presentation, but it is no longer only based on % met or exceeded. The academic indicator will eventually include science test results when we are in full implementation. Notice that Grade 11 is not included because it is embedded into the College and Career Indicator.
7) OVERALL: Most of these indicators are familiar and we have been tracking them over the last number of years.
State Indicators (School and LEA)
These are calculated by the state

10. Where does the data come from?
MiSiS
Certify
CALPADS
Data and Accountability
SLIDE 9
Presenter’s Notes:
All state indicators are based on CALPADs, so the accuracy depends MiSIS’s accuracy; school staff play a role in ensuring accuracy.
Our Certify system allows principals and data designees to get alerts so they can correct data errors. Data errors that affect CALPADS reporting are coded as critical, in red.
New Dashboard

11. Data Used in California Dashboard, Spring 2017 Note Data is from previous years’ CALPADS data.
SLIDE 10
Data and Accountability
Presenter’s Notes:
This slide is really for your reference so you know which years of data are in the reports
The Fall 2017 release will be updated with data.

12. 1
TAB ONE Add your own text here
TURN AND TALK
PROCESSING
What are the measures that your school has focused on this year?

13. A Closer Look: Academic Indicator
MAIN IDEA: How far above or below the minimum scale score for standards met are your students on average?
INTENT: Shows needed improvement to bring the average student to meeting standard.
KEY SHIFTS:
Scale Scores
Level 3 Comparison
Distance from Level 3
Every Student 1 2 3 4
SLIDE 11
Data and Accountability
Presenter’s Notes:
A big change in the indicators is how the academic indicator is reported. We are no longer simply considering the % of students who met or exceeded standards, and instead we are looking at each student’s scale score and how far they are from the minimum scale score for “met standards.”
So the main idea is “how far above or below the minimum scale score for standards met are your students on average”?
This is determined based on scale scores for all students and subgroups.
It establishes “level 3” based on the minimum score at each grade level that is needed to meet standard– student scores are compared to level 3
5) Then, it averages the distance from level 3 of all of the students (or students within a subgroup).
Based on scale scores for all students and subgroups
Establishes “level 3” score for each grade level as a comparison point (lowest score needed to meet standard)
Averages the “distance from Level 3” to determine performance
Every student will have a positive, negative, or neutral contribution to the school-wide, LEA-wide, or student group average.

14. SAMPLE: ELA Grade 3 Subgroup A Subgroup B
+193
+148
+118
+68
+58
Level 3 (2432)
+18
-28
SLIDE 12
Data and Accountability
This visual example shows the calculation
The green line shows the level 3 minimum scale score i.e., the minimum scale score for “met standard” at grade 3 in ELA (2432).
Each dot on the graph is a student and the number next to the dot is the “distance from level 3” so you’ll notice some are above the line and some are below.
Scores above the line are positive, scores below the line are negative.
Then, you average those numbers to produce an overall score, or an average distance from level 3. This can be calculated for subgroups as well. [click for animated] a subgroup’s score.
For ELA 3rd Grade, Level 3 equals a scale score of 2432
Each student’s distance from Level 3 is averaged
Overall (+27.75)
Subgroup A (+54.75)

15. A B C D DISTANCE FROM LEVEL 3 Across California Math Grade 8 SLIDE 13
Score Range
All scale score ranges are set by the Smarter Balanced Assessment Consortium and won’t change B
Across California
All students in the same grade across California are measured against the same scale score C
ELA Grade 6
All grade 6 students across California are measured against the 2531 scale score for ELA. . D
Math Grade 8
All grade 8 students across California are measured against the 2586 scale score for Math.
SLIDE 13
Data and Accountability
DISTANCE FROM LEVEL 3

16. A Closer Look: EL Progress Indicator
MAIN IDEA: Recognize English Learners that made progress on CELDT, and students who reclassified the prior year.
DENOMINATOR:
Total number of annual CELDT test takers (with two years of scores)
ELs who were reclassified in the prior year (Note: If a student moves after being reclassified, the student is included in the denominator of the school that reclassified the student.)
Annual CELDT test takers who:
Increased at least one CELDT level compared to the prior year
NUMERATOR
ELs who were reclassified in the prior year.
Maintained Early Advanced/Advanced English Proficient.
SLIDE 14
Data and Accountability
The EL Progress indicator looks at the progress that EL’s made in CELDT, similar to the AMAO 1 metric that measured how many students increased one CELDT level from the prior year. It also looks at the number of students that reclassified in the previous year. It is different from just adding the AMAO 1 and reclassification rate.
The numerator is the number of annual CELDT test takers that increased one CELDT level compared to the previous year or maintained Early Advanced or Advanced proficient from the previous year, plus the number of students that were reclassified in the prior school year.
The denominator is the total number of CELDT test takers (with two years of scores) plus the number of students that reclassified in the previous year.

17. Metrics for College and Career Indicator
SLIDE 15
Data and Accountability
This handout shows the data that comprises the College and Career Readiness Indicator
It is looking at the graduates in 4-yr. cohort and bases college/career preparedness using: Gr. 11 SBA scores, AP & IB tests, A-G completion with “C” or above, dual enrollment, career pathway completion with “C” or above

18. How do I (as a principal) get my school’s EL information?
THINK INK PAIR SHARE
How do I (as a principal) get my school’s EL information?
What tools do I know of currently?
PROCESSING
LD West

19. SLIDE 16
LD W – Beginning of status and change discussion – what do performance categories mean?

20. Performance Levels for State Indicators
Performance Levels are calculated using percentiles that combine Status and Change using a five-by-five colored table that produced 25 results represented by five colors.
Blue
Highest
Green
Yellow
Orange
Red
Lowest
SLIDE 117
Presenter’s Notes:
Up to this point we have been talking about all the indicators or metrics that are included in the system. Now we will talk about how the indicators will be displayed on the new dashboard.
When it comes to reports and results, all indicators will be regarded in terms of both STATUS (the result for that year) and CHANGE (the result in comparison to previous year(s))
Based on the status and change, schools will be given a performance level that ranges from Blue (high) to Red (low)
Each performance level was derived based on a combination of status and change.
You’ll notice that each level has wedges and might be wondering why it’s presented that way. This is so that a color blind person could read it, or a black and white copy could still communicate the results.
Let’s look at an example.

21. Five Status and Five Change Levels
Five Status Levels
Five Change Levels
Very High
Increased Significantly
High
Increased
Medium
Maintained
Low
Declined
Very Low
Declined Significantly
Blue
Green
Yellow
Orange
Red
SLIDE 18
LD W

22. Status and change sample
(3 yr. average)
2015 Grad Rate: 89%
3 yr. average rate: 91%
Difference between 2015 and 3 yr. average: -2
N/A
SLIDE 19
LD W
Presenter’s Notes: 1) This 5 by 5 matrix is how the performance level is determined.
2) For this example, the graduation rate is 89%. If you look at the status column, you will see that the school is considered “Median” given that 89% falls between 85%-90%. (click for animation)
3) The 3 yr. average graduation rate was 91%, which is 2 percentage points higher than the 2015 result. That means that the school is considered “Declined” (click for animation)
4) What is this school’s graduation performance level or color? Orange: median and declined. (click for animation)
5) This is a big difference from the past accountability systems because it takes more than one year into account.
6) The technical manual and forthcoming tables on the state website will provide cut off points for each of the performance levels. Cut off points vary for each indicator.
Gray = NA

23. SLIDE 20
LD W

24. REPORTED SUBGROUPS Race/ethnicity Socioeconomically disadvantaged.
Students with disabilities
Students with disabilities
Homeless and Foster will be added Fall 2017
English learners – definition varies by indicator
English learners – definition varies by indicator
SLIDE 21
Data and Accountability
Presenter’s Notes:
Each indicator will be reported on the state dashboard overall (all students) and by subgroup.
This slide shows the reported subgroups, all of which were reported under No Child Left Behind.
The dashboard will report any subgroup with more than 11 students, but there will be accountability for a subgroup for 30 or more students. This is different from API and AYP, when n-size was 50 students.
A potential area of confusion is the English Learners subgroup. The students that are included in this subgroup will vary depending on the indicator.
Homeless and Foster will be added Fall 2017
African-American, Asian, Filipino, Hispanic/Latino, Native American, Pacific Islander, Two or More Races, White

25. Criteria for English Learners in the New System
The Current EL annual CELDT* test takers (grades K-12) plus students reclassified in the prior year
English Learner Progress
ELs (grades 3-8) plus students who have been Reclassified fluent English proficient (RFEP) for four years or less** (Note: This is similar to the criteria used in the prior state and federal accountability systems)
Academic
Students with an EL status at any time in grades 9-12 (same criteria since the initial release of the cohort graduation rate)
Graduation
The Students with an EL status at any time in grades 9-12
College and Career Indicator
Current EL students (grades K-12)
SLIDE 22
Data and Accountability
Presenter’s Notes:
Depending on the state indicator, the students included in the EL subgroup is different.
[Review the contents of the slide, one indicator at a time]
Suspension (Chronic Absenteeism will be added when data is available)
*CELDT: California English Language Development Test
** This definition is based on what is permitted in the Every Student Succeeds Act

26. Suspension Report by Subgroup
SLIDE 23
Data and Accountability
Presenter’s Notes:
lastly, if you click on one of the indicators, you will get further details on the performance levels by subgroup. So in this example, while our all student group is blue (very low and declined) our students with disabilities are considered green (low and declined). You can see variation by subgroup to highlight disparities.
You will also be able to click on a subgroup to see the performance levels across indicators

27. Comprehensive Support and Intervention (overall low achievement)1
Comprehensive Support
Targeted Support and Intervention (achievement gaps) 2
Targeted Support
SLIDE 24
LD W
Presenter’s notes: [review contents of the slide]
Beginning in , based on the dashboard results, some schools will be identified for intervention in two categories:
Comprehensive Support and Intervention (overall low achievement)
Targeted Support and Intervention (achievement gaps)
Exactly how the school will be identified, what interventions will be required, and how schools will demonstrate improvement is TO BE DETERMINED .
School accountability and identification
Beginning in , based on the dashboard results, some schools will be identified for intervention in two categories:
Exactly how the school will be identified, what interventions will be required, and how schools will demonstrate improvement is TO BE DETERMINED .

28. Moving Forward
With an ear partner, share your understanding of Performance Levels for State Indicators
PROCESSING SLIDE

29. Inspect the Dashboard Report
Activity:
Inspect the Dashboard Report
Materials: Each participant should have a copy of a sample California Dashboard report
Step 1: Work silently on your own.
What questions do you have or do you anticipate your schools will have?
Step 2: Partner work
Seek out a partner in the room and discuss your questions.
What remaining questions do you have after your discussion?
Slide 29- Use Handout
LD W – if we have time
5 min.
10 min.

30. RESOURCES 1 2 3 4
The A link to the Technical Guide is:
The State is developed engagement toolkits:
Send feedback about the dashboard to
Reference Guide
Slide 30
LD W
Presenter’s Notes:
[Review slide contents]