1. MNPS Academic Performance Framework 2011 - 2013
Paul Changas, Alan Coverstone and Christine Stenson
Metropolitan Nashville Public Schools (MNPS)
October 29, 2013

2. The MNPS Framework Presentation Overview
Why the need for a Framework and how it will be used?
Our process in building a Framework
Components of our Framework – indicators and measures
Next steps
Lessons learned that might be helpful to other districts
A framework template for other districts/schools

3. The MNPS Framework Possible Framework Uses
To support efforts to raise student achievement
To support the district’s accountability status
To offer standardized accountability metrics to complement increased school-level autonomy
To inform — but not determine — decisions regarding rewards, supports, and resource allocation for schools
To provide school communities with a transparent set of indicators to understand school performance

4. Why the Need for a Framework
Why the Need for a Framework? (Why Not Rely on State Accountability Results?)
State absolute accountability system is primarily focused on districts rather than schools
Key school relative accountability results reported every three years rather than annually
State accountability is determined entirely by test scores – there are no school culture measures
There is value in an overall performance index
While a single number cannot tell the whole story, a “bottom line” based upon multiple measures is needed for making decisions
Transparency – outline exactly what measures will be included and how they will be weighted
Manageability – quantity of data can be overwhelming
3 M’s of data use: make it Manageable
make it Meaningful
make it Matter

5. State Accountability Flowchart

6. The MNPS Framework Indicators and Weighting

7. What Factors Should be Included in School Evaluation
What Factors Should be Included in School Evaluation? Audience Participation
Participants pair up
Identify 3-5 key factors or indicators that you would include in a school performance system (academic, non- academic, school culture, etc.)
How would you prioritize these factors (or should they be equally weighted)?
Discuss for 5 minutes and then we will report out

8. The MNPS Framework K-8 Measures and Weighting

9. The MNPS Framework High School Measures and Weighting

10. The MNPS Framework The Academic Performance Scale
Identify key performance measures
Determine the weight of each measure
Determine the performance scale of each measure
Assign performance points to each school based upon position on performance scale

11. The MNPS Framework The Academic Performance Scale
Determine the performance scale of each measure

12. The MNPS Framework School Ratings for Growth
Two measures (50% of Total Framework): Value Added and Mean Achievement Level Increase
K-8 subjects: Math, Reading/LA, & Science
HS subjects: Algebra I and II, English I, II, II, & Biology I
Not highly correlated to socioeconomic status

13. The MNPS Framework Mean Achievement Level Increase
School A
School B

14. The MNPS Framework Mean Achievement Level Increase Goal Example
Establish Proficiency Increase Goals Based Upon Prior-Year Results

15. The MNPS Framework K-8 Ratings for Achievement & College Readiness
Two measures (30% of Total): Percent Proficient/Advanced and percent of 4th/8th grade students projected to score 21 or higher on the ACT
Percent Proficient/Advanced includes Math (or Algebra I), Reading/LA, & Science
These measures are correlated with socioeconomic status (SES), but schools often break the pattern

16. The MNPS Framework High School Ratings for Achievement & College Readiness
Percent Proficient/Advanced includes Algebra I and II, English I, II, and III, and Biology I
ACT Composite score of 21 or higher is required for the Hope Scholarship and is the average of the subject area college readiness benchmarks.
These measures are correlated with Socioeconomic status, but schools often break the pattern.

17. The MNPS Framework Achievement Gap and Survey Data
Gap Closure (5% of framework)
K-8 subjects: Math, Reading/LA, & Science
HS subjects: Algebra I and II, English I, II, II, & Biology I
TELL and TRIPOD surveys each count as 5% of framework.
Parent survey to be added

18. The MNPS Framework School Rankings Across Measures

19. The MNPS Framework Report Format

20. Charter School Accountability MNPS Commitments
Set and hold charter schools accountable to clear, measurable, and attainable academic, financial, and operational performance standards and targets;
Close schools that fail to meet performance standards and targets; and
Work proactively to identify and establish new, high quality charter schools to serve students who attend schools identified for closure.

21. Outcome-based Performance Management
Performance Contracts
(Predictable, enforceable)
Accountability-based Interventions
(Transparent, balanced, comprehensive)
Consistent Communication
(Face validity, engagement)
District-Charter Collaboration Compact
Focus Groups Scorecard
Performance Management, Replication, and Closure (PMRC) Grant
Performance Frameworks Policies and Contracts
MNPS Research and Assessment
Technical Development Balanced Measures Broad Applicability

22. Outcome-based Performance Management How will we use the APF?
Publish Annual School Report Cards
Shape Renewal Process
Shape Recommendations each October

23. Overall Performance (APF)
Academic Performance
Mean Achievement Increase
TVAAS
TCAP (%PA)
Achievement Gap
School Culture Measures
ACT (21+)
Overall Performance (APF)
Year by year

24. Renewal Information Projection and Review Level
Renewal Application Deadline
5-year Review Year

25. Outcome-based Performance Management How will we use the APF?
Publish Annual School Report Cards
Shape Renewal Process
Shape Recommendations each October

26. 3-year status Action Timeline Renewal Review LevelNA
Simple Renewal
(Updated budget, plans, targets)
Watch
2-years
Full Renewal Review
(Full Renewal Application: Renewal Based on Best Interests Standard)
1-year
Conditional Renewal Review
(Renewal possible but unlikely; significant changes required; may recommend revocation depending on end of year results)
Current Cycle
No Renewal
(May recommend revocation to take affect end of year)

27. Outcome-based Performance Management How will we use the APF?
Publish Annual School Report Cards
Shape Renewal Process
Shape Recommendations each October

28. School 2013 Status 3-Year Status Recommended Action
KIPP Academy
Excelling
Simple Renewal Review
Lead Academy MS
Satisfactory
None
LEAD Academy HS
New Vision Academy
Liberty Collegiate Acad
2-years
STEM Prep Academy
Drexel Prep School
Target
Notice: Revocation in 2014 Likely
Nashville Prep School
East End Prep
Insuffic Data
Cameron College Prep
Review
Knowledge Academy
Achieving
1-year
Boy's Prep
Notice: Revocation in 2014 Possible
Smithson-Craigh Acad
Smithson-Craigh MS
Closed May 2013

29. What Decisions Would You Be Comfortable With? Audience Participation
Participants pair up
What types of decisions would you be comfortable making with a performance framework for your school(s)?
How many years of data would you need to make these decisions?
What additional information would you want in making decisions about school performance?
Discuss for 5 minutes and then we will report out

30. The MNPS Framework Next Steps
Show expanded data by individual year
Break out mean achievement level increase, TVAAS, and gap calculations by subject
Break out key results by subgroup
Provide professional development on the Framework
Expand documentation
Utilize results in evaluation of initiatives and in identifying schools needing support and resources

31. The MNPS Framework Lessons Learned
Include stakeholders in development
Transparency is critical
Flexibility is essential as standards, assessments and policies change, but the basic principles we value and include in the Framework should hold up over time
Construct the Framework in such a way that the top performance category is within reach of any school, regardless of socioeconomic factors
Performance measures can vary significantly from year to year, so multiple years of data should be utilized in critical decisions
Presentation format is important

32. The MNPS Framework Performance Framework Template

33. The MNPS Framework Decision-making Process for Each Indicator
Each Category of Data Requires the Following Decisions:
Which Measure
Subject Areas
Years of Data
Unit (e.g. School)
Performance Thresholds
Weighting/ Points
For Example: Value Added
• Should Math, Reading/LA, Science be included?
• Should Math, Reading/LA, Science be weighted equally and be combined into a single score?
• Should there be discrete score categories (e.g. 1-5), or a linear transformation of the scores such that schools earn from 0 to 100 percent of possible points, or should we use a method that takes into account the fact that most schools’ scores are going to be clustered around the average?
• Should the categories be discrete or continuous, do we base them on average growth from 2012 to 2013, or take other years of growth into consideration? If we use two years of data, we get a bigger range of possible scores.
• Do we set our evaluation based on what has been average in the school district and the state or based on the growth standard (i.e. zero growth is average)? If we use the growth standard, we may fall behind the state. If we use actual average NCE gains, we will be rating schools on a very different basis than the state does in its report card and on the TVAAS site (for grades 3-8 only).

34. The MNPS Framework
Questions?