1. How We Did The Story “Grading the Teachers” Molly Bloom / StateImpact Ohio * Patrick O’Donnell / The Cleveland Plain Dealer @m_bloom@m_bloom * mollypbloom@gmail.com * PaODonnell@plaind.commollypbloom@gmail.comPaODonnell@plaind.com

2. The Series Read ●Overview: Using Data To Evaluate TeachersOverview: Using Data To Evaluate Teachers ●Pay vs. Value-Added PerformancePay vs. Value-Added Performance ●Secrets Of Two “Most Effective” TeachersSecrets Of Two “Most Effective” Teachers ●Value-Added’s Poverty FactorValue-Added’s Poverty Factor ●How is Value-Added Calculated?How is Value-Added Calculated? Multimedia ●Audio: Measuring Performance Through GrowthAudio: Measuring Performance Through Growth ●Audio: Push for Performance PayAudio: Push for Performance Pay ●Video: Guide to Ohio’s New Way of Evaluating TeachersVideo: Guide to Ohio’s New Way of Evaluating Teachers Data ●Search Ratings For 4,200 Ohio TeachersSearch Ratings For 4,200 Ohio Teachers ●Map: See Teacher Ratings at Your SchoolMap: See Teacher Ratings at Your School ●Why We Published Teachers’ RatingsWhy We Published Teachers’ Ratings ●About Our AnalysisAbout Our Analysis

6. Grading the Teachers: Where the “Most Effective” Teachers Are (MAP)

7. Search Teacher Ratings http://www.cleveland.com/datacentral/index.ssf/ 2013/06/find_ohio_teacher_ratings_base.htm l http://stateimpact.npr.org/ohio/2013/06/16/gradi ng-the-teachers-search-value-added-grades- for-4200-ohio-teachers/

8. What We Found ●Some teachers find new evaluations useful, some don’t ●Little relationship between value-added scores and teacher pay ●Clear relationship between value-added scores and teachers’ ages ●Clear relationship between the % of poor kids in a school and teachers’ value-added scores

9. The department’s top research official, Matt Cohen, acknowledged that he can’t explain the details of exactly how Ohio’s value-added model works. He said that’s not a problem. “It’s not important for me to be able to be the expert,” he said. “I rely on the expertise of people who have been involved in the field.” 

10. How It Began We had some questions about value-added… 1) Is it really independent of student family income? 2) Does it show teachers getting better over time? In keeping with their pay increases? 3) Does it show any difference between teacher prep programs? 4) Does a masters degree improve teacher performance, as shown by value-added?

12. State’s contention: Performance may be related to income…

13. But value-added isn’t.

14. Teachers who receive top value- added rating are more likely to be in schools with fewer poor students

16. How We Did It ●Talking with teachers ●Interviewing experts ●Working with data

17. Teachers Meet Maria Plecnik

18. Experts Seriously: Value-Added is complicated

19. Data ●Know what you want to ask the data ●Get outside advice ●Bulletproof it ●Few people care about the details

20. This was our post explaining the analysis. Almost no one read it. StateImpact Ohio and The Plain Dealer analyzed the relationships between a statistical measure called value-added and teachers’ pay, and between value-added and teachers’ ages using data obtained from the Ohio Department of Education. Value-added looks at how much an individual teacher’s students learn in a given year, regardless of their level at the start of the year. Value-added is a key part of Ohio’s new teacher evaluation system, but is not an overall measure of teacher performance. The analysis included value-added scores and ratings for about 16,000 reading and math teachers in grades four through eight in more than 450 districts across the state. StateImpact Ohio and The Plain Dealer consulted with Ohio University researchers Anirudh Ruhil and Marsha Lewis, who are leading a statewide study of value-added and related measures, on the analysis. Anirudh Ruhil Marsha Lewis The pay analysis looked district-by-district at the correlation between teachers’ pay and value- added scores. It found significant correlations in just three districts across the state. None of those districts had more than 40 teachers rated. That analysis did not take into account other factors that might influence student learning, such as poverty levels, or other factors related to teachers, such as advanced degrees. It also did not take into account the small numbers of teachers who received value-added scores in some districts or the fact that some teachers had only a single year of value-added data available. Ruhil and Lewis cautioned that additional analysis is necessary to more completely describe the relationship between teacher pay and value-added.

21. Scale matters

22. The Naming of the Names ●If you publish, know why ●Explain to readers/listeners ●Engage online/offline