1. Educator Evaluation Systems & Effectiveness Labels Venessa Keesler, Ph.D. Office of Evaluation, Strategic Research and Accountability Michigan Department of Education and Carla Howe West Virginia Department of Education

2. Overview of Current Plan and Issues Key important messages: Key important messages: – This was the FIRST YEAR; 800+ different systems (we have data to show this) – Districts did MASSIVE amounts of work to accomplish this – We do not believe that huge numbers of MI teachers are ineffective

3. Current Circumstances Our current legislation has allowed for local systems of evaluations, which has given districts flexibility to design systems that work best for them. Over 800 systems across the state Over 800 systems across the state Varying degrees of implementation across the state Varying degrees of implementation across the state Public reporting of effectiveness labels is required by SFSF Released in November via mischooldata.org Released in November via mischooldata.org Teachers labels reported in aggregate by school (number of teachers in each of the four categories) Teachers labels reported in aggregate by school (number of teachers in each of the four categories) Principals/Administrators reported at the district level. Principals/Administrators reported at the district level.

4. Important Context for the 2011-12 Results First year of implementation of NEW systems based on student growth measures First year of implementation of NEW systems based on student growth measures State provided student growth measures are only available in grades 4-8 for reading and mathematics State provided student growth measures are only available in grades 4-8 for reading and mathematics Varying components across systems (i.e. between districts) Varying components across systems (i.e. between districts) Varying percentages of growth across systems (i.e. between districts) Varying percentages of growth across systems (i.e. between districts) Some districts on prior contract (i.e. No new system, but reporting labels was required) Some districts on prior contract (i.e. No new system, but reporting labels was required)

5. K-12 Educator Evaluation Survey 792 districts completed the survey about their Evaluation systems from April to August Required to be completed by SFSF Results provide valuable insight into local systems The types of frameworks used The % of student growth as a component (law states significant, but it isnt defined until 2013-14) Types of growth measures included Types of decisions informed by the results of evaluations

6. Other Frameworks reported include: Charlotte Danielson Framework AND a local component, Teacher Advancement Program, My Learning Plan, 5 Dimensions of Teaching and Learning, Local District or ISD framework, McREL, STAGES, Kim Marshall Rubrics 50% of reporting districts 54 districts with a prior contract did not have to incorporate growth or a new system in 2011-12 # of districts PRELIMINARY/DRAFT FINDINGS

7. Appropriate given the FIRST year of local evaluation systems # of districts

8. Other ways growth data are measures include: Combination of data from multiple assessments, pre/post test data, combination of local, state, national measures, benchmark testing, several sources as agreed upon in the professional growth plan # of districts

9. Others types of assessment data reported that factor into educator evaluations include: AIMSweb, DRA, Ed Performance Series, Fontes & Pinnell, STAR Reading and Math, CBM for Math, DELTA Math # of districts

10. Others types of assessment data reported that factor into educator evaluations include : AIMSweb, DRA, Ed Performance Series, Fontes & Pinnell, STAR Reading and Math, TerraNova, ITBS, DELTA Math # of districts

11. Others types of assessment data reported that factor into educator evaluations include : AIMsweb, Ed Performance Series, STAR Reading and Math, Study Island # of districts

12. Others types of assessment data reported that factor into educator evaluations include : common assessments, district benchmark assessments, Scantron Performance Series # of districts

13. Others types of decisions include: Assignment to committees or roles beyond the classroom, classroom support and assistance, layoff/recall/transfer, mentoring, staff placement, scheduling, setting improvement goals, merit pay # of districts

14. Other Factors Reported As Part of Evaluations Absenteeism rateProfessional responsibilities Classroom managementStudent achievement data Content knowledgeAdherence to School Improvement Plans Instructional practicesCommitment to School & District Use of technologyLearning environment Pedagogical knowledge and practice Parental communication/involvement Professional development participation Relationships with students

15. UNDERSTANDING EDUCATOR EVALUATION LABELS IN MI Overview of Statewide Results

16. Caveat…. Labels are not EQUAL across districts However, we know that people will want this type of analysis and we want it done appropriately

17. Statewide Results LabelNumberPercent Ineffective7750.82% Minimally Effective1,9982.11% Effective70,74274.74% Highly Effective21,14122.33% IMPORTANT NOTES: Based on the labels as determined by the local evaluation system; rigor of label designation is not consistent across districts THERE is differentiation in label reporting now, 22% of teachers are reported as highly effective moving away from a satisfactory/unsatisfactory system We do not believe that 1% of teachers labeled as ineffective is unreasonable in the first year

18. Impact of growth Law required districts to implement systems based in significant part on student growth How do the labels look when the district used growth in greater percentages?

19. Growth and eval labels Less than 10% 11-20%21-30%31-40%41-50%State Ineffective 0.6% 1%0.5%2%.82% Min Effective 1%2%3%6%3.5%2% Effective83%71%74%64%68%74% Highly Effective 15%26%23%30%27%22% 26,25322,63925,3095,6503,14387,575 More differentiation in labels when growth counts at a higher rate LESS differentiation without growth

20. Distribution of Labels By Percent of Evaluation Based on Growth

21. Key Takeaways Distribution of labels (i.e. number of teachers in each category): – Is appropriate in Year 1 of implementation – Reflects differentiation (esp highly effective/effective) BUT we also see that systems using higher proportions of growth are able to make those differentiations more accurately The statewide evaluation system will move us toward more growth measures at higher rates

22. Who is more likely to be rated as highly effective or effective? Teachers more likely to appear in highly effective category (versus other three) and in effective category (versus other two): Female teachers Those with more time in the same district Teachers with a professional certificate (as opposed to all others) Those with a masters degree or higher Teachers in districts with growth over 40% in their system

23. Who is less likely to be rated as effective or highly effective? Older teachers New teachers (those in their first year of teaching) Mathematics, science, social science, special education and world language teachers (relative elementary teachers) Teachers in systems where growth is less than 10% of the evaluation system

24. Relationship between effectiveness labels and Priority/Focus/Reward Important to remember: – A school-level designation does not mean that all teachers within that school are in a given level of effectiveness – Example: In a Priority School, there will be effective teachers as well as ineffective teachers

25. IneffectiveMinimally Effective EffectiveHighly Effective Priority2.5%6.3%74%18% Focus0.5%1.5%80%18% Reward0.5%0.85%74%24% Statewide0.8%2%74%22% Notes: There are significantly more teachers reported as ineffective and minimally effective in Priority Schools than the statewide number, and in Focus or Reward schools. Effectiveness Labels in Priority, Focus and Reward Schools

26. Key Takeaways from the Results 1)These results are reasonable for the first year; represent a huge effort on the part of districts 2)There is differentiation in the system; there will be more as growth becomes a higher component; but we still do not believe large numbers of Michigan teachers are ineffective

27. Questions? Contact Michigan Department of Education Office of Evaluation, Strategic Research and Accountability 877-560-8378 Option 6