1. Measuring Up: Effective Strategies for Teacher Evaluation
Patricia A. Popp, Ph.D.
State Coordinator, Project HOPE-VA
Clinical Associate Professor
Xianxuan Xu, Ph.D.
Post-Doctoral Research Associate
School of Education
The College of William and Mary

2. Teacher Evaluation in an Era of Accountability
Currently, ESEA flexibility has been granted to 34 states and the District of Columbia. As part of the flexibility requirements, the states were required to establish new teacher evaluation systems that factor in student achievement progress for statewide implementation by the end of the school year.
All states that received Race to the Top funding are undertaking substantial reforms with teacher evaluation.
“The quality of an education system cannot exceed the quality of its teachers” (Barber & Mourshed, 2008).

3. Share of At-Risk Students on PISA (Not reaching PISA baselines): Reading

4. Share of At-Risk Students on PISA (Not reaching PISA baselines): Math

5. Percentage of Country’s Students in PISA Top Performing Groups: Reading

6. Percentage of Country’s Students in PISA Top Performing Groups: Math

7. What do we do about it?

8. How Long Do Students Attend School? Days in the Academic Year
Country
Days in an Academic Year
Canada
Average: 188
Finland
187
Singapore
200
Shanghai
180
South Korea
204
United States

9. How Long Do Students Attend Schools? Minutes in the Day
Location
Minutes in a School Day
Canada
304
Finland
240
Shanghai
390
Singapore
330
South Korea
264
United States
402

10. Annual Expenditures Per Pupil
How Much Do We Spend?
Location
Annual Expenditures Per Pupil
Canada
8,045
Finland
7,216
Shanghai
N/A
Singapore
South Korea
6,663
United States
10,259

11. Student/Teacher Ratio
Country
Average
Student/Teacher Ratio
Canada 25
Finland 19
Shanghai 39
Singapore 35
South Korea 36
United States 24
Source: Available at worldbank.org.

12. Which factor is a strong predictor of student achievement gains?
Enhancing Teacher Quality: Questioning
Which factor is a strong predictor of student achievement gains?
Class size
Classroom heterogeneity
School resource differences
It’s the teacher.
Sources: Wright, Horn, & Sanders, 1997;
Hanushek, Kain, & Rivkin, 1998,
TQR Teacher Quality Resources, LLC (c) 2005

14. Influences on Student Achievement: Explained Variance
Source: Hattie, J. Teachers make a difference: What is the research evidence.
Retrieved 20Nov08 from

15. Dallas Research: Teacher Quality
Dallas, Texas data: students per cohort
Comparison of 3 “highly effective” & 3 “ineffective” teachers (Jordan, Mendro, & Weerasinghe, 1997)

16. Dallas Research: Teacher Quality

17. Time in School Year Needed to Achieve the Same Amount of Learning
1/4
1/2
3/4 1
25th Percentile
Teacher
75th Percentile
Years Needed
If we place a child in a bottom quartile teacher’s class--an ineffective teacher’s class--and it takes that child a full academic year to learn the knowledge and skills that he or she will learn...if we place that same child in a top quartile teacher’s class--a high performing teacher’s class--we find that that child learns the same knowledge and skills in three-quarters of the time.
Leigh, Economics of Education Review (2010)

18. Time in School Needed to Achieve the Same Amount of Learning
Source: Leigh, A. (n.d.). Estimating teacher effectiveness from two-year changes in
students’ test scores. Retrieved May 22, 2007, from

19. Sequence of Effective Teachers
Low
High
52-54
percentile
points difference
Look at the difference a series of ineffective teachers can have on a student’s achievement.
Sanders & Rivers (1996)

20. Sequence of Effective Teachers
Low
High 13
percentile
points difference
High
Consider the case where we place a child with a highly effective teacher for three years in a row versus if we place that child with two ineffective teachers and then an effective teacher. That effective teacher will not be able to compensate for the effects from the poor teachers.
Sanders & Rivers (1996)

21. Revised Teacher Evaluation System in Virginia: An Overview

22. Primary Purposes of the Evaluation System
Improve student achievement through the quality of instruction by assuring accountability for classroom performance
Contribute to the successful achievement of the goals and objectives defined in a school division’s educational plans
Provide a basis for instructional improvement through productive teacher appraisal and professional growth
Share responsibility for evaluation between the teacher and the evaluation team in a collaborative process that promotes self-growth, instructional effectiveness, and improvement of overall job performance

23. A Flawed System
Problem No. 1: Observation equals evaluation Problem No. 2: Likely to rely on intuition, not evidence, to make judgments about teacher performance Problem No. 3: One size fits all Problem No. 4: Don’t communicate Problem No. 5: Fragmented evaluation process Problem No. 6: Irrelevant evaluation Problem No. 7: One-point rating scales Problem No. 8: No impact evaluation
1: In many evaluation systems observation means evaluation. There is no other means by which evaluators gather data for teacher evaluation
2: Subject to evaluator bias
3. Many evaluation systems do not distinguish between teachers with different levels of experience. Teachers who have 20 years of experience are evaluation in exactly the same way as brand new teachers, with no acknowledgement of the fact that the new teacher may need more frequent feedback
4: Some evaluation systems do not have a mechanism to provide feedback to the teachers.
5. In many cases, there is a fragmented evaluation process. Teacher induction, support, professional development, and evaluation are not aligned.
6. Some evaluation systems do not measure teachers on the things that really matter....”inspector mode of evaluation”
When everyone is rated at the top, it does a disservice to everyone.
The evaluation system carries not weight. It does not inform PD, does not acknowledge teachers who are doing great things, and it does not identify areas where struggling teachers need support.

24. What is the basis of the teachers’ evaluation?
Question 1
What is the basis of the teachers’ evaluation?

25. Performance Indicators Performance Appraisal Rubric
Enhancing Teacher Quality: Questioning
Main Components
Performance Standard
Standard 2: Instructional Planning
The teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective
strategies, resources, and data to meet the learning needs of all students.
Sample Performance Indicators
Examples may include, but are not limited to:
The teacher:
2.1 Uses student learning data to guide planning.
2.2 Plans time realistically for pacing, content mastery, and transitions.
2.3 Plans for differentiated instruction.
2.4 Aligns lesson objectives to the school’s curriculum and student learning needs.
2.5 Develops appropriate long- and short-range plans, and adapts plans when needed.
Performance Indicators
Performance Appraisal Rubric
Exemplary
Proficient
Proficient is the expected level of performance.
Developing/
Needs Improvement
Unacceptable
In addition to meeting the standard, the teacher actively seeks and uses alternative data and resources and consistently differentiates plans to meet the needs of all students.
The teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective strategies, resources, and data to meet the needs of all students.
The teacher inconsistently uses the school’s curriculum, effective strategies, resources, and data in planning to meet the needs of all students.
The teacher does not plan, or plans without adequately using the school’s curriculum, effective strategies, resources, and data.
There are three main components of the Stronge Teacher Evaluation System.
Performance standards define the criteria expected when teachers perform their major duties. There are seven performance standards for teachers. When teachers are evaluated, they are evaluated against the performance standards.
Next are performance indicators. They provide examples of observable, tangible behavior that indicate the degree to which teachers are meeting each teaching standard. We often call these the “look-fors.” They are the kinds of things one would look for if the teacher was successfully performing the standards. In the Stronge Evaluation System, we do not use the performance indicators as a checklist. In other words, the evaluator should not look at the list of indicators and say “I only saw two of the five indicators, therefore, the teacher must be ineffective.” School districts may modify the indicators if they wish. In addition, they may chose to add indicators to emphasize a particular focus area for the district. Performance ratings are made at the performance standard level, NOT at the performance indicator level.
The third main component are the performance appraisal rubrics. We will talk more about them shortly. During the summative evaluation, the evaluator will use a “preponderance of evidence” to judge where the teacher should be rated. Note that the rubric description under the effective level is the exact wording of the performance standard. This is because effective is the expected level of performance.
TQR Teacher Quality Resources, LLC (c) 2005

26. Performance Standards
Professional Knowledge
Instructional Planning
Instructional Delivery
Assessment of and for Student Learning
Learning Environment
Professionalism
Student Academic
Progress

27. Teacher Performance Standard 1: Professional Knowledge
The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences.

28. Teacher Performance Standard 2: Instructional Planning
The teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective strategies, resources, and data to meet the needs of all students.

29. Teacher Performance Standard 3: Instructional Delivery
The teacher effectively engages students in learning by using a variety of instructional strategies in order to meet individual learning needs.

30. Teacher Performance Standard 4: Assessment of and for Student Learning
The teacher systematically gathers, analyzes, and uses all relevant data to measure student academic progress, guide instructional content and delivery methods, and provide timely feedback to both students and parents throughout the school year.

31. Teacher Performance Standard 5:
Learning Environment
The teacher uses resources, routines, and procedures to provide a respectful, positive, safe, student-centered environment that is conducive to learning.

32. Teacher Performance Standard 6: Professionalism
The teacher maintains a commitment to professional ethics, communicates effectively, and takes responsibility for and participates in professional growth that results in enhanced student learning.

33. Teacher Performance Standard 7: Student Academic Progress
The work of the teacher results in acceptable, measurable, and appropriate student academic progress.
Note: Performance Standard 7: If a teacher effectively fulfills all previous standards, it is likely that the results of teaching -- as documented in Standard 7: Student Academic Progress -- would be positive.
The Virginia teacher evaluation system includes the documentation of student growth as indicated within Standard 7 and recommends that the evidence of progress be reviewed and considered throughout the year.
The evaluation system also encourages the use of multiple measures of student academic progress.

34. How will teacher performance be documented?
Question 2
How will teacher performance be documented?
The role of a teacher requires a performance evaluation system that acknowledges the complexities of the job.
Multiple data sources provide for a comprehensive and authentic “performance portrait” of the teacher’s work.

35. Multiple Data Sources Data Source Evaluator Teacher 
Informal Observations 
Formal Observations
Student Surveys
Portfolios/Document Logs
Self-Evaluation
Measures of Academic Progress
Reviews/approves
Selects/develops
Multiple data sources are needed to capture the full range of responsibilities of a teacher.
The Code of Virginia requires two sources of data to be used in student achievement: Observations and Measures of Academic Progress
Virginia Department of Education recommends other data sources that can improve the objectivity of teacher evaluation systems. These include student surveys, portfolios/document logs, and self-evaluation.

36. Measures of Academic Progress
Teachers
Percentage of Evaluation based on Student Growth Percentiles (SGPs)
Percentage of Evaluation based on Other Growth Measures
Teachers of reading and mathematics for whom SGPs are available 20
Teachers who support instruction in reading and mathematics for whom SGPs are available
No more than 20
20 to 40
Teachers who have no direct or indirect role in teaching reading or mathematics in grades where SGPs are available
N/A 40
The Uniform Performance Standards and Evaluation Criteria incorporate student academic progress as a significant component of the evaluation while encouraging local flexibility in implementation.
These guidelines recommend that student academic progress account for 40 percent of an individual’s summative evaluation.
Other student academic progress measures may include student achievement goal setting, which teachers develop student achievement goals based on baseline data at the beginning of the year, monitor student progress and adjust instruction throughout the year, and determine goal attainment at the end of the year.

37. Student Achievement Goal Setting
Step 3:
Create and implement teaching and learning strategies
Step 4: Monitor student progress through ongoing formative assessment
Step 1:
Determine needs
Step 2:
Create specific learning goals based on pre- assessment
Step 5:
Determine whether the students achieved the goal
Student achievement goal setting involves a multi-step process. Baseline performance is established by reviewing and analyzing data. Baseline data can be reviewed individually or in a collaborative manner with other teachers. For example, a grade level may review the data.
Then, based on baseline data, the teacher decides to focus attention on student improvement. For example, a fourth-grade team determines that their students performed well in mathematics last year, but not in reading. Therefore, they decide to focus the goal on reading. Each teacher creates his or her own goal based on the performance of the students in his or her classroom, but the goal area is decided as a grade level.
Then, the teacher sets an attainable goal, meaning that the goal is within reach and yet is not too easy. For example, increasing a percentile ranking on a norm-referenced assessment from 50th percentile to 80th percentile would be quite difficult.
The teacher then develops strategies that would support goal attainment. Strategies are critical to the goal setting process as they provide the means to the end, which is increased student achievement or program progress. Strategies will vary from class to class due to differences in age levels, subject areas, etc. Team planning will make strategies similar, but students are not at the same level across classrooms. So the teacher must customize the goal to fit the needs of his/her students.
The strategies are then implemented and student/program progress is monitored. At the end of the year, data is analyzed to determine whether the goal was attained.
Steps 1 & 2 occur during the first month of the school year or course.

38. What are the Purposes of Student Achievement Goal Setting?
Focus on student results
Explicitly connect teaching and learning
Improve instructional practices and teacher performance
Tool for school improvement
Student achievement goal setting focuses on the students results. It explicitly connects the roles that the teacher plays with student progress, thereby improving instructional practices. Goal setting is also a tool for school improvement. A school may focus on improving achievement in one area and the academic goals developed support the overall school goal.
Student achievement goal setting does not replace classroom observation. Classroom observation is a crucial tool in assessing teacher performance.
Student achievement goal setting is not the only source used to inform evaluation decisions. It is one source among many, including other valid measures, classroom observation, and document logs.

39. How will teacher performance be rated?
Question 3
How will teacher performance be rated?
For an evaluation system to be meaningful, it must provide its users with relevant and timely feedback.
To facilitate this, evaluators should conduct both interim and summative evaluations of teachers.
Summative evaluation ratings are based on behaviorally-anchored performance rubrics.

40. Evaluations Interim Evaluation Summative Evaluation
Used to document evidence of meeting standards
Does NOT include rating of performance
Summative Evaluation
Comes at end of evaluation cycle
- One year for probationary teachers
- Three years for continuing contract teachers
Assessment of performance quality
- Four point rating scale
- Performance rubric for every standard
Some teacher evaluation systems include an interim review, especially for probationary teachers, in order to provide systematic feedback prior to the completion of a summative evaluation.
The multiple data sources discussed in Part 3 are used to compile a Teacher Interim Performance Report that indicates if a teacher has shown evidence of each of the performance standards.
The evaluator should share her/his assessment of the teacher’s performance by a given date (for example, the last school day before winter break each year for Probationary teachers)

41. Evaluating Performance
Enhancing Teacher Quality: Questioning
Evaluating Performance
Category
Description
Definition
Exemplary
The teacher maintains performance, accomplishments, and behaviors that consistently and considerably surpass the established standard.
Exceptional Performance
Sustains high performance over period of time
Behaviors have strong positive impact on learners and school climate
Serves as role model to others
Proficient
The teacher meets the standard in a manner that is consistent with the school’s mission and goals.
Effective Performance
Meets the requirements contained in job description as expressed in evaluation criteria
Behaviors have positive impact on learners and school climate
Willing to learn and apply new skills
Developing/
Needs Improvement
The teacher often performs below the established standard or in a manner that is inconsistent with the school’s missions and goals.
Below Acceptable Performance
Requires support in meeting the standards
Results in less than quality work performance
Leads to areas for teacher improvement being jointly identified and planned between teacher and evaluator
We will use common terms
Unacceptable
The teacher consistently performs below the established standards or in a manner that is inconsistent with the school’s missions and goals.
Ineffective Performance
Does not meet requirements contained in job description as expressed in evaluation criteria
Results in minimal student learning
May result in employee not being recommended for continued employment
TQR Teacher Quality Resources, LLC (c) 2005

42. Developing/Needs Improvement
Sample Performance Appraisal Rubric
Standard I: Professional Knowledge
The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences.
Exemplary
Proficient
Proficient is the expected level of performance.
Developing/Needs Improvement
Unacceptable
In addition to meeting the standard, the teacher consistently demonstrates extensive knowledge of the subject matter and continually enriches the curriculum.
The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences.
The teacher inconsistently demonstrates understanding of the curriculum, content, and student development or lacks fluidity in using the knowledge in practice.
The teacher bases instruction on material that is inaccurate or out-of-date and/or inadequately addresses the developmental needs of students.
Teachers who are exemplary often serve as role models and/or teacher leaders.
Please note: The rating of “proficient” is the expected level of performance. Additionally, the recommended performance rubrics presented here may be modified at the discretion of school division decision makers.

43. Summative Rating
Summative ratings should apply the rating for each of the seven performance expectations, with the most significant weight given to Standard 7 - Student Academic Progress.
Weight each of the first six standards equally at 10 percent each
Weight Standard 7 – Student Academic Progress at 40 percent
The rating scale is applied for the summative evaluation of all teachers.
The performance rubrics guide evaluators in assessing how well a standard is performed.
They are provided to increase reliability among evaluators and to help teachers to focus on ways to enhance their teaching practices.
Please note: The rating of “proficient” is the expected level of performance.
Additionally, the recommended performance rubrics presented here may be modified at the discretion of school division decision makers.

44. Developing/Needs Improvement
Lessons Learned
Most practitioners believe new models provide specific measures of teacher effectiveness that are useful for distinguishing effective from less effective teachers.
Exemplary
Proficient
Proficient is the expected level of performance.
Developing/Needs Improvement
Unacceptable
In addition to meeting the standard, the teacher actively seeks and uses alternative data and resources and consistently differentiates plans to meet the needs of all students.
The teacher plans using the state’s standards, the school’s curriculum, effective strategies, resources, and data to meet the needs of all students.
The teacher inconsistently uses the school’s curriculum, effective strategies, resources, and data in planning to meet the needs of all students.
The teacher does not plan, or plans without adequately using the school’s curriculum, effective strategies, resources, and data.

45. Lessons Learned Formal Observation Post-Conference
Most practitioners believe that new models have the potential to improve teaching and learning by providing useful feedback that can be used to diagnose and guide teacher improvement.
Formal Observation Post-Conference
Teacher Self-Assessment
Student Learning Objectives
Student Surveys

46. Lessons Learned
With more rigorous assessment to identify problems and recognize excellence, investments in teacher development can be better related to school and division goals for improvement.
Strengths and Weaknesses Identified
Targeted Professional Development
Evaluation

47. Lessons Learned
Practitioners appreciate the value in using multiple data sources to provide evidence of performance standards.
Observations
Student Learning Objectives
Teacher Performance Standards
Documentation
Logs
Student Surveys

48. Lessons Learned
Practitioners believe that new models set up realistic expectations for teacher performance, and they reflect the most important elements of effective teaching.
Professional Knowledge
Instructional Delivery
Learning Environment
Instructional Planning
Assessment of/for
Learning
Professionalism
Student Progress

49. Lessons Learned
The evaluation framework is valid in terms that the process standard ratings of teacher have a moderate ability to predict the student academic progress. In addition, there is a significant correlation between each of the six process standards and student academic progress.

50. Lessons Learned New models can be time-consuming to implement.
Practitioners mistrust the validity of student progress models as a measure of student growth.
Teachers believe that the nature, quality, and credibility of the evaluation process vary depending on the qualifications of the evaluators.

51. Concluding Thoughts on Transforming Teacher Evaluation
State
Convene stakeholders across the state to design, implement, and improve evaluation system.
Develop validated and reliable evaluation measures.
Provide incentives and on-going support to school divisions.
Make knowledge of new developments in teacher evaluation part of leadership and teacher preparation programs.
Make evaluation count.

52. Concluding Thoughts on Transforming Teacher Evaluation (Continued)
Schools and School Divisions/Districts
Clear expectations of the WHAT will be evaluated.
Clear communication of the evaluative criteria for effective performance.
Instruments and procedures for teachers to provide evidence from multiple sources of the HOW of their performance.
Increase the use of evaluation results to inform professional development.