1. Evaluating the Quality of Student Achievement Objectives
Module 5
2017
Welcome to the Polk County Evaluating the Quality of Student Achievement Objectives, Module. A series of five modules have been prepared with a focus on the needs of teachers and administrators that will help to deepen your understanding of the SAO components, as well as the information that supports it.
Student Achievement Objectives are an option for teachers who do not receive a VAM score. Along with the performance observation and student performance, an SAO may be used for the “district determined performance measure” as determined by the Florida state statute.
In order to expand your knowledge of SAOs we suggest you view each of the modules. You may also wish to visit the Center for Assessments SLO Toolkit at

2. Module 5: Evaluating the Quality of a SAO
SAOs for Teacher Evaluation SAOs are content- and grade or course- specific learning objectives that can be validly measured to document student learning over a defined and significant period of time. SAOs can constitute an instructional improvement process driven by teachers and supported by leaders!
Many states and districts are creating educator evaluation systems that include academic student performance information. SAOs are one method to document the influence that educators have on student learning over a specific amount of time. SAOs are content- and grade or course-specific learning objectives that can be validly measured to document student learning over a defined and significant period of time, such as a semester or school year. SAOs constitute an instructional improvement process, driven by teachers in all grades and subjects.
Student Achievement Objectives provide the opportunity for teachers to be able to:
set meaningful goals,
collaborate with other educators around shared goals,
monitor student and teacher progress toward goals, and
evaluate the extent to which goals were achieved.
SAOs can constitute an instructional improvement process driven by teachers and supported by leaders.

3. Module 5: Evaluating the Quality of a SAO
Student Achievement Objectives consists of three components:
Learning Goal
Assessment(s)
Targets
Student Achievement Objectives consists of three components: a learning goal, assessment(s), and targets. This module will describe the process for evaluating the quality of these components, and additional information can be found in the modules:
Module 1: Introduction to Student Achievement Objectives
Module 2: Developing Learning Goals,
Module 3: Determining Assessments,, and
Module 4: Selecting Baseline Data and Establishing Targets for Student Achievement Objectives.

4. Module 5: Evaluating the Quality of a SAO
Identify the expected outcomes by the end of the instructional period for the whole class as well as for different subgroups, as appropriate.
Targets
Assessments should be of high quality, and designed to best measure the knowledge and skills found in the learning goal of this SAO. The assessment should be accompanied by clear criteria or rubrics to describe what students have learned.
Assessments
A description of what students will be able to do at the end of the course or grade based on course- or grade-level content standards and curriculum.
Learning Goal
Modules 1-4 provided the considerations for the development of each component of an SAO.
The Learning Goal is a description of what students will be able to do at the end of the course or grade. It is based on one or more of the overarching big ideas that are central to a discipline or course and have a lasting value beyond the classroom. The Learning Goal and all of the planning information for the SAO should reflect a SMART goal.
Assessments should be standards-based, of high quality, and designed to best measure the knowledge and skills found in the SAO Learning Goal. The assessments should be accompanied by clear criteria or scoring rubrics to describe the level at which students have learned the content.
Targets are the expected student outcome by the end of the instructional period. There are two key components necessary to consider when setting targets: the starting level and end goal. In order to predict the expected student outcomes on the identified assessments that measure SAO Learning Goal, it is first necessary to consider students’ current and actual performance by examining baseline data.

5. Module 5: Evaluating the Quality of a SAO
Jim is an elementary school principal responsible for evaluating the quality of his teachers’ Student Achievement Objectives. He is dedicated to ensuring that the SAO process is implemented with fidelity in his school, and that the SAOs are coherent and cohesive from class-to-class and from grade-to-grade. He wants to be sure that the Learning Goals address deep and meaningful learning expectations, that the assessments are aligned to these expectations, and that the targets are rigorous, but ambitious.

6. Module 5: Evaluating the Quality of a SAO
Collaboration
Team Meetings
School-based data
Student Information
Jim is aware that there are some structures, conditions, and resources necessary for ensuring the successful implementation of SAOs by his teachers. Therefore, prior to beginning the school year, Jim creates a schedule that provides opportunities for teachers to meet in grade-level teams in order to collaborate on the development of the SAO Learning Goal.
Jim then collects a variety of school-based data, including results on district assessments and state tests, along with other student information such as attendance rates and special education needs, to help teachers determine the Learning Goal focus for their course and grade.
In the collaboration sessions, his aim is for the teachers to work together to make informed instructional decisions based on the strengths and needs of their grade level students as seen by the prior year’s test and assessments, writing samples, and other pertinent information sources. Although each teacher has different students, he knows that providing teachers with the opportunity to work and talk together will enable them to set appropriate targets and to discuss the monitoring of students’ progress toward the targets.

7. Module 5: Evaluating the Quality of a SAO
Jim also wants to be sure his new and novice teachers are set up for success in their teaching experience, as well as in the development and implementation of their SAO. Therefore, he sets up a mentoring schedule for the new teachers and the mentor teachers, who have expressed interest, to meet on a scheduled basis. He wants to be sure that there is a plan of action for these teachers to meet regularly and on a formal basis.

8. Module 5: Evaluating the Quality of a SAO
Jim recognizes the need for on-going professional development for all of his teachers to understand the SAO process. Although the district has provided some training, Jim realizes that his teachers will need on-going support and additional training. He is planning for this training to take place during staff meetings and common planning times throughout the year. He is also planning to schedule formal targeted training at several points throughout the year for teachers who may be struggling with the SAO process.

9. Module 5: Evaluating the Quality of a SAO
Analyzing Data
Mentoring
Selecting Assessments
Developing Assessments
In addition, Jim wants to be sure that the targeted or differentiated professional development that he provides for his teachers include topics related to SAOs. He considers possible professional development opportunities for his teachers including:
Developing or selecting assessments to measure student performance of SAO Learning Goals
Analyzing data, and
Opportunities for more experienced teachers to be a mentor teacher for new or novice teachers.

10. Module 5: Evaluating the Quality of a SAO
Beginning of the Year SAO Meeting
What is the learning goal that you selected for your SAO and why did you select this goal?
Describe the summative assessments you plan to administer and explain how they align to your Learning Goal.
Describe the types of assessments you are anticipating to use in order to monitor progress. What information will you gain from these assessments that will allow you to aid students in achieving the Learning Goal?
Describe the baseline data you will collect or have collected and how you will use this information to set your targets.
Jim then sets up his schedule for meeting with grade level teachers to take place approximately 6 weeks into the school year. At these meetings grade-level teams will present their Learning Goals, the assessments that are selected for measuring the learning goal, and the targets established for the students. The timeframe for these meetings allows teachers to know their students well enough to set targets for their class.
He also prepares several questions for his teachers to ensure that they have considered the issues of content alignment between their Learning Goal and the identified assessments. Some questions he prepares for this initial meeting include:
What is the learning goal that you selected for your SAO and why did you select this goal?
Describe the summative assessments you plan to administer and explain how they align to your Learning Goal.
Describe the types of assessments you are anticipating to use in order to monitor progress. What information will you gain from these assessments that will allow you to aid students in achieving the Learning Goal?
Describe the baseline data you will collect or have collected and how you will use this information to set your targets.

11. Module 5: Evaluating the Quality of a SAO
Learning Goal
Acceptable Quality
A description of what students will be able to do at the end of the course or grade based on course- or grade-level content standards and curriculum.
Quality Needs Improvement
Insufficient Quality
Appropriately identifies and thoroughly describes an important and meaningful learning goal, with:
the big idea and the standard(s) clearly aligned to and measured by the learning goal,
a clear explanation of the critical nature of the learning goal for all students in the specific grade/ course,
a clear description of how the learning goal allows students to demonstrate deep understanding of the content standards within the identified time span, and
specific and appropriate instruction and strategies described to teach the learning goal.
In preparation for the beginning of the year meetings, Jim reviews and distributes the Rubric for Rating the Quality of SAOs to be sure that everyone is clear about the acceptable quality of each component of the SAO and planning document.
He reviews the expectations for the Learning Goal which includes the appropriate identification and thorough description of an important and meaningful learning goal, with:
the big idea and the standard(s) clearly aligned to and measured by the learning goal,
a clear explanation of the critical nature of the learning goal for all students in the specific grade/ course,
a clear description of how the learning goal allows students to demonstrate deep understanding of the content standards within the identified time span, and
specific and appropriate instruction and strategies described to teach the learning goal.

12. Module 5: Evaluating the Quality of a SAO
Assessments and Scoring
Acceptable Quality
Assessments should be standards-based, of high quality, and be designed to best measure the knowledge and skills found in the learning goal of this SLO. They should be accompanied by clear criteria or rubrics to determine student learning from the assessment.
Quality Needs Improvement
Insufficient Quality
Appropriately identifies and clearly describes:
high quality assessments, with evidence to support how the appropriateness and quality of the assessments has been established
scoring rubrics that appropriately differentiate student performance, including evidence to support these rubrics have been validated, and
progress-monitoring measures that will be used, including how instruction will be differentiated for all learners based on this information.
Acceptable quality for the Assessments and Scoring include:
Appropriate identification and clear description of:
high quality assessments, with evidence to support how the appropriateness and quality of the assessments has been established
scoring rubrics that appropriately differentiate student performance, including evidence to support these rubrics have been validated, and
progress-monitoring measures that will be used, including how instruction will be differentiated for all learners based on this information.

13. Module 5: Evaluating the Quality of a SAO
Targets
Acceptable Quality
Identify the expected outcomes by the end of the instructional period for the whole class as well as for different subgroups, as appropriate.
Quality Needs Improvement
Insufficient Quality
Clearly and thoroughly explains how the data are used to define teacher performance, including:
appropriate baseline data/information used to establish and differentiate expected performance, and
rigorous expectations that are realistic and attainable for each group of students using the documented high quality assessments.
And acceptable quality for the Targets include:
Clear and thorough explanation of how the data are used to define teacher performance, including:
appropriate baseline data/information used to establish and differentiate expected performance, and
rigorous expectations that are realistic and attainable for each group of students using the documented high quality assessments.

14. Module 5: Evaluating the Quality of a SAO
Mid-Year SAO Meeting
How have you ensured that all students, including your special education and English language learners, have been able to access this Learning Goal?
What professional learning, if any, have you sought out to ensure that students would be successful toward this Learning Goal?
Based on what you’ve learned about your students and how they are progressing toward the Learning Goal you’ve identified, would you make any adjustments to the Learning Goal if you were to choose a similar learning goal next year?
Describe how you are providing students with descriptive formative feedback on their assessments?
Using your mid-year data, describe the progress your students are making.
Tell me about a student whose progress isn’t what you had anticipated when you set your target and what you’ve learned about that child’s instructional needs.
Jim wants his teachers to be prepared for the information that will be needed during the mid-year and end-of-year meetings.
He has developed questions for the mid-year meetings that go beyond the generic “What evidence have you gathered so far to determine student achievement?” and “Are you on target to meet your goals?”. These focused questions include:
How have you ensured that all students, including your special education and English language learners, have been able to access this Learning Goal? In other words, specifically, what supports and scaffolds have you put in place for all students, and especially for struggling students?
What professional learning, if any, have you sought out to ensure that students would be successful toward this Learning Goal? What types of professional learning or other resources might be helpful in reaching your learning goal?
Based on what you’ve learned about your students and how they are progressing toward the Learning Goal you’ve identified, would you make any adjustments to the Learning Goal if you were to choose a similar learning goal next year? If so, what would you adjust and why? In what ways would these adjustments impact the big idea and/or standards?
Describe how you are providing students with descriptive formative feedback on their assessments? How do you assist students with using this feedback so that they can improve their work?
Using your mid-year data, describe the progress your students are making and 1) whether you will be able to move your students to the levels of achievement that you have set, and 2) whether you will need to adjust your targets and why.
Tell me about a student whose progress isn’t what you had anticipated when you set your target and what you’ve learned about that child’s instructional needs.
He feels these questions will give him the information he is looking for and provide teachers with a means of focusing instruction and assessment for the second half of the year.

15. Module 5: Evaluating the Quality of a SAO
End-of-Year SAO Meeting
Discuss the status of students’ achievement of the Learning Goal.
What professional learning, if any, have you sought out to ensure that students would be successful toward this Learning Goal?
Have the results of the formative assessments caused you to rethink your Learning Goal, summative assessments, and/or targets?
How have you considered the appropriateness of your assessments?
Describe the progress that specific students are making using specific evidence from the rubric being used and the student work. Describe the struggles that specific students are having using specific evidence from the rubric being used and the student work.
What situations have you faced during the course of the implementation of the SAO that caused you to believe that you would or would not meet the targets that you have set?
Because Jim wants his teachers to have an opportunity to reflect on their learning and professional development, he develops the following end-of-year questions:
Discuss the status of students’ achievement of the Learning Goal. This is not about the targets that were set, and whether you have met these targets, but rather discuss whether the instruction, strategies, and time that you had planned for are sufficient for students. What other instruction and time was required that you hadn’t anticipated? What instruction and time was not necessary that you expected?
Consider the formative assessments that you have used to monitor student progress toward the Learning Goal: Have the results of the formative assessments caused you to rethink your Learning Goal, summative assessments, and/or targets? If so, describe what plan of action you are planning.
How have you considered the appropriateness of your assessments? Do they seem to be working as you thought they would in terms of providing you with credible evidence of your students’ learning?
Summative assessments should constitute a “body of evidence” that students are able to independently demonstrate understanding of the Learning Goal:
Describe the progress that specific students are making using specific evidence from the rubric being used and the student work.
Describe the struggles that specific students are having using specific evidence from the rubric being used and the student work.
What situations have you faced during the course of the implementation of the SAO that caused you to believe that you would or would not meet the targets that you have set? What steps did you take to meet your targets?

16. Module 5: Evaluating the Quality of a SAO
As Jim prepares for his teachers to arrive for the upcoming school year, he believes that he has set up the structures for his teachers to be instructional leaders in their classrooms, as well as in the school building.

17. Module 5: Evaluating the Quality of a SAO
SAO Reflection Questions:
In what ways do teachers have opportunities to collaborate on developing a Learning Goal, analyzing assessment results, and monitoring student progress?
How have teachers been training and been allotted multiple opportunities to collect and analyze student data related to their SAO Learning Goal?
In what ways are teachers able to access professional development that connects to their content and student needs to improve achievement toward their SAO Learning Goal?
Reflect on the leadership necessary for implementing SAOs in a school:
In what ways do teachers have opportunities to collaborate on the developing a Learning Goal, analyzing assessment results, and monitoring student progress?
How have teachers been training and been allotted multiple opportunities to collect and analyze student data related to their SAO Learning Goal?
In what ways are teachers able to access professional development that connects to their content and student needs to improve achievement toward their SAO Learning Goal?

18. jthompson@nciea.org 401-316-7840