1. Designing Student Growth Measures for CTE
January 22, 2013
Designing Student Growth Measures for CTE

2. OTES Provides for Multiple Evaluation Factors

3. Who should be evaluated?
…Any person who is employed under a teacher license issued under this chapter, or under a professional or permanent teacher’s certificate issued under former section of the Revised Code, and who spends at least fifty per cent of the time employed providing student instruction.
Would a guidance counselor with a teaching license be evaluated under OTES? No.
Would a speech therapist who holds a teaching license be required to be evaluated? No, if only performing speech duties.
Would an intervention specialist who works as an inclusion teacher be evaluated? Yes.
ORC

4. HB 555 Changes Category A
Under the Student Growth Measures, teachers are divided into three categories:
Category A: Teachers for whom teacher-level value-added data is available
Category B: Teachers for whom data from an assessment that is on the ODE Vendor Approved List is available
Category C: Teachers for whom no teacher-level value-added or vendor approved assessment data is available
LEAs can choose to implement LEA measures for Category A and Category B teachers. If LEAs choose to implement multiple measures of student growth, they must allocate a specific percentage weight to the measures as shown in the graphic below.

5. Category A is now divided into A1 and A2

6. Timeline for Implementation
school year with the following exceptions:
TIF/SIG/RttT schools according to their grant timelines or scope of work
If bargaining agreement was entered into prior to 9/24/12, immediately upon expiration of agreement

7. Suggested Implementation Timeline

8. Definition of Student Growth
For the purpose of use in Ohio’s evaluation systems, student growth is defined as the change in student achievement for an individual student between two or more points in time.
Measuring growth rather than achievement is more fair to teachers and principals whose students start the school year already behind and below grade level.
Excerpted from Measuring Student Growth for Teachers in Non-Tested Grades and Subjects: A Primer

9. 3 Potential Measures for Student Growth
Depending on LEA decisions and the availability of data:
Value-added
Approved vendor-created assessments
LEA determined SGMs using SLOs
The challenge for measuring student growth is that there is not a single assessment that can be used for all teachers.

10. 3 Measures of Student Growth
Value-Added
Approved Vendor Assessments
LEA Measures
EVAAS Reports
4-8 MRM/URM
Extended Reporting w/Terra Nova and ACT EOC exams
Approved ODE List
Vendor had to show a metric for student growth
List is fluid
Student Learning Objectives (SLOs)
Shared Attribution
(Other) Vendor Assessments
Diagnostic
Reliable
Valid
Fair
Rigorous
Attributable
Comprehensive
3 Measures of Student Growth

11. 1. Teacher Value-Added MUST use if available
10-50% if applicable for
Category A1 >25% effective July 2013
Category A1: full 50% effective July 2014
Category A2-proportionate
EVAAS Value-Added metric, aggregated across subject areas
1-year report; or 2- or 3-year rolling average, based on availability
Phased-in implementation of reading & mathematics, Grades 4-8
Extended Reporting (other grades & subjects) being piloted
In Ohio, Value‐Added refers to the EVAAS Value‐Added methodology, provided by SAS, Inc. This is distinct from the more generic use of the term “value‐ added,” which can represent a variety of statistical modeling techniques. The Ohio EVAAS Value‐Added measure of student progress at the district and school level has been a component of the Ohio Accountability System for several years.
Value‐Added calculations currently utilize data from the Ohio Achievement Assessments (OAA). As the new Partnership for Assessment of Readiness for College and Careers (PARCC) assessments become operational and replace the current assessment system, they will be integrated into the Value‐Added calculations.
Additionally, the EVAAS data reporting system has added several features to help educators use this important data. Battelle for Kids (BFK) is providing professional development and other related services across the state.

12. 2. Approved Vendor Assessments
From ODE-Approved List
Vendors demonstrate how assessment can measure growth
MUST use for Category B Teachers
10-50% if applicable and no Value-Added data available
Category A1-LEA decision in SY , not applicable for
Category A2-LEA decision
HB 153 requires ODE to develop a list of student assessments that measure mastery of the course content for the appropriate grade level, which may include nationally normed standardized assessments, industry certification examinations, or end‐of course examinations for grade levels and subjects for which the Value‐Added measure does not apply (the non‐tested grades).
ODE released a Request for Qualifications (RFQ) so interested vendors could demonstrate that their assessments qualified for use in Ohio schools. The list of approved assessments will be maintained and updated by ODE.

13. 3. LEA-Determined Measures
MAY use: LEA decision for Category A and B
0-40% if used in combination with Category A or B measures
24% max for Category A1 in SY
0% for Category A1 in and thereafter
Proportionate to Category A2 teacher’s schedule
MUST use if Category C Teacher
50% if no Category A or B data available
For subjects in which traditional assessments are not an option – such as art or music – LEAs should establish a process to create
student learning objectives (SLOs) to measure student progress in those courses.

14. 3. LEA-Determined Measures
Three types of LEA-Determined Measures
Student Learning Objectives (SLOs)
Shared Attribution
Approved Vendor assessments
Category A2 teachers can use local measures after VA is applied proportionately.
Category A1 teachers can use local measures in only (24% max).
Student Learning Objectives process for using measures that are specific to relevant subject matter. Measures must be district-approved and may include:
Locally developed assessments;
Pre-Post assessments;
Performance-based assessments
Portfolios.
Shared Attribution Measures are student growth measures that can be attributed to a group. These measures encourage collaborative goals and may be used as data in the student growth component. Shared attribution may include:
Building or District Value-Added is recommended if available;
Building teams (such as content area) may utilize a composite VA score;
Building- or District-based SLOs
Vendor Assessment as a Local Measure: Teacher Category A (with Value‐Added) also may use Vendor
assessments as an LEA‐determined measure if using both.

15. What is an SLO?
A measurable, long‐term academic goal informed by available data that a teacher or teacher team sets at the beginning of the year for all students or for subgroups of students.
A measure that demonstrates a teacher’s impact on student learning within a given interval of instruction.

16. On Educator Evaluation page, click SGMs section.
SGM Resources
On Educator Evaluation page, click SGMs section.
Scroll down on the ODE home page and click “Educator Evaluation”. The page will display next. ODE’s Educator Evaluation webpage with link to SGMs.

17. SLO Resources Introduction to SLOs
Student Learning Objective Guidebook
Student Growth Measures for Teachers
SLO Writing Template
SLO Checklist for Writing & Approving
Sample SLOs
Snapshot of ODE’s SGM webpage with resources.

18. Reinforce Best Teaching Practice Through
Setting goals for students Using data to assess student progress Adjusting instruction based upon progress

19. Why is Ohio Using SLOs?
Reinforce promising teaching practices Can be used in all subjects/content areas Are adaptable Provide teachers some ownership on how they are evaluated
These qualities encompass good teaching practices.
Educators using best practices already follow an informal SLO process!
SLOs help formalize good teaching by requiring each of these steps and using the growth targets to inform evaluation results.

20. Potential For Collaboration
SLOs can be used to promote collaboration and reflection on practice among teachers.

21. Measures for SLOs
SLOs can be created drawing on different data sources:
teacher created assessments
performance assessments
rubric‐based assessments
business & industry certification
state or national assessments
SLOs are not dependent upon the availability of standardized assessment scores.
Instead, SLOs can draw upon different data sources such as end of course exams, performance-based assessment scored by a rubric, or district-created or team-created assessments.
SLOs can be highly adaptable, quickly reflecting changes in curriculum and available assessments.

22. Do all Teachers Need to Write SLOs?
ODE recommends that all teachers create at least one SLO this year to gain experience with the SLO process.
In subsequent years, teachers should create two to four SLOs per year.
Category A1 will not have the option to use local measures beginning July 2014.
LEAs can determine which measures teachers will use for the student growth measure.
2-4 SLOs per teacher representative of the teacher’s schedule.
We felt like we needed a couple of high school teachers’ schedule as an example here. The group got sidetracked asking how many SLOs a teacher needs. A couple examples would be helpful. Explain each situation and district is unique and move on.

23. The SLO Development Process
LEAs have some flexibility to shape the process to fit local contexts, but ODE recommends the following steps:
1. Review baseline data;
2. Create SLOs;
3. Obtain SLO approval, per local process; ODE recommends teams review and approve;
4. Monitor progress toward attainment of SLO growth targets;
5. Review evidence and evaluate progress towards and attainment of SLO growth targets.
ODE also recommends that building-level committees review and approve all of the SLOs so that they develop an understanding of the type and rigor of the SLOs across a school. Please note, however, that LEAs are not required to use the committees and can choose their own evaluator(s) for the SLO process. By centralizing the approval process, the committee will be able to support those teachers who set targets that are too high or too low, ensuring consistency within the building. Also, the committee will ensure that all SLOs are aligned to the academic standards as well as school and district priorities.

24. High-Quality SLOs Include:
Baseline & Trend Data
Student Population
Interval of Instruction
Standards & Content
Assessment(s)
Growth Target(s)
Rationale for Growth Target(s)
These are reflected on the SLO writing template.

25. Student Learning Objective Template

27. SLO Template Checklist
Ensure you keep this template beside you while writing SLOs. If you can check off each box (a couple could be N/A), then your SLO will likely be approved and you will not to revise it.
In order to approve an SLO, all checkboxes, save a couple that could be marked N/A, should be checked.
If not, send back for revisions and allow approximately 10 days for resubmission.

28. SLO Approval SLOs are approved at the local level.
ODE recommends an existing committee.
Provide feedback: both cool & warm.
Recommend allowing 10 days for revisions.

29. SLO Approval Process
Committees should go through a calibration process.
Read over the entire SLO first.
Using the checklist, review the SLO.
Discuss whether it meets each criterion and provide feedback to the teacher.
Develop a plan for tracking SLOs returned for revisions.

30. High-quality SLOs include the following criteria:
Baseline and Trend Data
Student Population
Interval of Instruction
Standards and Content
Assessment(s)
Growth Target(s)
Rationale for Growth Target(s) 30

31. Baseline and Trend Data:
Identifies source(s) and summarizes student information (test score from previous years, results of pre-assessments) in numerical and narrative form.
Reviews trend data to inform the objective and establish the amount of growth that should take place.
Identifies student strengths and weaknesses.
There are 7 components of an SLO. We will now review each component.
Helps the teacher better understand how prepared their students are for the standards addressed during the year/course.
Could include:
End-of-year data from the previous year
Baseline data from a district assessment
Pretest
Student Work Samples
Conduct a detailed analysis of the student data.

32. Student Population: Includes all students in the class.
Describes the student population (number, course, grade level, etc.).
Considers any contextual factors that may impact growth.
Does not exclude subgroups of students that may have difficulty meeting targets. Provide details.
Should cover as many students as possible. If any students are excluded, teacher should acknowledge this and explain why.
Students covered under an SLO must be proportional and representative of the teacher’s schedule. For example, if a science teacher teaches four sections of biology and two sections of earth science, two SLOs might cover the biology classes and one might apply to the earth science classes.
Note any other important characteristics of the student population. For example, any special needs, ELL, TAG?
How representative is this group of students to all of the students the teacher typically instructs?

33. Interval of Instruction:
The duration of the course that the SLO will cover including the start and end dates.
Include how frequently the course meets and length of a class period.
Include any other relevant information that could impact student growth.
Does the class meet daily?
Is it an academic year or a semester course?

34. Standards and Content:
Specify which standards the SLO covers.
Broad enough to represent the most important learning or overarching skills, but narrow enough to be measured.
What content do the students need in order to be successful next year or in the next course? Explain why.
Is this a targeted SLO?
The content or skill area should represent the essential learning and big ideas/domains of the course content such as key skills or overarching content, and should be selected based upon the identified areas from the data analysis.

35. Assessment(s):
Identify the assessment. Who created/ reviewed it? Describe its structure.
Describe how the assessment provides “stretch” for low and high achieving.
Identify supplemental assessments.
Provide specific details on how multiple tests will be combined into a summative score.
Follow assessment guidelines.
The content or skill area should represent the essential learning and big ideas/domains of the course content such as key skills or overarching content, and should be selected based upon the identified areas from the data analysis.

36. Identifying Appropriate Assessments

37. Selecting Assessments for SLOs
Selecting and approving assessments can be one of the most challenging and important steps of the SLO process. Assessments enable teachers to determine growth toward and attainment of the SLO.
ODE highly recommends that educators consult the Guidance on Selecting Assessments.
This can be challenging, but it is one of the more important steps of the SLO process.

38. Criteria for Selecting Assessments
Was the assessment(s) reviewed by content/industry experts?
Is the assessment aligned to both the SLO and the standards?
Does the assessment have enough stretch?
Is the assessment valid & reliable?
Ensure assessments have sufficient “stretch” for both low- and high-achieving students. Assessment contains questions that are of varying difficulty and cover some prerequisite and advanced knowledge or skills. Teachers may not be able to make an informed judgment about the needed stretch of the assessment until they have analyzed the baseline performance.
When identifying assessments, educators should keep in mind that assessments must:
Be aligned to national or state standards and to the SLO growth target (it measures the skills or content addressed by the SLO).
Be reliable, meaning that the assessment produces accurate and consistent results.
Be a valid measure, meaning that the assessment measures what it is designed to measure.
Be realistic in terms of the time required for administration.
The assessment should be, at a minimum, reviewed at the district level by grade-level and/or content-level experts.

39. Selecting Assessments for SLOs
ODE strongly recommends LEAs not allow assessments created by one teacher for use in his/her classroom.
If a teacher must create an assessment that is unique to his or her classroom, ODE strongly recommends that the teacher develop the assessment with:
a school or district administrator with expertise in assessment,
a special educator, an English language learner (ELL) specialist,
and/or a content team member.
At a minimum, the assessment should be reviewed at the district level by content experts such as grade-level or subject-level content experts.

41. Growth Target(s)
Growth targets should be developmentally appropriate, rigorous, and attainable.
Ensure all students have a growth target (not necessarily a passing score).
Refer to baseline/pretest data and course requirements.
Provide tiered targets to encompass all learners.
*Rank order pretest scores and look for breaks in data.
Because SLOs are flexible, they allow teachers to create growth targets that are appropriate to the teacher’s individual class, grade, or subject.
In the beginning years of SLO implementation, knowing how to set rigorous yet realistic targets may be a challenge for teachers.
Growth targets should include specific indicators of growth; such as percentages or questions answered correctly that demonstrate an increase in learning between two points in time.
Recommended: The target can be tiered for specific students in the classroom to allow all students to demonstrate growth or the target can be equally applicable to all students in a class, grade, or subject. Using tiered targets for students allows teachers to identify where each student begins the course and to determine the appropriate amount of growth for the student based on their baseline data.
Should be rigorous, yet attainable, as determined by the baseline or pretest data.
Acceptable growth targets allow the teacher to demonstrate growth for all students while the unacceptable SLOs focus solely on student mastery.

42. Acceptable & Unacceptable Growth Targets
First Unacceptable Target: (This is unacceptable because it does not show growth.) Achievement based for 80% of the students.
Second Unacceptable Target: (This is unacceptable for two reasons: (1) students that are scoring at 50 need to make greater gains than only 10 points to reach an acceptable level of growth; (2) teachers must aim to grow all students, so those students scoring high on the pre-assessment need to be challenged with a higher goal and likely an additional assessment to illustrate their growth.)

43. Rationale for Growth Target(s)
Rationale ties it ALL together.
Explain how you used the data to establish the targets.
Refer to students strengths/weaknesses based upon data.
Explain why this content is the most important.
Rationales should be aligned to broader school and district goals.
*Connect pre-test data + tiered targets = expectations (show connections & accountability)
High-quality SLOs include strong justifications for why the growth target is appropriate and achievable for this group of students.
The rationale should be a precise and concise statement that describes the student needs and refers to the evidence (data) that informed the creation of targets.
When applicable, rationales should also reference school and district goals or priorities.

44. QUESTIONS?

45. Regional Student Growth Measure Specialists:
Support
Carolyn Everidge-Frey, Assistant Director
Office of Educator Equity and Talent
Regional Student Growth Measure Specialists:
Chad Rice SE Region
Mark Robinson NE Region
Donna Huber Central
Apryl Ealy NW Region
Katrina Wagoner SW Region

46. SLO SCORING TEMPLATE

47. Percentage of students that met or exceeded growth target
SLO Scoring matrix
Percentage of students that met or exceeded growth target
Descriptive rating
Numerical rating
90-100
Most Effective 5
80-89
Above Average 4
70-79
Average 3
60-69
Approaching Average 2
59 or less
Least Effective 1

48. Final SLO Rating

49. Student Growth Measures
Evaluation Matrix
Teacher Performance 4 3 2 1
Student Growth Measures
Above
Accomplished
Proficient
Developing
Expected
Below
Ineffective
This is the lookup table.

50. Research and Best Practices
Updated as research and best practices emerge
Ohio Education Research Center (OERC)