1. Colorado Department of Education, Educator Effectiveness
Implementing the Specialized Service Professional State Model Evaluation System
Colorado Department of Education, Educator Effectiveness

2. How Do You Feel About SB 10-191?
12/2/2018
How Do You Feel About SB ?
Choose a picture from below that best reflects your feelings regarding SB Why did you select the picture you did?

3. What’s on Your Mind? 3

4. Purposes of S.B
A system to evaluate the effectiveness of licensed personnel and continually improve the quality of education and student outcomes.
Provide meaningful feedback for professional growth and continuous improvement.
Provide a basis for making decisions in the areas of hiring, compensation, promotion, assignment, professional development, earning and retaining non-probationary status, dismissal, and nonrenewal of contract.

5. Critical Effects of S.B. 10-191
Requires statewide minimum standards for what it means to be an _________ Specialized Service Professional
Requires ______ evaluation of all Specialized Service Professionals
Requires that all Specialized Service Professionals be evaluated at least ___% on the _________ of the students they support
“effective”
annual 50
outcomes

6. Priorities of Implementation
Human judgment
Data should inform decisions, but human judgment will always be a part of the process
Processes and techniques are recommended to improve individual judgment and minimize errors and bias
Embodiment of continuous improvement by monitoring
Data from pilot and rollout intended to capture what works and what doesn’t
Changes in practices and tools
Emerging research and best practices

7. Priorities of Implementation
Providing credible and meaningful feedback with:
Actionable information
Opportunities for improvement
Idea that this is a process and not an event
Involves all stakeholders in a collaborative process
Families, teachers, specialized service professionals, administration, school board, etc.
Specialized service professionals involved throughout development process

8. Priorities of Implementation
Takes place within a larger, aligned and supportive system
All components of the system must focus on increasing the number of specialized service professionals and students who are successful
Turn and Talk
Which priority resonates with you and why?

9. SSP Timeline of Implementation
12/2/2018
SSP Timeline of Implementation
Year 1:
Create 9 work groups comprised of experts from the field
Work groups provide recommendations for definition, standards and elements (aligned with teacher) and measures of student outcomes to State Council for Educator Effectiveness
SCEE presents recommendations to State Board of Education
CDE facilitates work group creation of draft professional practice rubrics
Release application for pilot districts/BOCES; Select pilot sites
19 pilot sites including:
4 BOCES
Large suburban/urban districts
Small rural districts
Online school

10. SSP Timeline of Implementation
12/2/2018
SSP Timeline of Implementation
Year 2:
All SSPs evaluated using current system employed in district/BOCES
Pilot draft professional practice rubrics for Specialized Service Professionals
Collect feedback on professional practice rubrics and measures of student outcomes
Refine rubrics and measures of student outcome based on feedback collected
Provide training to districts and BOCES wanting to use the State Model System for SSPs

11. SSP Timeline of Implementation
12/2/2018
SSP Timeline of Implementation
Year 3:
Statewide implementation of all SSP Standards and Elements, including measures of student outcomes
Conduct a validation study for professional practice rubrics using pilot sites
If able to earn non-probationary status, this year considered “hold harmless”
Year 4:
Continued statewide implementation of SSP Standards and Elements, including measures of student outcomes
First year an ineffective or partially effective rating would count toward two consecutive years of ineffectiveness for those able to earn non-probationary status

12. Prior to the beginning of Spring Semester 1.
Training 2.
Annual Orientation 9.
Goal-Setting and Performance Planning 3.
Self-Assessment
Evaluation Process
for
Specialized Service
Professionals
End of May
Mid-June
May 15
End of September.
Prior to the beginning of Spring Semester
Train: Prior to the beginning of School.
Orient: Within the first week of School.
End of June
End of September. 8.
Final Ratings 4.
Review of Annual Goals and Performance Plan 7.
End-of-Year Review 5.
Mid-Year Review 6.
Evaluator Assessment

13. Building Your Evaluation System
12/2/2018
Building Your Evaluation System
Now that you are familiar with the evaluation process, think about how this will work in your setting.
Who will serve as evaluators?
Will you have more than one person contributing to the evaluation results?
How many observations will you conduct?
What will those observations look like?
How will you handle observing itinerant staff?
How can an evaluator provide a meaningful evaluation to an SSP who’s area of expertise they may not share?
How can an SSP support an evaluator in gaining knowledge of their expertise in order to provide a meaningful evaluation?

14. Definition of Specialized Service Professional Effectiveness
Effective specialized service professionals in the state of Colorado are vital members of the education team and have the knowledge and skills necessary to ensure that diverse student populations have equitable access to academic instruction and participation in school-related activities. Effective specialized service professionals develop and/or implement evidence-based services or specially designed instruction to meet the unique needs of their students. They support growth and development to close achievement gaps and prepare students for postsecondary and workforce success. They have a deep understanding of the interconnectedness of the home, school and community and collaborate with all members of the education team to strengthen those connections. Through reflection, advocacy, and leadership, they enhance the outcomes and development of their students.

15. Framework for System to Evaluate Specialized Service Professionals
Colorado Department of Education
Framework for System to Evaluate Specialized Service Professionals
Definition of Specialized Service Professional Effectiveness
Quality Standards
I. Professional Expertise
II. Learning Environment
III. High Quality Delivery
IV. Reflect on Practice
V. Leadership
VI. Student Outcomes
50% Professional Practice Standards % Measures of Student Outcomes
Observations of Professional Practice
Expert Input
Other Measures Aligned with CDE Guidance
Weighting: How Much Does Each Standard Count Towards Overall Performance?
Match of Measures of Student Learning to Assigned Duties
Weighting:
State Scoring Framework: How Do Measures of Quality Standards
Result in a Determination of Individual Performance?
Effectiveness Ratings
Ineffective Partially Effective Effective Highly Effective
Appeals Process*
Applies when professionals are not at-will employees

16. State Model Rubric Basics
Cumulative in content
Each level of the rubric represents an increase in the quality, intensity, consistency, breadth, depth, and complexity of practice
Effectiveness marked by the addition of practices that improve the overall performance of the specialized service professional and drives to student/other stakeholder outcomes
Standards based
Outlines the practices that you must meet to be at standard

17. Rubric Structure and Rating Level Focus
The focus of the Basic rating is on the foundational elements of service provision. The SSP rated as Basic is typically performing at a foundational level and does not meet state quality standards. Every SSP is expected to perform Basic professional practices in their day-to-day work.
The focus of Partially Proficient
and Proficient levels is what SSP do on a day-to-day basis to achieve state performance standards and assure that students are achieving at expected levels.
The focus of Accomplished and
Exemplary ratings shifts to the
outcomes of the SSP’s practices, including expectations for staff, students, parents and community members, as a result of practices exhibited under Basic, Partially Proficient and Proficient rating levels.

18. Professional Practices
SSP Quality Standards
Performance
Rating Levels
Elements of the Standard
Elements of the Standard
Professional Practices

19. Scoring the Rubric
Determining the professional practices rating is a three-step process that involves rating the individual elements and standards and using those to determine the overall rating on professional practices.
Rating the Elements
Rating the Standards
Determining the Overall Professional Practices Rating

20. Understanding the Scoring “Business” Rule
Starting in the Basic rating level and moving across, look for the first unchecked
professional practice. Move one column back to identify the rating for the element.

21. Determining the Element Rating
Starting in the Basic rating level and moving across, look for the first unchecked
professional practice. Move one column back to identify the rating for the element.

22. Determining the Element Rating
Starting in the Basic rating level and moving across, look for the first unchecked
professional practice. Move one column back to identify the rating for the element.

23. Rubric Rating Level Points
Standard
Basic
Partially Proficient
Proficient
Accomplished
Exemplary
Element
Professional Practices
SSP’s performance on professional practices is significantly below the state quality standard. 1
SSP’s performance on professional practices is below the state quality standard. 2
SSP’s performance on professional practices meets state quality standard. 3
SSP’s performance on professional practices exceeds state quality standard. 4
SSP’s performance on professional practices significantly exceeds state quality standard.

24. 12/2/2018 3 2 1 2 2 10
Proficient

25. Determining Overall Professional Practice Rating
12/2/2018
Determining Overall Professional Practice Rating
First, determine each Standard’s contribution to the overall professional practice rating by using the following formula:
(Total Pts. Earned for Std.) X (Std. Weight X No. of Stds.)
(Number of Elements Associated with Standard)
Example:
10 X (.20 X 5) 5
= 2
Once you’ve completed this step for every Standard, you are then able to determine the overall professional practice rating.

26. Determining Overall Professional Practice Rating
12/2/2018

27. 12/2/2018
Determining Measures of Student Outcomes for Specialized Service Professionals

28. Framework for System to Evaluate Specialized Service Professionals
Colorado Department of Education
Framework for System to Evaluate Specialized Service Professionals
Definition of Specialized Service Professional Effectiveness
Quality Standards
I. Professional Expertise
II. Learning Environment
III. High Quality Delivery
IV. Reflect on Practice
V. Leadership
VI. Student Outcomes
50% Professional Practice Standards % Measures of Student Outcomes
Observations of Professional Practice
Expert Input
Other Measures Aligned with CDE Guidance
Weighting: How Much Does Each Standard Count Towards Overall Performance?
Match of Measures of Student Learning to Assigned Duties
Weighting:
State Scoring Framework: How Do Measures of Quality Standards
Result in a Determination of Individual Performance?
Effectiveness Ratings
Ineffective Partially Effective Effective Highly Effective
Appeals Process*
Applies when professionals are not at-will employees

29. Guardrails for Evaluation System Building
12/2/2018
Guardrails for Evaluation System Building
Requirement language from State Board Rule:
At least fifty percent of the evaluation shall be based on at least two measures of student outcomes, which measures shall be aligned with the role and duties and the individual SSP being evaluated

30. Guardrails for Evaluation System Building
12/2/2018
Guardrails for Evaluation System Building
Requirement language from State Board Rule:
Data used in evaluating SSPs shall be collected from the sites, or a representative sample of the sites, at which the SSP provides services.

31. 12/2/2018
How Do We Get There?
Individual SSP Decisions
Individual SSP and evaluator will make decisions unique to their professional responsibilities. Weights and measures will vary for every SSP in the district/BOCES.
What process will work best for your school, district or BOCES?
There is no right or wrong answer to this question! What process will work best given your context, what is feasible, meaningful and valued?
SSP Category/Group Decisions
Each SSP category will make general decisions about weights of measures, however actual outcome measures may differ. Each SSP category may have different percentages.
All SSPs Decisions
All SSPs in the district/BOCES will have the same weights or measures included in their evaluation. The weights will remain consistent, however the measures will vary based on the SSP.

32. Steps for Identifying and Determining Student Outcome Measures
12/2/2018
Steps for Identifying and Determining Student Outcome Measures
Step 1: Determine SSP role and responsibilities
Step 2: With input from SSP, select and preliminarily weight appropriate outcome measures
Step 3: Set criteria for each rating category
Step 4: Score results from multiple student outcomes at the end of the year
Step 5: Combine weighted scores into a “student outcomes” rating
Decision Framework

33. Student Learning Objective Process: Tools and Resources
12/2/2018
Student Learning Objective Process: Tools and Resources
You can use the Excel Tool to record the targets and scales you have established.

34. Student Learning Objective Process
12/2/2018
Student Learning Objective Process
What is the Student Learning Objective Process?
Enables specialized service professionals to establish outcomes for individual or groups of students, monitor students’ progress toward those outcomes and evaluate the degree to which students achieve those outcomes using relevant, meaningful measures
In evaluation systems, this process serves as a method of measuring student outcomes

35. Student Learning Objective Process: Strategies for Target Setting
12/2/2018
Student Learning Objective Process: Strategies for Target Setting
Possible strategies to consider when setting targets for students:
Proficiency data Targets – students meeting grade level expectations
Examples: a % of students will pass an exam; a % of students will achieve a specific score or better on a measure
Growth data Targets – students growing over the course of instruction or service delivery
Examples: a % of students will progress one level; a % of students will make projected growth or better
Averaging data Targets – students’ average score on a measure
Examples: Students will answer, on average, 80% of the questions correctly; students will achieve a 75% attendance rate

36. Student Learning Objective Process: Strategies for Target Setting
12/2/2018
Student Learning Objective Process: Strategies for Target Setting
Possible strategies to consider when determining how students will be captured in target setting:
Individual – SSP sets individual goals for students and then determines how many students met their individual goals
Whole Class –SSP sets a goal that would be applied to all students in a class
Subgroup of Students –SSP sets a goal for a group(s) of students with similar baseline data levels
Case Load – SSP sets a goal for the specific students he or she supports
Some of these strategies may work better with some SSP assignments. Can you think of which SSP assignments might work best with which strategy based on your local context?

37. Student Learning Objective Process: Strategies for Scale Setting
12/2/2018
Student Learning Objective Process: Strategies for Scale Setting
SSPs set ranges (how many students are expected to meet the set target) across 4 rating levels: more than expected, expected, less than expected, and much less than expected
SSPs collaboratively set ranges with their evaluator
Ranges will vary from year to year and possibly class to class or case load to case load as baseline data will vary due to student beginning points
Collaboratively setting ranges will allow SSPs and evaluators to calibrate about expected student outcomes

38. Contact Us EE Leadership Amy Skinner: Communications
Katie Lams: Communications
Britt Wilkenfeld: Data Fellow
EE Leadership
Katy Anthes: Executive Director
Toby King: Director Evaluation and Support
Jean Williams: Evaluation Design Specialist
Tricia Majors: Project Mgr.
Disclaimer: The contents of this PowerPoint were developed under a grant from the Department of Education. However, those contents do not necessarily represent the policy of the Department of Education, and you should not assume endorsement by the Federal Government.

39. Contact Us Implementation Support and Development Dee Blecha
12/2/2018
Contact Us
Implementation Support and Development
Dee Blecha
Mary Bivens
Courtney Cabrera
Paul Jebe
Sed Keller
Dawn Pare
Joslyn Robich
Chris Vance