Evaluating the “Caseload” Educators Laura Goe, Ph.D. Research Scientist, ETS, and Principal Investigator for the National Comprehensive Center for Teacher Quality Connecticut Performance Evaluation Advisory Committee Sub-committee on Pupil Services Evaluation Webinar CT  April 11,

2 Laura Goe, Ph.D. Former teacher in rural & urban schools  Special education (7 th & 8 th grade, Tunica, MS)  Language arts (7 th grade, Memphis, TN) Graduate of UC Berkeley’s Policy, Organizations, Measurement & Evaluation doctoral program Principal Investigator for the National Comprehensive Center for Teacher Quality Research Scientist in the Performance Research Group at ETS 2

3 The National Comprehensive Center for Teacher Quality A federally-funded partnership whose mission is to help states carry out the teacher quality mandates of ESEA Vanderbilt University Learning Point Associates, an affiliate of American Institutes for Research Educational Testing Service

4 Today’s presentation available online To download a copy of this presentation or look at it on your iPad, smart phone or laptop now, go to  Go to Publications and Presentations page  Today’s presentation is at the bottom of the page 4

5 The goal of teacher evaluation The ultimate goal of all teacher evaluation should be… TO IMPROVE TEACHING AND LEARNING

6 Questions to be discussed Who are the “caseload” educators (CLEs)?  Definition(s)  Categories - Direct impact on learning - Indirect impact on learning How do they contribute to student learning outcomes? How should they be evaluated?  Standards  Measures  Evaluators

7 The “caseload” educators For nurses, counselors, librarians and other professionals who do not have their own classroom, what counts is the “caseload”  May be all the students in the school  May be a specific set of students  May be other teachers  May be all of the above!

8 What is a caseload? CLEs usually have multiple caseloads  Individual students  Specific groups of students - Chronically absent students - Students with IEPs - Students with health risk (diabetes, allergies, etc.) - Students referred for speech testing  All students in the school  Teachers and other school professionals  Parents

9 Goe, Bell, & Little (2008) definition of teacher effectiveness 1.Have high expectations for all students and help students learn, as measured by value-added or alternative measures. 2.Contribute to positive academic, attitudinal, and social outcomes for students, such as regular attendance, on-time promotion to the next grade, on-time graduation, self-efficacy, and cooperative behavior. 3.Use diverse resources to plan and structure engaging learning opportunities; monitor student progress formatively, adapting instruction as needed; and evaluate learning using multiple sources of evidence. 4.Contribute to the development of classrooms and schools that value diversity and civic-mindedness. 5.Collaborate with other teachers, administrators, parents, and education professionals to ensure student success, particularly the success of students with special needs and those at high risk for failure.

10 Delaware Definition of Specialist (2011) The seven specialist categories are school counselors, instructional support specialists, library media specialists, school psychologists, speech pathologists, school nurses, student support specialists, and therapeutic services specialists (pg. 2).

11 Massachusetts Teacher Association definition (date?) MTA defines Caseload Educator as : An educator who serves individual or small groups of students through consultation with the regular classroom teacher – for example, special education teacher, guidance counselor or speech and language pathologist (pg. 32). From: Reinventing Educator Evaluation: Connecting Professional Practice with Student Learning.

12 Are special education teachers CLEs? In Massachusetts, yes In Delaware, no In my view, CLEs maximize opportunities to learn rather than teach in a content area  Do not have a “classroom” of their own where they provide content instruction to the same group of student each day  Do have multiple caseloads, not limited to a specific group of students with IEPs

13 Direct or indirect impact on student learning growth Direct impact on student learning would apply to those educators who actually teach specific subject matter content to students Indirect impact on student learning would apply to those educators who provide supports, services, and conditions that maximize students’ opportunities to successfully learn subject matter content

14 Examples of indirect contributions to student learning growth Conducting staff development to inform teachers about signs of potential child abuse and reporting procedures Assessing needs of homebound students to ensure they have as much access as possible to classroom interactions Informing parents about technology and resources available in the school library and ways to encourage children to take maximum advantage of them

15 CT’s evaluation of PSPs (2010) Adopt discipline-specific criteria for evaluating support services specialists, using the competencies and indicators developed by the CSDE (pg. 4) From Connecticut State Board of Education Position Statement on Student Support Services

16 Principles for CT’s Pupil Services Professionals (1999) Pupil services are an integral component of quality education programs for all students. � Pupil services promote optimal development, health and learning for all students. � Pupil services are organized and delivered so as to help teachers, parents and other members of the school community provide optimum teaching and learning experiences for students. Pupil services are comprehensive in scope, with emphasis on prevention and early intervention. � Collaboration with students, parents, school personnel and community providers is key to the success of pupil services programs. � Decision-making and service provision by pupil service professionals are guided by ethical principles. From Developing Quality Programs for Pupil Services: A Self-Evaluative Guide

17 CT SBE Position Statement on Student Support Services (2010) Disciplines providing support services include school counseling, school nursing, school psychology, school social work, speech-language pathology and audiology. These services assist the student population, parents and the entire school community in establishing a full range of prevention and intervention systems that promote healthy development, provide early intervention to address problems as soon after onset as possible and assist with chronic and severe problems (pg. 1) From Connecticut State Board of Education Position Statement on Student Support Services

18 Measures/processes for evaluating CLEs (MTA) Formal Observations – at least one instructional block or one student encounter for caseload educators (pg. 18) Artifacts for Caseload educator: IEPs, 504 Plans, non-confidential reports, student work if applicable (pg. 19) Those observing and/or judging caseload educators…must have caseload experience. (pg. 29) Massachusetts Teacher Association. Reinventing Educator Evaluation: Connecting Professional Practice with Student Learning

19 Logic models and student learning growth (1) A logic model helps to make a case for how CLEs impact learning growth For example, a logic model might show that a school nurse’s efforts to work with doctors, parents and teachers are helping to maximize chronically ill children’s opportunity to attend school and/or receive high-quality instruction when they are unable to be in the classroom  Maximizing students’ opportunities to learn should result in learning gains for most students

20 Logic models and student learning growth (2) A good logic model will help determine which aspects of a CLE’s performance may need to be evaluated in order to document the CLE’s contribution to student learning growth (through maximizing opportunities to learn) Schoolwide learning gains may also be included (required in Delaware), but many CLEs do not provide direct instruction so it is not possible to directly attribute student gains to their efforts

21 Results inform professional growth opportunities Are evaluation results discussed with individual CLEs? Do CLEs collaborate with managers to develop a plan for improvement and/or professional growth?  All CLEs (even high-scoring ones) have areas where they can grow and learn Are effective CLEs provided with opportunities to develop their leadership potential? Are struggling CLEs provided with coaches and given opportunities to observe/be observed (with feedback)?

22 Final thoughts The limitations:  Changing the culture of evaluation is hard work  There are few example of how to evaluate CLEs  Research has not yet provided guidance on how to evaluate CLEs for effectiveness The opportunities:  Evidence collected during evaluation can be used to trigger support for struggling CLEs and acknowledge effective ones  Multiple sources of evidence can provide powerful information to improve CLEs’ ability to maximize opportunities to learn for students they serve  Collecting evidence as part of a comprehensive evaluation system is more valid than relying on “judgment” and provides better information for CLEs to improve practice

23 Resources and links Connecticut State Board of Education. (2010). Position Statement on Student Support Services. Connecticut State Department of Education. (1999). Developing Quality Programs for Pupil Services: A Self-Evaluative Guide. Delaware Department of Education. (2011). Delaware Performance Appraisal System: DPAS II Guide for Specialists pdf Massachusetts Teacher Association. Reinventing Educator Evaluation: Connecting Professional Practice with Student Learning. (date?). df 23

24 Questions?

25 Laura Goe, Ph.D National Comprehensive Center for Teacher Quality 1000 Thomas Jefferson Street, NW Washington, D.C