1. 2010-2011 Holistic Rating Training Requirements Texas Education Agency Student Assessment Division ©2010 TEXAS EDUCATION AGENCY. ALL RIGHTS RESERVED.

2. Topics  Rater credentials and responsibilities  Overview of holistic rating training requirements and process

3. Rater Credentials Each teacher selected to rate an ELL must 1)have the student in class 2)be knowledgeable about the student’s ability to use English in instructional and informal settings 3)hold valid education credentials such as a teacher certificate or permit 4)be appropriately trained, as required by TEA

4. TELPAS Rater Responsibilities  A student’s TELPAS rater is the teacher designated by the district as the official rater of the student’s English language proficiency.  The student’s rater must rate the student in all domains for which the student is eligible. A student is not permitted to have one rater for some domains and another rater for other domains.

5. TELPAS Rater Training  Districts and campuses should determine in the fall who their TELPAS raters will be in the spring.  A training flowchart is provided on the next slide.

7. Two Types of Training  Online basic training course This course is for new raters. It provides instruction on using the PLD rubrics and gives raters practice rating students in each language domain. There are separate courses for K–1 and 2–12.  Online calibration This is for all raters. It ensures that all raters are prepared to use the PLDs to accurately and consistently assess students for the spring TELPAS administration.

8. K–1 NewK–1 Returning Individuals who have not completed K–1 holistic rating training requirements Individuals who completed K–1 holistic rating training requirements in a previous year New and Returning Rater Definitions for K–1 and 2–12 2–12 New2–12 Returning Individuals who have not completed 2–12 holistic rating training requirements Individuals who completed 2–12 holistic rating training requirements in a previous year 8 These definitions will be in the coordinator and rater manuals and on the training webpage. Raters will be informed that districts may sometimes require a returning rater to complete new-rater training and to consult with their testing coordinator if they need clarification.

9. Why is calibration necessary?  Calibration helps ensure that raters have adequate training, including ample practice and feedback, before they assess their students in the spring.  Calibration ensures that raters clear their heads and consider only the elements of student performance included in the PLDs.  Calibration supports assessment validity and reliability and is an important part of holistically scored assessment processes.

10. Is calibration a test? No, it is a training method that ensures that raters have enough guidance, practice, and support to assess students consistently and accurately.

11. Will raters be able to refer to any resources during calibration activities? Yes, raters should use their TELPAS rubrics (PLDs) and refer to, as needed, information from the:  online basic training course  holistic rating PowerPoints produced by TEA  TELPAS Manual for Raters and Test Administrators  Educator Guide to TELPAS

12. What happens if a rater is unsuccessful in calibrating? Two outcomes are possible for individuals who complete the calibration activities but are not successful:  The district may choose not to assign the individual to be a TELPAS rater.  If the individual is needed to serve as a rater, the district must implement rater support procedures to ensure that the rater’s students are evaluated consistent with the rating rubrics. Individuals are not authorized to serve as raters unless they complete the required training components.

13. How many students must be rated successfully? To be successful, raters need to rate students in their assigned grade cluster with at least 70% accuracy.

14. 14 Recap of Calibration Process  3 sets of 10 students  Those who calibrate on set 1 are done  Those who don’t do set 2  Supplemental support after set 2  Third and final set

15. Grade Clusters of Training and Calibration  Raters must know their assigned grade cluster to select the appropriate online training.  Raters should consult with their campus testing coordinator if they are unsure of their assigned cluster. Grade Clusters Grades K–1 Grade 2 Grades 3–5 Grades 6–8 Grades 9–12

16. 16 When and where do raters take the online training?  Depending on campus arrangements, raters may either complete training and calibration during school hours, after school, or on weekends.  The online training system allows raters to access the training from any computer that meets the minimum system requirements.  Campuses inform raters of the details during annual administration procedures training.

17. Other Things to Know About Calibration  Raters have the option of rating a few warm-up writing collections before starting the calibration activities.  Raters affirm online that they will keep the contents of the calibration sets secure and confidential.  Calibration activities are taken from a bank and randomized. Trainees will rate different sets of students.

18.  Raters can work at their own pace, go back and review students, and change ratings as they work.  Raters can exit and return later to finish. They click a “submit” button when they are finished with a set.  After completing a calibration set, raters immediately see results. Results show the rating assigned by the rater and the correct rating.  Raters see annotations explaining the ratings. Raters should use the annotations to go back and review any incorrectly rated students. Other Things to Know About Calibration

19.  Raters who take the basic training course get a certificate from the online TrainingCenter after completing the course components.  Raters will receive a certificate of successful calibration when they calibrate. Course and Calibration Certificates

20. TELPAS Administration Procedures Training  As part of annual spring TELPAS administration procedures training, holistic rating training requirements are reviewed with raters, as well as information about how to access the online training and calibration components.  In addition, raters receive training on assessment procedures such as how to assemble writing collections, how to record students’ proficiency ratings, etc.