2015 TELPAS Holistic Training For CTCs Assessment, Research & Evaluation Department January 27, 2015

Disclaimer This training does NOT take the place of reading the appropriate manuals.

General Information

2014 TELPAS Changes New standards were set for TELPAS reading in 2014 Domain weights shifted in composite score calculations. Reading is 50% Writing is 30% Listening and Speaking are each weighted 10%

Years in US Schools Data Collection Years in US Schools Data Collection Changes (DCCM pg. T-30) Beginning with calculations made and reported in 2014, students must be enrolled for 60 consecutive school days in a school year for that year to count as a year in US schools. Enrolled for 60 consecutive school days = 1 year in the US Count restarts if student WD’s prior to reaching 60 days and Does not enroll in another US school for 10+ consecutive school days If enrolled within the final 60 school days of a school year, then next year will still be 1st year in US.

Years in US Schools Data Collection Important Reminders Campuses cannot change the value for years in U.S. schools submitted in a previous school year. Campuses will use the data previously submitted to generate data that will be submitted this spring. The number of years in U.S. schools on record for a student cannot decrease. The value will either remain the same or increase by 1.

TELPAS Assessment Components Grades K-1 Grades 2-12 Holistically rated observational assessments Listening Speaking Reading Writing Multiple choice online tests in grades 2-12 Reading Holistically rated Writing collections Holistically rated observational assessments Listening Speaking

Eligibility Requirements All ELL’s in grades Kinder – 12th are required to participate. including PD’s In rare circumstances, it may be necessary for the ARD & LPAC committee to determine a student should not be assessed in any given domain. Participation must be considered on a domain-by-domain basis and reasons must be supported and documented in the IEP Send the DTC a TELPAS Move Form beginning February 18, 2015. Handout #1

Eligibility Requirements Handout #1 Date:______ An ELL from another school district, state, or country who enrolls on or after the first day of the TELPAS testing window will not be assed by the receiving district in the HOLISTICALLY rated domains All newly enrolled ELL’s in grades 2-12 who enroll on or after the first day of the testing window, are ONLY required to take the TELPAS reading test; however, TELPAS coordinators must verify with DTC that the student hasn’t already completed testing elsewhere Date:______

Test Security and Confidentiality Requirements 2015 TELPAS Rater Manual – pg. 3

Test Security Maintaining the confidentiality of the TELPAS program involves protecting the contents of all online assessments and student performance documentation. This requires compliance with, but is not limited to, the following guidelines Personnel meet the requirements, have been trained and have signed the appropriate oath to have access to the assessment materials and information Campuses implement the controls necessary to ensure proper storage of secure materials throughout the test administration Campuses place confidential documentation (rating rosters) in a limited-access locked storage

Test Security Campuses must account for all materials before during after each test administration. Campuses ensure that: Only personnel who meet the requirements participate They have been trained They have signed an oath Materials are properly stored (locked storage) Rating rosters are secured when not in use

Confidentiality Requirements Testing personnel undergo training and sign the oath affirming they understand their obligations concerning the security and confidentiality of the TELPAS program Testing personnel administer the tests in strict accordance with the instructions Testing personnel do not view, reveal or discuss the contents of an assessment before, during, or after a test administration

Penalties for Prohibited Conduct Any person who violates, assists in the violation of, or solicits another to violate or assist in the violation of test security or confidentiality, as well as any person who fails to report such violation is subject to the following penalties Placement of restrictions on renewal or holding of educator certificate Issuance of an inscribed or non-inscribed reprimand Suspension of a Texas educator certificate for a set term Revocation or cancelation of an educator certificate without opportunity for reapplication for a set term or permanently

Penalties for Prohibited Conduct (continued) Release or disclosure of confidential test content could result in criminal prosecution under TEC §39.0303, Section 52.352 of the Texas Government Code, and Section 37.10 of the Texas Penal Code. Further, 19 TAC §249.15 stipulates that the State Board for Educator Certification may take any of the above actions based on satisfactory evidence that an educator has failed to cooperate with TEA in an investigation. Additionally, irregularities resulting in a breach of test security or confidentiality may result in the invalidation of students’ assessments.

Security Oaths and Confidentiality Statements All personnel must meet the eligibility requirements and sign a security oath Any person who has more than one testing role must receive appropriate training and sign a security oath for EACH role Each oath must be read and completed AFTER the training and BEFORE handling or viewing any secure materials or confidential information Oaths must be kept on file for at least five years

Testing Irregularities Serious Examples These constitute severe violations of test security and/or confidentiality and can result in the individual(s) responsible being referred to the TEA Educator Standards and Certification Legal Division for consideration of disciplinary action. Falsifying TELPAS ratings or writing samples Assisting students directly or indirectly Tampering with student responses Directing a student to rework or change answers

Testing Irregularities NOTE!!!! It is a violation of state assessment procedures and a serious testing irregularity to record, discuss, or share answers to the rating practice and calibration activities. NEW: After completion of calibration activities, raters must destroy all notes taken about specific student profiles.

Testing Irregularities Procedural Examples: Less severe and more common irregularities that are typically the result of minor deviations in testing procedures. Eligibility error IEP implementation issues Improper accounting of secure materials Monitoring error Procedural error

Testing Irregularities (cont.) Procedural Irregularities (examples) Eligibility error Student not rated in one or more domains Student not administered the reading test IEP Implementation error SPED student was/was nor provided an accommodation

Testing Irregularities (cont.) Procedural Irregularities (examples) Improper Account of Secure Materials error Teacher misplaced a writing collection Monitoring error TA left room unattended when online tests were open Student was not prevented from using cell phone Procedural error TA who wasn’t trained rated students A writing collection was not submitted according to the assembly criteria TA failed to provide student the correct authorization for online testing

Reporting Test Irregularities Each person is responsible for reporting ANY violation or suspected violation Campus staff should notify their CTC or DTC if they witness an irregularity or suspect one has occurred DTC will contact TEA to report ANY incidents as soon as DTC is made aware of the situation Contact DTC if unsure about whether an irregularity has occurred

Campus Training Requirements

Campus Training Requirements Responsibilities of TELPAS Coordinators You are expected to: Train staff on holistic rating training requirements. Ensure raters understand the importance of being properly trained on the holistic rating process Ensure new and returning raters understand which training to take and for which grade clusters

Campus Training Requirements Responsibilities of TELPAS Coordinators You are expected to: Train staff on how to access the online training and calibration components. Monitor that all staff complete their training requirements and receive supplemental training support if needed Train all staff (as applicable) on assessment procedures such as how to assemble writing collections, how to record students’ proficiency ratings, etc.

Prepare For Training Coordinators ensure that: All ELL’s are identified and assessed TELPAS Raters, SSP & Verifiers are identified and properly trained Procedures are established to collect & verify the years in US for all ELL’s There are protocols at the campus to ensure the validity and reliability of the rating process Student records are verified Materials are collected and filed as appropriate

TELPAS Roles Holistic Rater

Rater Credentials Each teacher selected to rate must: RATER: Teacher designated to be the official rater of an ELL’s language proficiency Each teacher selected to rate must: have student in class at the time of the spring assessment be knowledgeable about the student’s language ability (instructional and informal) have a valid Texas teaching certificate be appropriately trained rate the student in all eligible domains Districts must implement validity and reliability checks during the testing window (2nd rater). 28

New or Returning Rater? TEA requires that raters who have not rated during the last three years complete the BASIC training.

Holistic Rating Process Spring TELPAS Administration Procedures Training As a key part of this training, information from the TELPAS Rater Manual is reviewed to prepare raters to proceed with online holistic rating training New Raters Returning Raters Online Basic Training Course *Set 2 required only if not successful on Set 1 Online Calibration (Sets 1 and 2*) Online Calibration (Sets 1 and 2*) If not calibrated: Supplemental Holistic Rating Training Final Online Calibration (Set 3)

Online Training Components Online BASIC TRAINING Course Required for new raters and raters who have not completed rater training within the last three school years. Gives raters practice rating students in each language domains. Separated in two courses: (approximately 4-5 hours) Basic Training for Grades Kinder-1st – covers the four language domains Basic Training for Grades 2nd – 12th – covers listening, speaking, and writing Raters who have completed K–1 training but not 2–12 training will be new raters if they switch to 2–12 training, and vice versa.

Online Training Components Online CALIBRATION Course Required for ALL 2015 raters. Each calibration set contains 10 students to rate and all holistically assessed language domains are represented in each set. (approximately 1 hour ) Raters taking the grades Kinder-1st course – will rate listening, speaking, reading and writing Raters taking the Grades 2nd – 12th course – will rate in listening, speaking and writing A rater needs to rate at least 70 percent of the students correctly to demonstrate sufficient calibration.

Training & Calibration Activities When and where does it take place? Depending on campus arrangements, raters may either complete training and calibration during school hours after school or on weekends Raters may access the training from any computer that meets the minimum system requirements. Campuses must inform staff of the expectations during the Holistic campus training.

Training & Calibration Grade Clusters Grades K-1 Grade 2 Grades 3-5 Grades 6-8 Grades 9-12 Raters must know their assigned grade cluster to select the appropriate online training. Raters should consult with their campus testing coordinator if they are unsure of their assigned cluster.

Grade Cluster Organization Online Training Process Basic Training Course Kinder - 1 Basic Training Course 2 – 12 Calibration Clusters Grade 2 Grades 3 – 5 Grades 6 – 8 Grades 9 – 12 Calibration Clusters Kinder - 1

Raters with Students in Multiple Grade Clusters Including students in Grades 2-12 Raters are required to train and calibrate in the cluster in which they have the most ELLs. As a best practice, these raters should also review the online basic training course practice activities in the additional cluster(s). Example: A grade 2–5 ESL teacher has most of her ELLs in grade 3. She must complete training and calibration in grades 3–5. As a best practice, she should review the online practice activities for grade 2 to check her readiness to apply the rubrics appropriately.

Raters with Students in Multiple Grade Clusters Including students in Grades K-1 These raters must complete training and calibration for K–1 and at least one other cluster in 2–12. Raters with more than one additional cluster should train in the cluster in which they have the most ELLs. As a best practice, these raters should also review online basic training course practice activities in the additional cluster(s). Example: A grade 1–3 ESL teacher has most of his ELLs in grade 2. He must complete training and calibration in grades K–1 and 2. As a best practice, he should review the online practice activities for grades 3–5 to check his readiness to apply the rubrics appropriately for his 3rd grade students.

Preparing for Calibration Sets Handout #2 New raters must complete the online basic training course Before beginning calibration. All raters have the option to review the online basic training course (which includes practice rating activities) Before beginning calibration. Basic Training must be completed by February 15, 2015!!!

Online Calibration Activities Raters complete only as many sets as it takes to calibrate Many raters will be able to calibrate on the first set of activities and many others will be able to calibrate by the end of the second set. With supplemental support, very few raters should have difficulty calibrating by the end of the third and final set IMPORTANT NOTE Individuals are not authorized by TEA to serve as raters unless they complete the state-required training and calibration activities

Resources during calibration Can raters use them? Yes, raters should use their rating rubrics (PLD’s) found on page 33-37 of the Rater Manual and refer to, as needed, any other information from the: online basic training course handouts holistic rating PowerPoints by TEA TELPAS Rater Manual Educator Guide to TELPAS

Other Things to Know About Calibration Raters affirm online that they will keep the contents of the calibration sets secure and confidential. Calibration activities are taken from a bank and randomized. Trainees will rate different sets of students. Raters can work at their own pace, go back and review students, and change ratings as they work.

Other Things to Know About Calibration (cont.) Raters can exit and return later to finish. They click a “submit” button when they are finished with a set. After completing a calibration set, raters immediately see results. Results show both the rating assigned by the rater as well as the correct rating. Raters see annotations explaining the correct ratings. Raters should use the annotations to go back and review any incorrectly rated students.

Tools for TELPAS Coordinators TELPAS Coordinators can: modify the last name of a user send a user password reset email modify the email address of a user retrieve an email address associated with a user’s TrainingCenter account retrieve a username associated with a user’s TrainingCenter account see account status (active/deactivated) for all users.

TELPAS Additional Roles Supplemental Support Providers Assembly & Verifiers

SSP Criteria A Supplemental Support Provider must: have been a 2013-2014 fully trained rater in the applicable K-1 or 2-12 span (last school year) complete the SSP training in the Texas Training Center (current school year) and calibrate successfully on either set 1 or set 2 in 2015 (current school year)

SSP Training The SSP training is delivered through a Web-based presentation accessible from the SSP resources section of the Texas TrainingCenter Separate trainings for K–1, 2–5, and 6–12 Training will be available beginning January 26 Both new and returning supplemental support providers need to complete this training Campuses must train at least one supplemental support provider Reviewing the number of raters unable to calibrate by end of calibration set 2 last year may help anticipate this year’s needs

Providing Supplemental Support Handout #2 CTC’s will notify the DTC who the SSP is by the designated district deadline (February 22, 2015) Raters not successful after sets 1 and 2 must receive supplemental training The rater will meet with a campus-appointed supplemental support provider (SSP) (no later than February 26, 2015) After the rater has received supplemental training, raters will be given an access code to proceed with the third and final calibration set (no later than March 6)

Models for Providing Support Individual or group sessions Raters attend a one-on-one or group session with a supplemental support provider before completing their third and final calibration set. Brief meeting followed by additional consultation at rater’s request Rater meets briefly with the supplemental support provider for instructions on resources to review. Rater schedules time with the support provider to get clarification, if needed, before completing the third and final calibration set. Note: The models are described in detail in the recorded Web-based training.

Assemble and Verify Grades K-12 Student Writing Collections An online training course on writing collection assembly and verification is available on the Texas TrainingCenter website Writing assigned on or after February 16, 2014 may be considered for the writing collection (writing assigned before February 16 is not eligible for inclusion) Writing samples may continue to be gathered until March 26th, the date raters are required to turn in students’ rating to the campus coordinator All writing should reflect a student’s current proficiency level

Assemble and Verify Grades K-12 Writing Collections Type 1: Basic descriptive writing on a personal/familiar topic Type 2: Writing about a familiar process Type 3: Narrative writing about a past event Type 4: Personal narratives and reflective pieces Type 5: Expository and other extended writing on a topic from language arts What Not to Include in a Collection (p. 22) Verifying the Writing Collection Components (p. 23)

Monitoring Course Completions and Performance on Calibration Activities

Training Certificates Training certificates are generated for raters who complete training courses and calibration activities. A certificate of completion is issued when a rater completes the online Basic Training or Assembly & Verifying course. A certificate of successful calibration is issued when a rater demonstrates sufficient calibration (70 percent or better) on a calibration set.

https://texas.pearson.desire2learn.com/ Available Reports https://texas.pearson.desire2learn.com/ For online training and calibration TELPAS Returning Rater Planning Roster Uses records starting from the 2008–2009 school years to generate a list of previously trained raters (Basic Training) TELPAS Confidential Course Completion Roster Lists online basic training course completion and in-progress status and performance by user for the current year TELPAS At-a-Glance Training & Calibration Report Provides a comprehensive list showing basic training course and calibration completions for all registered users for the current year (includes time spent in each calibration set) TELPAS Confidential Calibration Summary Report Provides calibration summary information by grade cluster (K–1, 2, 3–5, 6–8, 9–12) and for grade clusters combined

TELPAS Returning Rater Planning Roster

TELPAS Confidential Course Completion Roster

TELPAS At-a-Glance Training & Calibration Report

TELPAS Confidential Calibration Summary Report

Important Notes About Reports In the Texas TrainingCenter… TELPAS Coordinators are able to access reports to monitor online training and calibration. The reports are updated nightly. In order for users to appear in the correct reports, their location (region, district, campus) must be up to date in the My Info section. Users who have registered in the Training Center and have started but not completed a calibration set are not included in the summary reports. Only completed sets are presented in the reports. However, users who have started a training course will appear in the course roster as being “in progress.”

What’s next? For TELPAS Coordinators

Holistic Process Where do I start? Plan your trainings Handout #3 Where do I start? Plan your trainings With your principal, determine how your campus will conduct basic training and calibration activities Verify trainings have been completed (obtain certificates from personnel) Download rating rosters, coversheets, etc from ARE website Be sure to collect and have the correct Years in US schools of all students (ELLs)

TELPAS Student Rating Roster

Schedule TELPAS Student Tutorials As in the past, tutorials are separate from tests Recommended for all students taking TELPAS Several different test item formats Since no sample items with test, practice with item formats and online interface is useful Tutorials available at http://www.TexasAssessment.com/TELPAS-tutorials

Add Dates to Your Calendar

e-Schools extraction of TELPAS registration file Next Training TELPAS– Online Testing Training Friday, February 20, 2015 1:00 – 4:00 CAB Cafeteria *Bring YEARS in US!!! e-Schools extraction of TELPAS registration file

Questions and Answers…