1. 2015 TELPAS Holistic Training For CTCs
Assessment, Research & Evaluation Department
January 27, 2015

2. Disclaimer
This training does NOT take the place of reading the appropriate manuals.

3. General Information

4. 2014 TELPAS Changes New standards were set for TELPAS reading in 2014
Domain weights shifted in composite score calculations.
Reading is 50%
Writing is 30%
Listening and Speaking are each weighted 10%

5. Years in US Schools Data Collection
Years in US Schools Data Collection Changes (DCCM pg. T-30)
Beginning with calculations made and reported in 2014, students must be enrolled for 60 consecutive school days in a school year for that year to count as a year in US schools.
Enrolled for 60 consecutive school days = 1 year in the US
Count restarts if student
WD’s prior to reaching 60 days and
Does not enroll in another US school for 10+ consecutive school days
If enrolled within the final 60 school days of a school year, then next year will still be 1st year in US.

6. Years in US Schools Data Collection
Important Reminders
Campuses cannot change the value for years in U.S. schools submitted in a previous school year.
Campuses will use the data previously submitted to generate data that will be submitted this spring.
The number of years in U.S. schools on record for a student cannot decrease. The value will either remain the same or increase by 1.

7. TELPAS Assessment Components
Grades K-1
Grades 2-12
Holistically rated observational assessments
Listening
Speaking
Reading
Writing
Multiple choice online tests in grades 2-12
Reading
Holistically rated
Writing collections
Holistically rated observational assessments
Listening
Speaking

8. Eligibility Requirements
All ELL’s in grades Kinder – 12th are required to participate.
including PD’s
In rare circumstances, it may be necessary for the ARD & LPAC committee to determine a student should not be assessed in any given domain.
Participation must be considered on a domain-by-domain basis and reasons must be supported and documented in the IEP
Send the DTC a TELPAS Move Form beginning
February 18, 2015.
Handout #1

9. Eligibility Requirements
Handout #1
Date:______
An ELL from another school district, state, or country who enrolls on or after the first day of the TELPAS testing window will not be assed by the receiving district in the HOLISTICALLY rated domains All newly enrolled ELL’s in grades 2-12 who enroll on or after the first day of the testing window, are ONLY required to take the TELPAS reading test; however, TELPAS coordinators must verify with DTC that the student hasn’t already completed testing elsewhere
Date:______

10. Test Security and Confidentiality Requirements
2015 TELPAS Rater Manual – pg. 3

11. Test Security
Maintaining the confidentiality of the TELPAS program involves protecting the contents of all online assessments and student performance documentation. This requires compliance with, but is not limited to, the following guidelines
Personnel meet the requirements, have been trained and have signed the appropriate oath to have access to the assessment materials and information
Campuses implement the controls necessary to ensure proper storage of secure materials throughout the test administration
Campuses place confidential documentation (rating rosters) in a limited-access locked storage

12. Test Security Campuses must account for all materials
before
during
after
each test administration.
Campuses ensure that:
Only personnel who meet the requirements participate
They have been trained
They have signed an oath
Materials are properly stored (locked storage)
Rating rosters are secured when not in use

13. Confidentiality Requirements
Testing personnel undergo training and sign the oath affirming they understand their obligations concerning the security and confidentiality of the TELPAS program
Testing personnel administer the tests in strict accordance with the instructions
Testing personnel do not view, reveal or discuss the contents of an assessment before, during, or after a test administration

14. Penalties for Prohibited Conduct
Any person who violates, assists in the violation of, or solicits another to violate or assist in the violation of test security or confidentiality, as well as any person who fails to report such violation is subject to the following penalties
Placement of restrictions on renewal or holding of educator certificate
Issuance of an inscribed or non-inscribed reprimand
Suspension of a Texas educator certificate for a set term
Revocation or cancelation of an educator certificate without opportunity for reapplication for a set term or permanently

15. Penalties for Prohibited Conduct
(continued)
Release or disclosure of confidential test content could result in criminal prosecution under TEC § , Section of the Texas Government Code, and Section of the Texas Penal Code. Further, 19 TAC § stipulates that the State Board for Educator Certification may take any of the above actions based on satisfactory evidence that an educator has failed to cooperate with TEA in an investigation. Additionally, irregularities resulting in a breach of test security or confidentiality may result in the invalidation of students’ assessments.

16. Security Oaths and Confidentiality Statements
All personnel must meet the eligibility requirements and sign a security oath
Any person who has more than one testing role must receive appropriate training and sign a security oath for EACH role
Each oath must be read and completed AFTER the training and BEFORE handling or viewing any secure materials or confidential information
Oaths must be kept on file for at least five years

17. Testing Irregularities
Serious
Examples
These constitute severe violations of test security and/or confidentiality and can result in the individual(s) responsible being referred to the TEA Educator Standards and Certification Legal Division for consideration of disciplinary action.
Falsifying TELPAS ratings or writing samples
Assisting students directly or indirectly
Tampering with student responses
Directing a student to rework or change answers

18. Testing Irregularities
NOTE!!!!
It is a violation of state assessment procedures and a serious testing irregularity to record, discuss, or share answers to the rating practice and calibration activities.
NEW: After completion of calibration activities, raters must destroy all notes taken about specific student profiles.

19. Testing Irregularities
Procedural
Examples:
Less severe and more common irregularities that are typically the result of minor deviations in testing procedures.
Eligibility error
IEP implementation issues
Improper accounting of secure materials
Monitoring error
Procedural error

20. Testing Irregularities (cont.)
Procedural Irregularities (examples)
Eligibility error
Student not rated in one or more domains
Student not administered the reading test
IEP Implementation error
SPED student was/was nor provided an accommodation

21. Testing Irregularities (cont.)
Procedural Irregularities (examples)
Improper Account of Secure Materials error
Teacher misplaced a writing collection
Monitoring error
TA left room unattended when online tests were open
Student was not prevented from using cell phone
Procedural error
TA who wasn’t trained rated students
A writing collection was not submitted according to the assembly criteria
TA failed to provide student the correct authorization for online testing

22. Reporting Test Irregularities
Each person is responsible for reporting ANY violation or suspected violation
Campus staff should notify their CTC or DTC if they witness an irregularity or suspect one has occurred
DTC will contact TEA to report ANY incidents as soon as DTC is made aware of the situation
Contact DTC if unsure about whether an irregularity has occurred

23. Campus Training Requirements

24. Campus Training Requirements
Responsibilities of TELPAS Coordinators
You are expected to:
Train staff on holistic rating training requirements.
Ensure raters understand the importance of being properly trained on the holistic rating process
Ensure new and returning raters understand which training to take and for which grade clusters

25. Campus Training Requirements
Responsibilities of TELPAS Coordinators
You are expected to:
Train staff on how to access the online training and calibration components.
Monitor that all staff complete their training requirements and receive supplemental training support if needed
Train all staff (as applicable) on assessment procedures such as how to assemble writing collections, how to record students’ proficiency ratings, etc.

26. Prepare For Training Coordinators ensure that:
All ELL’s are identified and assessed
TELPAS Raters, SSP & Verifiers are identified and properly trained
Procedures are established to collect & verify the years in US for all ELL’s
There are protocols at the campus to ensure the validity and reliability of the rating process
Student records are verified
Materials are collected and filed as appropriate

27. TELPAS Roles
Holistic Rater

28. Rater Credentials Each teacher selected to rate must:
RATER: Teacher designated to be the official rater of an ELL’s language proficiency
Each teacher selected to rate must:
have student in class at the time of the spring assessment
be knowledgeable about the student’s language ability (instructional and informal)
have a valid Texas teaching certificate
be appropriately trained
rate the student in all eligible domains
Districts must implement validity and reliability checks during the testing window (2nd rater). 28

29. New or Returning Rater?
TEA requires that raters who have not rated during the last three years complete the BASIC training.

30. Holistic Rating Process
Spring TELPAS Administration Procedures Training
As a key part of this training, information from the TELPAS Rater Manual is reviewed to prepare raters to proceed with online holistic rating training
New Raters
Returning Raters
Online Basic Training Course
*Set 2 required only if not successful on Set 1
Online Calibration
(Sets 1 and 2*)
Online Calibration
(Sets 1 and 2*)
If not calibrated: Supplemental Holistic Rating Training
Final Online Calibration (Set 3)

31. Online Training Components
Online BASIC TRAINING Course
Required for new raters and raters who have not completed rater training within the last three school years.
Gives raters practice rating students in each language domains. Separated in two courses:
(approximately 4-5 hours)
Basic Training for Grades Kinder-1st – covers the four language domains
Basic Training for Grades 2nd – 12th – covers listening, speaking, and writing
Raters who have completed K–1 training but not 2–12 training will be new raters if they switch to 2–12 training, and vice versa.

32. Online Training Components
Online CALIBRATION Course
Required for ALL 2015 raters.
Each calibration set contains 10 students to rate and all holistically assessed language domains are represented in each set.
(approximately 1 hour )
Raters taking the grades Kinder-1st course – will rate listening, speaking, reading and writing
Raters taking the Grades 2nd – 12th course – will rate in listening, speaking and writing
A rater needs to rate at least 70 percent of the students correctly to demonstrate sufficient calibration.

33. Training & Calibration Activities
When and where does it take place?
Depending on campus arrangements, raters may either complete training and calibration
during school hours
after school or
on weekends
Raters may access the training from any computer that meets the minimum system requirements.
Campuses must inform staff of the expectations during the Holistic campus training.

34. Training & Calibration
Grade Clusters
Grades K-1
Grade 2
Grades 3-5
Grades 6-8
Grades 9-12
Raters must know their assigned grade cluster to select the appropriate online training.
Raters should consult with their campus testing coordinator if they are unsure of their assigned cluster.

35. Grade Cluster Organization
Online Training Process
Basic Training Course
Kinder - 1
Basic Training Course
2 – 12
Calibration Clusters
Grade 2
Grades 3 – 5
Grades 6 – 8
Grades 9 – 12
Calibration Clusters
Kinder - 1

36. Raters with Students in Multiple Grade Clusters
Including students in Grades 2-12
Raters are required to train and calibrate in the cluster in which they have the most ELLs.
As a best practice, these raters should also review the online basic training course practice activities in the additional cluster(s).
Example: A grade 2–5 ESL teacher has most of her ELLs in grade 3. She must complete training and calibration in grades 3–5. As a best practice, she should review the online practice activities for grade 2 to check her readiness to apply the rubrics appropriately.

37. Raters with Students in Multiple Grade Clusters
Including students in Grades K-1
These raters must complete training and calibration for K–1 and at least one other cluster in 2–12.
Raters with more than one additional cluster should train in the cluster in which they have the most ELLs.
As a best practice, these raters should also review online basic training course practice activities in the additional cluster(s).
Example: A grade 1–3 ESL teacher has most of his ELLs in grade 2. He must complete training and calibration in grades K–1 and 2. As a best practice, he should review the online practice activities for grades 3–5 to check his readiness to apply the rubrics appropriately for his 3rd grade students.

38. Preparing for Calibration Sets
Handout #2
New raters must complete the online basic training course Before beginning calibration.
All raters have the option to review the online basic training course (which includes practice rating activities) Before beginning calibration.
Basic Training must be completed by February 15, 2015!!!

39. Online Calibration Activities
Raters complete only as many sets as it takes to calibrate
Many raters will be able to calibrate on the first set of activities and many others will be able to calibrate by the end of the second set.
With supplemental support, very few raters should have difficulty calibrating by the end of the third and final set
IMPORTANT NOTE
Individuals are not authorized by TEA to serve as raters unless they complete the state-required training and calibration activities

40. Resources during calibration
Can raters use them?
Yes, raters should use their rating rubrics (PLD’s) found on page of the Rater Manual and refer to, as needed, any other information from the:
online basic training course handouts
holistic rating PowerPoints by TEA
TELPAS Rater Manual
Educator Guide to TELPAS

41. Other Things to Know About Calibration
Raters affirm online that they will keep the contents of the calibration sets secure and confidential.
Calibration activities are taken from a bank and randomized. Trainees will rate different sets of students.
Raters can work at their own pace, go back and review students, and change ratings as they work.

42. Other Things to Know About Calibration (cont.)
Raters can exit and return later to finish. They click a “submit” button when they are finished with a set.
After completing a calibration set, raters immediately see results. Results show both the rating assigned by the rater as well as the correct rating.
Raters see annotations explaining the correct ratings. Raters should use the annotations to go back and review any incorrectly rated students.

43. Tools for TELPAS Coordinators
TELPAS Coordinators can:
modify the last name of a user
send a user password reset
modify the address of a user
retrieve an address associated with a user’s TrainingCenter account
retrieve a username associated with a user’s TrainingCenter account
see account status (active/deactivated) for all users.

44. TELPAS Additional Roles
Supplemental Support Providers
Assembly & Verifiers

45. SSP Criteria A Supplemental Support Provider must:
have been a fully trained rater in the applicable K-1 or 2-12 span (last school year)
complete the SSP training in the Texas Training Center (current school year) and
calibrate successfully on either set 1 or set 2 in 2015 (current school year)

46. SSP Training
The SSP training is delivered through a Web-based presentation accessible from the SSP resources section of the Texas TrainingCenter
Separate trainings for K–1, 2–5, and 6–12
Training will be available beginning January 26
Both new and returning supplemental support providers need to complete this training
Campuses must train at least one supplemental support provider
Reviewing the number of raters unable to calibrate by end of calibration set 2 last year may help anticipate this year’s needs

47. Providing Supplemental Support
Handout #2
CTC’s will notify the DTC who the SSP is by the designated district deadline (February 22, 2015)
Raters not successful after sets 1 and 2 must receive supplemental training
The rater will meet with a campus-appointed supplemental support provider (SSP) (no later than February 26, 2015)
After the rater has received supplemental training, raters will be given an access code to proceed with the third and final calibration set (no later than March 6)

48. Models for Providing Support
Individual or group sessions
Raters attend a one-on-one or group session with a supplemental support provider before completing their third and final calibration set.
Brief meeting followed by additional consultation at rater’s request
Rater meets briefly with the supplemental support provider for instructions on resources to review.
Rater schedules time with the support provider to get clarification, if needed, before completing the third and final calibration set.
Note: The models are described in detail in the recorded Web-based training.

49. Assemble and Verify Grades K-12 Student Writing Collections
An online training course on writing collection assembly and verification is available on the Texas TrainingCenter website
Writing assigned on or after February 16, 2014 may be considered for the writing collection (writing assigned before February 16 is not eligible for inclusion)
Writing samples may continue to be gathered until March 26th, the date raters are required to turn in students’ rating to the campus coordinator
All writing should reflect a student’s current proficiency level

50. Assemble and Verify Grades K-12 Writing Collections
Type 1: Basic descriptive writing on a personal/familiar topic
Type 2: Writing about a familiar process
Type 3: Narrative writing about a past event
Type 4: Personal narratives and reflective pieces
Type 5: Expository and other extended writing on a topic from language arts
What Not to Include in a Collection (p. 22)
Verifying the Writing Collection Components (p. 23)

51. Monitoring Course Completions and Performance on Calibration Activities

52. Training Certificates
Training certificates are generated for raters who complete training courses and calibration activities.
A certificate of completion is issued when a rater completes the online Basic Training or Assembly & Verifying course.
A certificate of successful calibration is issued when a rater demonstrates sufficient calibration (70 percent or better) on a calibration set.

53. https://texas.pearson.desire2learn.com/
Available Reports
For online training and calibration
TELPAS Returning Rater Planning Roster
Uses records starting from the 2008–2009 school years to generate a list of previously trained raters (Basic Training)
TELPAS Confidential Course Completion Roster
Lists online basic training course completion and in-progress status and performance by user for the current year
TELPAS At-a-Glance Training & Calibration Report
Provides a comprehensive list showing basic training course and calibration completions for all registered users for the current year (includes time spent in each calibration set)
TELPAS Confidential Calibration Summary Report
Provides calibration summary information by grade cluster (K–1, 2, 3–5, 6–8, 9–12) and for grade clusters combined

54. TELPAS Returning Rater Planning Roster

55. TELPAS Confidential Course Completion Roster

56. TELPAS At-a-Glance Training & Calibration Report

57. TELPAS Confidential Calibration Summary Report

58. Important Notes About Reports
In the Texas TrainingCenter…
TELPAS Coordinators are able to access reports to monitor online training and calibration.
The reports are updated nightly.
In order for users to appear in the correct reports, their location (region, district, campus) must be up to date in the My Info section.
Users who have registered in the Training Center and have started but not completed a calibration set are not included in the summary reports. Only completed sets are presented in the reports.
However, users who have started a training course will appear in the course roster as being “in progress.”

59. What’s next?
For TELPAS Coordinators

60. Holistic Process Where do I start? Plan your trainings
Handout #3
Where do I start?
Plan your trainings
With your principal, determine how your campus will conduct basic training and calibration activities
Verify trainings have been completed (obtain certificates from personnel)
Download rating rosters, coversheets, etc from ARE website
Be sure to collect and have the correct Years in US schools of all students (ELLs)

61. TELPAS Student Rating Roster

62. Schedule TELPAS Student Tutorials
As in the past, tutorials are separate from tests
Recommended for all students taking TELPAS
Several different test item formats
Since no sample items with test, practice with item formats and online interface is useful
Tutorials available at

63. Add Dates to Your Calendar

64. e-Schools extraction of TELPAS registration file
Next Training
TELPAS– Online Testing Training
Friday, February 20, 2015
1:00 – 4:00
CAB Cafeteria
*Bring YEARS in US!!!
e-Schools extraction of TELPAS registration file

65. Questions and Answers…