Presentation on theme: "TELPAS Training 2015 Writing Collection, Rater, & Verifier"— Presentation transcript:
1 TELPAS Training 2015 Writing Collection, Rater, & Verifier This training does not take the place of reading the appropriate manual(s).
2 General informationAdditional TELPAS information (training center link, TELPAS manual links, PLDs, dates) can be found by going to selecting Departments, selecting Testing & Assessment services, and selecting the TELPAS tab.
3 Please Note: This training does NOT take the place of reading the appropriate manuals. 2015 TELPAS RATER Manual2015 TELPAS Reading Test Administrator Manual
4 TELPAS – What is it?The assessment components for grades K–1 and 2–12 include the following:■ Grades K–1holistically rated observational assessments of listening, speaking, reading, and writing■ Grades 2–12multiple-choice online reading tests in six grade clusters: 2, 3, 4–5, 6–7, 8–9, and 10–12holistically rated student writing collectionsholistically rated observational assessments of listening and speaking
5 Who is it for?All K–12 ELLs are required to participate in TELPAS, including students classified as limited English proficient (LEP) in PEIMS who have parents who have declined bilingual/ESL program services (PEIMS code C). ELLs are required to be assessed annually until they meet bilingual/ESL program exit criteria and are reclassified as non-LEP.This includes foreign exchange students and homebound students classified as an ELL.
6 Test Security and Confidentiality Requirements Maintaining the confidentiality of the TELPAS program involves protecting the contents of all online assessments and student performance documentation. This requires compliance with, but is not limited to, the following guidelines:Before handling secure test materials, all testing personnel who meet the requirements to participate in state testing must undergo training and must sign the appropriate security oath affirming that they understand what is considered confidential.Before accessing the secure online administrative features of the Assessment Management System, trained and qualified testing personnel who will be administering online assessments must first read and accept a statement of confidentiality.All tests must be administered in strict accordance with the instructions contained in the test administration materials.
7 Test Security and Confidentiality No person may view, reveal, or discuss the contents of an online assessment before, during, or after a test administration unless specifically authorized to do so by the procedures outlined in the test administration materials. If circumstances necessitate that an online assessment be examined, permission MUST first be obtained from TEA.No person may duplicate, print, record, write notes about, or capture by any electronic means any portion of a secure assessment instrument without prior approval from TEA.Only students may respond to test questions.Test administrators who have permission to view secure materials in order to provide an approved accommodation must be aware that responding to test questions, recording the information they see, scoring the test, or discussing the content of the test at any time is strictly prohibited. As a reminder of this obligation, individuals who have permission to view secure materials in order to provide an approved accommodation are required to sign a separate section of the test administrator oath.No person may review or discuss student responses or student performance data during or after a test administration unless specifically authorized to do so by the procedures outlined in the test administration materials.
8 Penalties for Prohibited Conduct In accordance with 19 TAC §101.65, any person who violates, assists in the violation of, or solicits another to violate or assist in the violation of test security or confidentiality, and any person who fails to report such a violation, may be subject to the following penalties:placement of restrictions on the issuance, renewal, or holding of a Texas educator certificate, either indefinitely or for a set term;issuance of an inscribed or non-inscribed reprimand;suspension of a Texas educator certificate for a set term; orrevocation or cancellation of a Texas educator certificate without opportunity for reapplication for a set term or permanently.Release or disclosure of confidential test content could result in criminal prosecution under TEC § , Section of the Texas Government Code, and Section of the Texas Penal Code. Further, 19 TAC § stipulates that the State Board for Educator Certification may take any of the above actions based on satisfactory evidence that an educator has failed to cooperate with TEA in an investigation.Additionally, irregularities resulting in a breach of test security or confidentiality may result in the invalidation of students’ assessments.
9 Security OathsAll district and campus personnel who participate in state-mandated testing and/or handle or view secure test materials or confidential information must meet the eligibility requirements and be trained and sign a security oath. Any person who has more than one testing role (e.g., a TELPAS rater who also serves as a reading test administrator) must receive appropriate training and sign a security oath for each role.Security oaths for TELPAS raters, writing collection verifiers, and test administrators can be found in Appendix A.NOTE: Any person who serves as a test administrator, even as a relief for a regular test administrator, must receive appropriate training and sign the test administrator oath.Each oath for TELPAS raters, writing collection verifiers, and test administrators must be read and completed after training and before handling or viewing any secure test materials or confidential information.Oaths must be returned to the campus coordinator and will be kept on file in the district for at least five years.
10 Serious Testing Irregularities Serious irregularities constitute severe violations of test security and/or confidentiality and can result in the individual(s) responsible being referred to the TEA Educator Standards and Certification Legal Division for consideration of disciplinary action (including suspension or termination of educator certification credentials). Examples of serious violations involve, but are not limited to, the following:directly or indirectly assisting students with responses to test questionstampering with student responsesfalsifying TELPAS holistic ratings or writing samplessharing answers to the online calibration activities or practice rating activities in the online training coursesviewing secure test content during or after an assessment (unless specifically authorized to do so by the procedures outlined in the test administration materials)discussing secure test content, student responses, or student performancescoring student reading tests, either formally or informally duplicating, recording, or capturing (electronically or by any other means) confidential test content without permission from TEA
11 Procedural Testing Irregularities Procedural irregularities are less severe, more common, and are typically the result of minor deviations in testing procedures. The following are some examples of procedural irregularities specific to TELPAS, however please review the general test security training for other procedural irregularities applicable to all state assessments.Eligibility ErrorEligible students were not rated in one or more domains.Eligible students were not administered the grades 2–12 reading test.Ineligible students were assessed.Improper Accounting for Secure MaterialsA rater, test administrator, campus coordinator, or district coordinator lost or misplaced writing collections or other secure materials.Monitoring ErrorA campus coordinator did not require raters to return confidential materials.Procedural ErrorA TELPAS writing collection was not submitted in accordance with required assembly criteria.A test administrator failed to provide a student with the correct student authorization to access the TELPAS reading test.
12 Testing Irregularities Reporting Testing IrregularitiesEach person participating in the Texas student assessment program is responsible for reporting any violation or suspected violation of test security or confidentiality.Campus staff should notify their campus or district testing coordinator if they witness an irregularity or suspect that one has occurred.
14 SECURE Testing Material All writing samples are considered SECURE testing material, and therefore should be locked up when not being verified or rated.Do not:Place writing samples in a teacher’s mailbox.Leave writing samples unattended on a teacher’s desk.
15 TELPAS ExpectationsCollect a minimum of 5 writing samples which includes:At least one narrative about a past eventTwo academic writing samples from science, social studies, or mathematicsAll writing assignments include students name and dateNo worksheets or question-answer assignmentsNo papers showing teacher correctionsNo writing samples before February 16, 2015Each collection includes samples written primarily in English
16 TELPAS Expectations Gr K-1 Collect a minimum of 5 writing samples which include a balance of social and academic writingsUse the Proficiency Level Descriptors to rate each domainNEW: Collaboration with another teacher is required prior to assigning a rating for listening and speaking.NEW: All writing collections are rated a second time.The following must be completedPrimary Rater: PLD for Writing, Reading, Speaking, Listening per student2nd Rater: PLD for WritingCollaborating teacher: PLD for Speaking and ListeningRating RosterWriting collection cover sheet for each student’s collection
17 TELPAS Expectations Gr 2 - 12 Use the Proficiency Level Descriptors to rate each domainNEW: All writing collections are rated a second time.NEW: To determine students’ ratings for listening and speaking, raters collaborate with other teachers of these students. (The other teachers may or may not be raters.)The following must be completed :Primary Rater: PLDS for Writing, Speaking, Listening per student2nd Rater: PLD for WritingCollaborating Teacher: PLDs for Listening and SpeakingRating RosterWriting collection cover sheet for each student’s collection
18 Assemble and verify grades 2–12 student writing collections Each writing collection must contain at least five writing samples and mustinclude at least one narrative writing sample about a past event and at least two academic writing samples from mathematics, science, or social studies.Writing assigned on or after February 16, 2015, may be considered for the writing collections. Writing assigned before February 16, 2015, is not eligible for inclusion.All writing should reflect a student’s current proficiency level.All samples should be verified prior to rating.After rating, each collection will be re-rated by another qualified TELPAS rater.
19 Developing promptsUse specific information in your prompts to help students focus on the topic of the writing selection.Sample prompts Page in rater manualMany of the examples are vague to show teachers the types of prompts that are acceptable.ALL CORE TEACHERS!!!General Example: Write an expository piece about an important historical or current event.This is not an exhaustive list, but it encompasses the majority of eligible types of writing to include in the collections.Specific Example: Explain why the Bill of Rights was created and why it is important.
20 ENGLISH Narrative writing about a past event Examples: Basic descriptive writing on a personal/familiar topicWriting about a familiar processExamples:Writing about yourself, your family, your best friend, your school, etc.Describing what you see in this picture, photo, piece of art, etc.Comparing yourself to a friend or relative by describing how you are alike and how you are differentComparing two friends, two pictures, two photos, two places you’ve lived or houses you’ve lived in, etc.Examples:A daily routine (getting ready for school, what you do after school or on weekends)How to prepare a familiar food or recipe (sandwich, taco, fruit drink, scrambled eggs)How to play a familiar game or sportREQUIREDNarrative writing about a past eventExamples:Stories based on something shown in a picture or picturesNarratives about something that happened or that you did that was disappointing, unforgettable, surprising, interesting, unexpected, funny, unfair, etc.Original stories composed in creative writing activitiesNarratives about what you did this morning, yesterday, last weekend
21 Personal narratives and reflective pieces Expository and other extended writing on a topic from language artsExamples:A time when you learned a lesson, resolved a problem with another person, etc.What true friendship is, the importance of believing in yourself, setting goals, etc.What you like, don’t like, or would like to see changed about certain school rules or policiesA person you admire, a person who has influenced your life, etc.How first impressions of people can changeWhat it was like to move to the United States, learn a new language, etc.What you thought about the United States or Texas before you moved here compared to what you think nowExamples:Writing reflective pieces linked to stories, literature, poems, and drama read in class (for example, comparing events in a text with personal experiences; relating a character’s conflict in a text to a personal experience, experience of a friend, or experience of a relative; etc.)Describing and analyzing a change that a character undergoesComparing story variants, a movie and novel based on the same story, etc.Writing persuasively to influence an audience to take action on an issueWriting responses to literary or expository texts and providing text evidence to demonstrate understanding
22 math science Social studies Examples:Writing about a way you have used mathematics outside of schoolExplaining the steps you use in a mathematical processWriting about something you are learning in your mathematics classWriting to reflect the thinking you do to solve a problemExamples:Explaining a scientific process you have learned aboutExplaining the steps in an experiment or scientific investigation you have doneWriting about something you are learning in your science classWriting about why lab rules are importantWriting about how to use a certain device or piece of science equipmentSocial studiesExamples:Writing about a historical figure, the person’s contributions or significance, etc.Writing an expository piece about an important historical or current eventWriting about something you are learning in your social studies classDefending a point of view about a governmental policy or controversial issueWriting a persuasive piece to influence a change in policy or law
23 Missing Last Name & Date White-OutFront containswhite-out areasMissing Last Name & Date
24 Sample needs to be about 1 page long. Too ShortSample needs to be about 1 page long.Only the Name & Date were cut for this example
25 Use English if possible Must use sentencesOnly the Name & Date were cut for this exampleA list of words is not a writing sample. Also, no worksheetsUse English if possibleWriting samples should be in English as much as possible.
26 Collaborating: Listening and Speaking Each primary rater will collaborate with one other teacher for each student using the listening and speaking PLDSCollaborating teachers do not have to be trained raters.Collaborating teachers need to be familiar with the Listening and Speaking PLDs.Teachers selected to collaborate will be given copies of the listening and speaking rubrics to select levels for each ELL student. The rubrics will be turned in to the Campus Testing Coordinator by March 11, 2015.
27 K-12 Listening Authentic, performance based activities: Primary raters and selected collaborative teachers will determine listening proficiency levels for each ELL student by reflecting on how well the student understands the English he or she hears during authentic, performance-based activities. The Listening PLD will be used as a guide in determining the listening levels.Authentic, performance based activities:Reacting to oral presentationsResponding to text read aloudFollowing directionsCooperative group workInformal interactions with peersLarge group and small group instructional interactionsIndividual student conferences
28 K-12 SpeakingAuthentic, performance based activities:Cooperative group workOral presentationsInformal interaction with peersLarge group and small group instructional interactionsOne-on-one interviewsClassroom discussionsArticulation of problem-solving strategiesIndividual student conferencesPrimary raters and selected collaborative teachers will determine speaking proficiency levels for each ELL student by reflecting on how well the student speaks English during authentic, performance-based activities. The Speaking PLD will be used as a guide in determining the speaking levels.
32 General Information About Holistically Rated TELPAS Assessments The holistically rated components of TELPAS consist ofwriting collections for grades 2–12;observational assessments in listening and speaking for grades 2–12; andobservational assessments in listening, speaking, reading, and writing for K–1.For these assessments, teachers are trained to use rating rubrics, the proficiency level descriptors (PLDs) from the ELPS, to determine the English language proficiency levels of students based on ongoing classroom observations and written student work.Your campus coordinator will assign students for you to rate. You are responsible for holistically rating the English language proficiency of your assigned students in accordance with the prescribed holistic rating procedures. It is essential for you to complete your rater training requirements and follow the assessment procedures in this manual. After your training, you will:assess the English language proficiency of students and assign one of four ratings: beginning, intermediate, advanced, or advanced high;Work with other raters to ensure validity and reliabilityrecord the students’ ratings and related information on the TELPAS Student Rating Roster
33 General Information About Holistically Rated TELPAS Assessments Your students’ ratings will be reviewed by the campus principal.You will gather writing samples and create student writing collections.If you are a rater of students enrolled in grades 2–12, the writing collections will be verified by another trained individual to ensure that they contain the appropriate types and number of writing samples. The TELPAS Writing Collection Cover Sheet and Writing Collection Verification Checklist in must be completed and stapled to each student’s writing collection.Your campus coordinator will ensure that the writing collections are kept in the students’ permanent record files for two years from the time of rating.School districts use TELPAS assessment results to plan for instruction and to meet state and federal assessment requirements.
34 Summary of process:Teachers unfamiliar with the ELPS and/or TELPAS should receive foundational training in the fall as a best practice. Spring TELPAS rater training will be a smoother process for teachers who already have this foundation.In the spring, testing coordinators cover holistic rating training requirements of raters during TELPAS administration procedures training.New raters complete an online basic training course, which includes practice rating activities for each language domain – listening, speaking, reading (K–1 only), and writing.New raters and raters from previous years complete calibration activities online to ensure that they are applying the PLD rubrics consistently and accurately.Raters from previous years have the option of reviewing the online basic training course, if desired, before completing calibration activities.There are two sets of initial calibration activities. Each set contains 10 students. Each language domain is represented in each set. Raters successful on the first set do not calibrate further.Only individuals who do not rate accurately enough on the first set complete the second set. Those not successful on the second set will receive supplemental training, followed by the third and final calibration set.Individuals not successful on the final set will either not be used as raters (a district decision) or will be provided rater support in accordance with test administration regulations.
35 Holistic Rating Training Basic Online Training (NEW RATERS) due by Friday, February 6, 2015Calibration (NEW and RETURNING RATERS)Attempt 1: due by February 20, 2015Attempt 2, if needed: due by February 24, 2015Supplemental Support training (required for those needing a third attemptTBA: February 24-March 4Attempt 3: due by March 6, 2015
36 Training Certificates Training certificates are generated for raters who complete training courses and calibration activities.A certificate of completion is issued when a rater completes the online basic training course.A certificate of successful completion is issued when a rater demonstrates sufficient calibration (70 percent or better) on a calibration set.
37 Raters who have completed K–1 training but not 2–12 training will be new raters if they switch to 2–12 training, and vice versa.Training for K–1 and 2–12 is separate because of differences in the tested components and rating rubrics (PLDs).Note – Raters who completed K-1 training but not 2-12 training will be new raters if they switch to 2-12 training, and vice versa.37
38 Online Training Components Online Basic Training Course (Required for New Raters)One for raters of K–1 and one for raters of grades 2 and aboveIncludes rating activities to give raters opportunities to practice applying the rubrics (no state-required level of accuracy established)Online Calibration (Required for New and Returning Raters)Separate calibration activities for grades K–1, 2, 3–5, 6–8, and 9–12Assembling and Verifying Grades 2–12 Writing CollectionsRequired of the TELPAS Quality Team
39 Grade Clusters of Training K–1 training components are separate from those of grades 2 and aboveIt is important that raters know which grade cluster of training to completeFor raters with students in multiple clusters in grades 2–12Rater required to train in the cluster with most ELLsAs a best practice, rater should review some online basic training course practice activities in additional cluster(s)For rater with students in multiple clusters including K–1Rater required to train in K–1 and one other cluster in grades 2–12Rater of K–1 who will rate more than one additional cluster within grades 2–12 should train in the cluster (2, 3–5, 6–8, 9–12) with the most ELLsAs a best practice, rater should review some online basic training course practice activities in other cluster(s)
40 Online Calibration Activities Raters complete only as many sets as it takes to calibrate.Many raters will be able to calibrate on the first set of activities, and many others will be able to calibrate by the end of the second set.With supplemental support, very few raters should have difficulty calibrating by the end of the third and final set.Important Note:Individuals are not authorized by TEA to serve as raters unless they complete the state-required training and calibration activities.Raters who complete all requirements but do not successfully calibrate may be authorized to serve as raters at the discretion of the district, but districts are required to provide rating support in all domains for these raters.
41 Online Calibration Activities There are three sets of online calibration activities.Each calibration set contains 10 students to rate, and all holistically assessed language domains are represented in each set.For K–1: listening, speaking, reading, and writingFor 2–12: listening, speaking, and writingA rater needs to rate at least 70 percent of the students correctly to demonstrate sufficient calibration.Raters not successful after calibration set 2 to get supplemental support and then complete set 3 prior to March 6thRaters will receive an access code after completion of supplemental training to complete set 3.
43 Prepare to rate your students Listening (pg.27)The ability to understand spoken language, comprehend and extract information, and follow social and instructional discourse through which information is providedSpeaking (pg.28)The ability to use spoken language appropriately and effectively in learning activities and social interactionsReading (K-1 – pg.28)The ability to comprehend and interpret written text at the grade appropriate levelWriting (pg.29)The ability to produce written text with content and format to fulfill grade appropriate classroom assignments
44 Process for 2 raters Each rater has his/her own PLD Assembler/verifier confirms writing is completeRater 1 rates the writing collection using Rater 1 PLDRater 2 rates the writing collection using Rater 2 PLDIf raters agree on writing rating, that’s FANTASTIC!If raters had differing scores, the two raters discuss to reach agreement.If no agreement can be made, another qualified will rate the collection
50 Newly Enrolled ELLsIf new ELL enrolls before March 16 (start of TELPAS Reading testing window), receiving campus responsible for all TELPAS assessmentsIf new ELL enrolls on or after March 16, receiving campus only responsible for grade 2–12 reading test (if not already administered)Applies whether new student is from another school district, state, or country