1. Why Data Integrity? The Importance of Processes and Procedures TASSP Conference, June 2014 Texas Education Agency, Program Monitoring and Interventions Michael Benedict, Heather Christie, Gordon Franzen, Lizette Ramos Copyright © Texas Education Agency 2014. All rights reserved.

2. Today’s Purpose The background of and differences between the three areas monitored by the Texas Education Agency. An overview of TEA's data validation monitoring processes, interventions, and supports. Data integrity tips for secondary schools based on issues and causal factors identified over the years. Copyright © Texas Education Agency 2014. All rights reserved.

3. Performance-Based Monitoring (PBM) Background The Performance-Based Monitoring (PBM) system was developed in 2003. Performance-based Monitoring Analysis System (PBMAS) Data Validation Monitoring (DVM). DVM is a data-driven system that relies on data submitted by districts. Includes annual data validation analyses that examine districts’: Leaver and dropout data; Discipline data; and Student Assessment data. Copyright © Texas Education Agency 2014. All rights reserved.

4. Differences between DVM & PBMAS Copyright © Texas Education Agency 2014. All rights reserved.

5. Differences between DVM & PBMAS Copyright © Texas Education Agency 2014. All rights reserved.

6. Leavers - Legal TEC § 39.308 Amended to require annual electronic audit of drop out records Designed to identify districts that are at a high risk of having inaccurate dropout records, potentially leading to on-site monitoring of drop-out records. Copyright © Texas Education Agency 2014. All rights reserved.

7. 2013 Leavers - Eight Data Validation Indicators 1. Leaver Data Analysis 2. Underreported Students 3. Use of leaver Reason Codes with No Dropouts 4. Use of One or More Leaver Reason Codes 5. Use of Certain Leaver Reason Dropout Codes 6. Missing PET* Submission (August 19, 2013-September 20, 2013) 7. Missing PET Submission (2012-2013 Reporting Year) 8. Continuing Students’ Dropout Rate (Class 2011), as of Fall 2012 *PET is the Person Identification Database (PID) Enrollment Tracking extension. Copyright © Texas Education Agency 2014. All rights reserved.

8. Leavers – 2013 DVM Data Sources #1-5 and #8 reflect the 2011-2012 leaver data submitted by districts in the fall of 2012. #1 and #8 also include data submitted in the fall of 2011 #1 also includes data submitted in the fall of 2010. #6 and #7 – PID* Enrollment Tracking reports for August 20, 2012 through September 20, 2013. * Person Identification Database Copyright © Texas Education Agency 2014. All rights reserved.

9. 2013 DVM Leavers Indicator 4 – Use of One or More Leaver Codes Code 60 – Home School Parent must verify that student IS being home schooled Federal requirement - not enough to document that student intends to be home schooled Proper Documentation Letter, signed and dated by parent/guardian Copyright © Texas Education Agency 2014. All rights reserved.

10. 2013 DVM Leavers Indicator 4 – Use of One or More Leaver Codes Code 81 – Private School in Texas/Texas Job Corps Diploma Program Campus must document that student has actually enrolled in a private school in Texas or the Texas Job Corp Diploma Program Federal requirement - not enough to document that student intends to be home schooled Proper Documentation Transcript request Verification by the superintendent or authorized campus/district administrator of the receiving district Verification by the parent/guardian or qualified student Copyright © Texas Education Agency 2014. All rights reserved.

11. DVM Leavers Indicator 5-Use of Certain Leaver Reason Dropout Codes Code 88 – Student was ordered by a court to attend a GED program and has not earned a GED certificate Students in this category used to be coded 98 – drop-out Required Documentation Copy of court order stating that the student has been ordered to attend a high school equivalency or GED program Copyright © Texas Education Agency 2014. All rights reserved.

12. Leavers - Documentation Requirements and Additional Resources Appendix D of the 2012-2013 PEIMS Data Standards provides definitions and specific guidelines on acceptable documentation for each of the leaver reason codes. PBM contacts at each ESC are available to provide districts with technical assistance concerning the 2013 leaver records data validation indicators (See Appendix C of Leaver PBM Manual). The PEIMS Data Standards, which describe the PEIMS data reporting requirements and provide descriptions of data elements and the codes used to report them. Copyright © Texas Education Agency 2014. All rights reserved.

13. Leavers - Documentation Requirements and Additional Resources Five PEIMS EDIT+ reports in particular that districts may find helpful as part of a local review of leaver coding. PRF8D002: School Leaver Roster PRF8D003: School Leaver Summary PRF8D004: Non-Dropout Non-Graduate Leaver Roster PRF6D002: Dropout Roster PRF0B032: Presumed Underreported Students List Copyright © Texas Education Agency 2014. All rights reserved.

14. DVM – Leavers Next Steps Use Appendix D for documentation requirements Written Procedures Checks and balance verification system Training Administrative staff Office staff

15. DVM-Discipline Legal TEC Chapter 7.028 Allows TEA to monitor data integrity for the purposes of The Public Education Information Management System (PEIMS) and Accountability under TEC Chapter 39 TEC Chapter 37.008 Requires an electronic evaluation of discipline data TEC Chapter 39.057 Authorizes special accreditation investigations when extraordinary numbers of student placements in disciplinary alternative education programs, other than placements under Section 37.006 and 37.007 are determined Copyright © Texas Education Agency 2014. All rights reserved.

16. List of 2013 Discipline Data Validation Indicators 1. Length of Out-of-School Suspension 2. Length of In-School Suspension (Report Only) 3. Unauthorized Expulsion: Students Age 10 and Older 4. Unauthorized Expulsion: Students under Age 10 5. Unauthorized DAEP Placement: Students under Age 6 6. High Number of Discretionary DAEP Placements 7. African American (Not Hispanic/Latino) Discretionary DAEP Placements 8. Hispanic Discretionary DAEP Placements 9. No Mandatory Expellable Incidents Reported for Multiple Years

18. Discipline – 2013 DVM Data Sources Current data validation is for school year 2012-2013 discipline data, which was submitted in June 2013. Indicator 9 also includes discipline data from the 2010-2011 and 2011-2012 school years. Copyright © Texas Education Agency 2014. All rights reserved.

19. DVM Discipline - Indicator 1 Length of Out of School (OSS) Suspension TEC Chapter 37.005 regulates that no out-of-school suspension can last longer than 3 days A partial day counts as a day Check appropriate coding OSS is 05 and 25 Was it OSS or in-school suspension (06 and 26) or a DAEP placement Copyright © Texas Education Agency 2014. All rights reserved.

20. DVM Discipline – Indicator 3 Unauthorized Expulsion – Age 10 and Older A reason code of 21 – violation of student code of conduct is not an allowable expulsion offense Incorrect action codes entered The disciplinary action was actually a DAEP placement, but was coded as an expulsion to DAEP (codes 03 and/or 04) Documentation reviewed Referrals Student code of conduct Due process documentation Copyright © Texas Education Agency 2014. All rights reserved.

21. DVM Discipline - Indicator 6 High Number of Discretionary DAEP Placements Action codes monitored 07, 08, 10 For reason code 21 – violation of student code of conduct- to be appropriate, DAEP has to be listed in published district student code of conduct as applicable consequence for specific incident listed on referral Appropriate conference has to be held according to TEC 37.009 Copyright © Texas Education Agency 2014. All rights reserved.

22. Due Process Conferences TEC 37.009 DAEP Student given written or oral notice of reasons for removal Explanation for basis of removal Opportunity to respond to the reasons Expulsion Student given written notice of reasons for removal Explanation for basis of removal Opportunity to respond to the reasons

23. Due Process Conference Documentation Notice of conference Date, time, and who was in attendance Content of conversation Final decision of placement, including reason and length Ensure students receiving special education services have appropriate ARD committee meetings prior to final placement decisions Copyright © Texas Education Agency 2014. All rights reserved.

24. DVM Discipline – Next Steps Written policies and procedures Checks and balance verification system Training Administrative staff Staff entering discipline codes Review/revise student code of conduct

25. DVM Student Assessment - Legal TEC § 39.057 Authorizes special accreditation investigations when excessive numbers of absences of students eligible to be tested on state assessment instruments are determined. TEC § 7.028 Allows TEA to monitor data integrity for the purposes of The Public Education Information Management System (PEIMS) and Accountability under TEC Chapter 39 Copyright © Texas Education Agency 2014. All rights reserved.

26. DVM – Student Assessment Indicators 1. STAAR 3-8 Absent Rate Mathematics* 2. STAAR 3-8 Absent Rate Reading* 3. STAAR 3-8 Absent Rate Science* 4. STAAR 3-8 Absent Rate Social Studies* 5. STAAR 3-8 Absent Rate Writing* * Indicator is calculated for 11 student groups

27. DVM – Student Assessment Indicators 6. STAAR 3-8 Absent Other Mathematics* 7. STAAR 3-8 Absent Other Reading* 8. STAAR 3-8 Absent Other Science* 9. STAAR 3-8 Absent Other Social Studies* 10. STAAR 3-8 Absent Other Writing* * Indicator is calculated for 11 student groups

28. DVM – Student Assessment Indicators 11. TELPAS Reading Absent Rate 12. TELPAS Reading Other Rate 13. STAAR End-of-Course (EOC) Test Participation Rate* 14. Discrepancy between PEIMS** Career and Technical Education (CTE) Status and STAAR EOC Answer Documents * Calculated for 6 subject area tests **Public Education Information Management System

29. Student Assessment – 2013 DVM Data Sources Student 3-8 STAAR and EOC answer documents (latest spring 2013) Indicator 13 (EOC Test Participation Rate) EOC answer documents Course completion data summer submission Indicator 14 (Discrepancy between PEIMS CTE status and STAAR EOC answer documents EOC answer documents Fall PEIMS snapshot submission (2012) Copyright © Texas Education Agency 2014. All rights reserved.

30. Student Assessment – Indicator 6 STAAR 3-8 Other Rate Mathematics Students testing above grade level The “O” code goes on the enrolled grade level answer document The actual test taken document is marked “S” for score Possible next steps DCCM* coding reference System for tracking students issued an “O” score code * District Campus Coordinator Manual Copyright © Texas Education Agency 2014. All rights reserved.

31. Student Assessment – Indicator 13 EOC Test Participation Rate Percentage of students who completed a course for which an EOC assessment is required (according to PEIMS record 415) that are Missing an EOC answer document, Coded absent on an EOC answer document, or Coded other on an EOC answer document It is the “missing” component that is causing districts to trigger this indicator Copyright © Texas Education Agency 2014. All rights reserved.

32. Student Assessment – Indicator 13 EOC Test Participation Rate Possible causes TAKS students PEIMS course completion submission (415 record) not correct Possible next steps Verify PEIMS 415 record before submission System to verify all required students are scheduled for testing Copyright © Texas Education Agency 2014. All rights reserved.

33. Student Assessment – Indicator 14 Discrepancy Between PEIMS and Answer Document Coding for CTE Validates students with a CTE course code of 2 or 3 on fall PEIMS snapshot to students with CTE course code of 2 or 3 on EOC answer documents Two possible causes Not clearly defining CTE programs in fall Not following answer document precode verification protocols Possible next steps Have system for verifying precode data Checks and balance verification system Copyright © Texas Education Agency 2014. All rights reserved.

34. Today’s Key Take Aways Establish written district-wide processes and procedures Develop and follow through on annual training Design second verification before PEIMS submissions Copyright © Texas Education Agency 2014. All rights reserved.

35. 2014 DVM – Tentative Release Dates Leavers October 31 Discipline November 21 State Assessment December 19 Copyright © Texas Education Agency 2014. All rights reserved.

36. Resources Leaver DVM Manual Discipline DVM Manual Student Assessment DVM Manual Appendix D – Leaver codes and required documentation Appendix E – Discipline Reason Codes and Action Codes District and Campus Coordinator Manual Copyright © Texas Education Agency 2014. All rights reserved.

37. Questions

38. Contact Information Division of Program Monitoring and Interventions Phone: (512) 463-5226 Email: PMIdivision@tea.state.tx.usPMIdivision@tea.state.tx.us Copyright © Texas Education Agency 2014. All rights reserved.

39. Copyright © Notice The materials are copyrighted © and trademarked ™ as the property of the Texas Education Agency (TEA) and may not be reproduced without the express written permission of TEA, except under the following conditions: 1. Texas public school districts, charter schools, and Education Service Centers may reproduce and use copies of the Materials and Related Materials for the districts’ and schools’ educational use without obtaining permission from TEA. 2. Residents of the state of Texas may reproduce and use copies of the Materials and Related Materials for individual personal use only without obtaining written permission of TEA. 3. Any portion reproduced must be reproduced in its entirety and remain unedited, unaltered and unchanged in any way. 4. No monetary charge can be made for the reproduced materials or any document containing them; however, a reasonable charge to cover only the cost of reproduction and distribution may be charged. Private entities or persons located in Texas that are not Texas public school districts, Texas Education Service Centers, or Texas charter schools or any entity, whether public or private, educational or non-educational, located outside the state of Texas MUST obtain written approval from TEA and will be required to enter into a license agreement that may involve the payment of a licensing fee or a royalty. For information contact: Office of Copyrights, Trademarks, License Agreements, and Royalties, Texas Education Agency, 1701 N. Congress Ave., Austin, TX 78701-1494; phone 512-463-9270 or 512-463-9713; email: copyrights@tea.state.tx.us Copyright © Texas Education Agency 2014. All rights reserved.