1. KENTUCKY: POLICIES & PRACTICE Preventing, Detecting, and Investigating Test Security Irregularities: A Comprehensive Guidebook On Test Security For States NCSA Conference 2013: June 20, 2013 1

2. 2 How Many People Cheat On State Tests? 5% ? 10% ? 20% ?

3. NCSA Conference 2013: June 20, 2013 3 · Kentucky Annual Summative and EOC Testing Total Number Percent Cheating Possible Number Cheating Students Tested380,0001/10 of 1%380 Teachers and Administrators45,0001/10 of 1%45

4. NCSA Conference 2013: June 20, 2013 4 Prevention Detection Investigation

5. 1.Security Plan 2.Roles and Responsibilities 3.Budget and Funding 4.Legal Precautions and Funding 5.Test and Item Design NCSA Conference 2013: June 20, 2013 5

6. 6.Test/Item Development and Maintenance 7.Test Publication 8.Test Administration 9.Test Scores and Results 10.Physical Security NCSA Conference 2013: June 20, 2013 6

7. 11.Information Security 12.Web and Media Monitoring (checking the internet for test content) 13.Security Awareness and Training NCSA Conference 2013: June 20, 2013 7

8. 1.Security Plan o Findings: The Department of Education does not maintain a single, comprehensive security plan/manual o Action Tasks: Developing a comprehensive security plan/manual Providing security plan/manual to all key staff and portions to outside contributors as appropriate NCSA Conference 2013: June 20, 2013 8

9. 2.Roles and Responsibilities o Findings: All Department assessment staff are involved in test security with responsibilities for different test security processes; however there is no formal security management structure o Action Tasks: Appointment of a “Security Coordinator” Design a plan instilling a strong security culture within the KDE Build security roles and responsibilities into job descriptions and performance appraisals NCSA Conference 2013: June 20, 2013 9

10. 3.Budget and Funding o Findings: Strong commitment to test security from KDE management, there is no separate budget for test security activities o Action Tasks: Develop an overall security budget Plan for forensic analysis and increased allegations of cheating NCSA Conference 2013: June 20, 2013 10

11. 8.Test Administration o Action Tasks: Test security issues are covered in Test Administration Manuals; however continue to review and improve. Implement the requirement that two persons are present whenever test books are counted or accessed in preparation for testing, during testing and return stages. Modify Student Response Booklets to better engage students in maintaining test security. Capture additional data for future analysis of data. NCSA Conference 2013: June 20, 2013 11

12. NCSA Conference 2013: June 20, 201312

13. 12.Web and Media Monitor o Action Tasks: Monitor the internet looking for test items and passages that appear on KDE assessments. Monitor discussion boards, especially around testing periods. Formalize plans for handling significant test breaches. NCSA Conference 2013: June 20, 2013 13

14. Data Forensics Analysis o Action Tasks: Adding additional data forensics analysis Erasure Analysis Similarity Analysis Score Growth and Losses NCSA Conference 2013: June 20, 2013 14

15. Where Student is Tested Each Student Response Booklet has the location within the school where student is tested. NCSA Conference 2013: June 20, 2013 15

16. Kentucky Investigations Conducting Security Investigations o Findings: Kentucky has procedures for investigating possible security incidents and staff respond swiftly as needed. o Action Tasks: Increase investigative resources to handle the significant in allegations to avoid extended time from reporting to resolution. Change the current de facto policy on initiating a security investigation if the allegation has not come from the field. NCSA Conference 2013: June 20, 2013 16

17. Test Security Reporting of Allegations Get the facts straight. Record the date and time of the incident. List all involved (i.e., staff, volunteers, students). Identify content area and test parts impacted. Identify specific content question(s), if applicable. Record what you learn. Report the incident. NCSA Conference 2013: June 20, 2013 17

18. 1.Commitment From Senior Leadership 2.Professional Audit NCSA Conference 2013: June 20, 2013 18

19. Crosses Many Divisions in State Education Agency Can Involve Other State Organizations Requires District and School Leadership Support Paper and Pencil On-line Testing NCSA Conference 2013: June 20, 2013 19

20. Reasons for Independent Evaluation o Legislature o Broader Experience Low Hanging Fruit Medium Effort Solutions High Cost Solutions NCSA Conference 2013: June 20, 2013 20

21. Roger W. Ervin Systems Administrator Kentucky Department of Education Office of Assessment and Accountability Capital Plaza Tower, 17 th Floor 500 Mero St. Frankfort, KY 40601 Roger.Ervin@education.ky.gov 502-564-2256 NCSA Conference 2013: June 20, 2013 21