1. Missouri’s Experience with Automated Scoring National Conference on Student Assessment June 2014

2. Missouri and Automated Scoring 2 English II Writing Prompt Responses On demand writing Previously administered Paper/pencil Online Online deployment Administered in spring 2013

3. Missouri and Automated Scoring 3 Process Conversion of training papers to electronic format compatible with scoring engine Training of human scorers to score training sample Turning the engine on

4. Missouri and Automated Scoring 4 Monitoring Performance of Engine Human validity scores Scoring engine validity scores Daily reports

5. Missouri and Automated Scoring 5 Challenges Scoring engine and human score inter- rater reliability Daily reports Training the engine Time

6. Contact Us www.dese.mo.gov Email: assessment@dese.mo.gov Phone: 573-751-3545 The Department of Elementary and Secondary Education does not discriminate on the basis of race, color, religion, gender, national origin, age, or disability in its programs and activities. Inquiries related to Department programs and to the location of services, activities, and facilities that are accessible by persons with disabilities may be directed to the Jefferson State Office Building, Office of the General Counsel, Coordinator – Civil Rights Compliance (Title VI/Title IX/504/ADA/Age Act), 6th Floor, 205 Jefferson Street, P.O. Box 480, Jefferson City, MO 65102-0480; telephone number 573- 526-4757 or TTY 800-735-2966; email civilrights@dese.mo.gov.