1. How Next Generation, APIP, Online Assessments Impact Schools, Teaching, and Learning Dale Cornelius Section Chief, Online Assessments Maryland State Department of Education

2. Introductions My background: Former middle school and high school teacher, Edusoft (assessment data integration), Oakland Unified School District (benchmark assessment), Maryland State Department of Education (online testing). FACT: standardized testing has changed more in the last 5 years than the previous 150+ (going back to 1845 Horace Man Boston Public School first achievement tests). Today Maryland tests approximately 95% online. We started testing online in 2008.

3. What are next-gen assessments? Assessments administered online, aligned to new standards, with technology enhanced items, platform independent*, APIP compliant. (Warning: Definition subject to change) *Platform independent means available on WIFI devices used for LEARNING: DEVICES: desktop, laptop, netbook, tablet, thin client; and OPERATING SYSTEMS: Windows, Mac, Linux, iOS, Chrome OS, Android.

4. APIP: Universal Design of Assessment Who here has heard of APIP? APIP: Accessible and Portable Item Protocol APIP = QTI in 3-D APIP addresses two long-standing needs in assessments: – Transfer content between vendors, standardized XML format. – Supplies accessibility information to support needs of all students. Content tailored to meet the access needs of each student. Origins of APIP: state driven, not vendor (https://www.imsglobal.org/apip/IMSAssessmentPrimerv1p0.html#t oc-6)

5. APIP Supported PARCC* Assessments Accessibility Features for All Students Presentation Accommodations for Students with Disabilities *PARCC (Partnership for the Assessment of Readiness of College and Career) Accessibility Manual: http://www.parcconline.org/assessments/acce ssibility/manual

6. APIP Accessibility Features for All Students Answer Masking (through Student Registration/Personal Needs Profile aka SR/PNP) Audio Amplification (audio controls on computer) Bookmark (flag items for review within online testing platform) Color Contrast (Background/Font Color) (SR/PNP) Eliminate Answer Choices Highlight Tool Line Reader Masking Tool Magnification/Enlargement Device Notepad Pop-up Glossary Spell Check or External Spell Check Device Text-to-Speech for the Mathematics Assessments (SR/PNP) Writing Tools

7. APIP Presentation Accommodations for Students with Disabilities Assistive Technology (e.g., JAWS screen reader) Screen Reader Version (blind or visually impaired) Refreshable Braille Display with Screen Reader Version for ELA/Literacy Tactile Graphics Closed-Captioning of Multimedia Text-to-Speech ASL Video ASL Video for the Mathematics Assessments Examples of Accommodations: https://parcc.pearson.com/tutorial/

8. APIP Text-to-Speech (TTS) vs Kurzweil TTS Note: Pearson supported TTS for online testing limits in functionality due to needed constraints around file size. Goal should be to make supports online as similar as those available for learning. Example (use IE): http://epat- parcc.testnav.com/client/index.html#login?username=TTS ELATutorial&password=PWD PARCC APIP TTS for TestingKurzweil TSS for Learning One voice options: Female12 male and female voice options Speed options: slow, medium, fastSpeed options: adjust words per minute Reading backs up to beginning of paragraph or graph Point to read or reread word or element

9. Part 2: Question: How Next-Gen Assessment impacts states, schools, and teaching and learning.

10. TestsAPIPTStandards/ContentGrades PARCC: Partnership for Assessment of Readiness for College and Careers Yes(8 states plus DC, DoD, and BIE) Maryland College and Career Ready Standards: English Language Arts/Mathematics 3-8 and high school WIDA: Access 2.0 for English Language Learners No38 states and territories WIDA English Language Development Standards: Listening, Speaking, Reading, and Writing. 1-12 MSAA: Multi State Alternate Assessment (1% population) No12 states and territories English Language Arts and mathematics 3-8 and 11 MISA: Maryland Integrated Science Assessment NoMarylandNext Generation Science Standards (NGSS)-Adopted by 12 states 5 and 8 Maryland State Tests NoMarylandMaryland College and Career Ready Standards: Government and Biology High school Maryland State Tests: Trending Towards Consortiums

11. Next-Gen Online Testing Mode Effects Tests given Online and on Paper. – PARCC Maryland 2015 Case Study: 1 District Tested PARCC 100% Paper and did better than students who took tests online. Solution: Test 100% online. Tests 100% Online. – NAEP 4th grade online writing assessment 2011. – Mode effects can be “equated out” at post test. MD has no plans to do this. It’s good information. – Solution anyone? Give ALL students computers.

12. Next-Gen Assessments Impact: Schools Online Testing can be very disruptive to schools: – Online Testing Decreases Time Spent Learning Online. Lack of 1-1 student-to-device ratio in schools means devices tied up with testing for weeks or months. – Online Testing Decreases Total Time Spent Teaching and Learning in Schools. PARCC tests do not fit into school’s schedule. PARCC tests range from 75-110 minutes. Typical class length is 40 minutes. Needs to change. Computer Adaptive Testing possible now that we have multiple years of PARCC data on item reliability. Districts need to give less district-specific tests.

13. Next-Gen Assessments Impact Teaching and Learning Mathematics: done almost exclusively with paper and pencil in the classroom and on assessments. New online assessment tools such as the PARCC equation editor means children need to know how to write formulas online. Question: Does the PARCC Equation Editor assess mathematics or computer literacy? Both. Example (use IE): http://epat- parcc.testnav.com/client/index.html#login?username =PARCC-EE-6-8-Blanks&password=Tutorial http://epat- parcc.testnav.com/client/index.html#login?username =PARCC-EE-6-8-Blanks&password=Tutorial

14. Next-Gen Assessment Impact: Teaching and Learning Curriculum and Instructional Changes: – Teaching keyboarding before grade 3 – Teaching and evaluating listening and speaking – Doing math and writing equations online

15. The Road Ahead: Moving Beyond Traditional, Multiple Choice Assessments New Item Types that Assess Deeper Levels of Understanding (are Harder) – Technology Enhanced Items – Simulations (require reading comprehension, writing, and data analysis). How will simulations be made accessible to low vision or blind students? – More Constructed Response (written) to analyze thought process not just whether or not someone can pick the right answer option from a list. http://www.nationsreportcard.gov/science2009ict/sun/sun1.aspx

16. Next-Gen Assessment in Five Years Trend toward consortia likely to continue (PARCC, WIDA, MSAA) APIP Supported assessments increase More tests administered in classrooms on devices for learning Increase in tests delivered 100% online in many states Increase in new item types: Technology Enhanced Items Simulations (Question: How do you make simulations accessible to low vision or blind people?) With NCLB out and ESSA in, testing for teaching and learning and not just accountability. Getting teachers more involved in item development and scoring of state tests (what a novel idea!) Finally, tests that actually increase student learning and that students enjoy taking. They’re coming. Students already prefer online tests to paper.

17. Thank You! And remember, one last thing, Albert Einstein failed his University Entrance Exams. That should tell you something about the importance of assessment. Q&A?