1. CALIFORNIA DEPARTMENT OF EDUCATION Tom Torlakson, State Superintendent of Public Instruction California Educational Research Association December 3−5, 2014 Lily Roberts, Ph.D., and Sheila Self, M.Ed. English Language Proficiency and Alternate Assessments Office Transition to New Alternate Assessments: A Paradigm Shift for Assessing Students with Significant Cognitive Disabilities

2. TOM TORLAKSON State Superintendent of Public Instruction Overview Background Theory of Action Core Content Connectors Communicative Competence Paradigm Shift National Center and State Collaborative (NCSC) Pilot Administration Model Phase I Pilot Results Phase II Pilot Results New Alternate Assessments 2

3. TOM TORLAKSON State Superintendent of Public Instruction Background In July 2014, the State Board of Education provided the following guidance: –Discontinue the administration of the California Alternate Performance Assessment (CAPA) for English−language arts (ELA) and mathematics. –Administer CAPA science in grades five, eight, and ten until there is a successor. –Move forward with the Phase II Pilot to include California students in data analyses. –Seek a waiver from double testing. –Explore alternate assessment options. 3

4. TOM TORLAKSON State Superintendent of Public Instruction Theory of Action 4

5. TOM TORLAKSON State Superintendent of Public Instruction Core Content Connectors Core Content Connectors (CCCs) are: –Connected to a pathway of knowledge and skills in a subject area –Linked to the Common Core State Standards (CCSS) –Less complex than the CCSS –Focused on the main academic content in each subject and grade, to deepen students understanding in later grades –Used to plan instruction and assessment for students who will take the alternate assessment 5

6. TOM TORLAKSON State Superintendent of Public Instruction Core Content Connectors (cont.) Students typically follow a pathway of knowledge as they move upward through the grade’s “learning progression.” Adjusted alternate achievement standards called “essential understandings” are content-based and are entry points for students beginning to interact with grade-level curriculum. 6

7. TOM TORLAKSON State Superintendent of Public Instruction Core Content Connectors (cont.) 7 Progress Indicator: E.RL.k identifying central ideas and key details to derive author’s purpose, message, or theme. Common Core State Standard Core Content Connectors: 4 Essential Understanding 4.RL 2 Determine a theme of a story, drama or poem from details in the text; summarize the text. 4.RL.k2 Determine the theme of a story, drama, or poem; refer to text to support answer. Determine the topic of a story or poem.

8. TOM TORLAKSON State Superintendent of Public Instruction Communicative Competence The foundation of the NCSC model for achieving college and career readiness is communicative competence. –Communicative competence is the basis for learning and is necessary for active community participation. –Students must develop expressive communication skills through direct instruction in order to have an effective, individualized system of communication at home, at school, and in the community. 8

9. TOM TORLAKSON State Superintendent of Public Instruction Communication at some level is possible and identifiable for all students regardless of functional “level.” –For students who have not yet developed communicative competence, this must be an immediate objective. –Communication needs must be addressed during the individualized education program (IEP) process to ensure routine use of the appropriate communication system during instruction and assessment. Communicative Competence (cont.) 9

10. TOM TORLAKSON State Superintendent of Public Instruction Paradigm Shift “Some people believe that students with significant cognitive disabilities cannot successfully learn academic content. Studies show that this is a misperception. The education system should start with the assumption that every child can learn. This is called ‘the least dangerous assumption’ because exposing students to learning is not harmful, but keeping them from it is.” National Center and State Collaborative 10

11. TOM TORLAKSON State Superintendent of Public Instruction Common Care State Standards and Alternate Assessments Alternate assessments must be linked to the Common Core State Standards. The alternate assessment is derived from the CCSS, using links called core content connectors. Not all CCSS are measured, only the essential elements or “big ideas.” The benefit of aligning the alternate assessment with CCSS is that all educators can benefit from CCSS professional development and can collaborate on “how to” teach the standards for all students. 11

12. TOM TORLAKSON State Superintendent of Public Instruction NCSC Administration Model NCSC is a consortium of 26 states, including California, and five national organizations. NCSC’s purposes are to: –Build an alternate assessment based on alternate achievement standards aligned with the CCSS through CCCs. –Assess students with SCD in grades three through eight and grade eleven. –Create grade-level content for ELA and mathematics that has four tiers with increasing levels of complexity. –Use computer-based technology for test administration. 12

13. TOM TORLAKSON State Superintendent of Public Instruction NCSC Administration Model (cont.) Student eligibility requirements: –Student has significant cognitive disabilities –Student learning content is linked to CCSS –Student requires extensive direct individualized instruction and substantial supports to achieve measureable gains in grade and age-level curriculum. 13

14. TOM TORLAKSON State Superintendent of Public Instruction NCSC Administration Model (cont.) The pilot administration’s features included: –One-to-one testing –Test forms for grades three through eight and grade eleven for ELA and mathematics –Scripted directions for administration –Whole passages re-read if the student requested or would benefit from repetition –Some questions that referenced specific passage parts to be read to the student –Items at all levels of difficulty Hand over hand was not permitted 14

15. TOM TORLAKSON State Superintendent of Public Instruction NCSC Administration Model (cont.) The Learner Characteristics Inventory (LCI) provided student-level data about vision, hearing, and classroom setting. The Student Response Check (SRC) was used before testing to observe the student’s communication ability by demonstrating a consistent response to three content neutral test questions by using a mouse, verbalizing answer, gesturing or pointing, using an eye gaze chart, circling or marking answers on a paper copy. 15

16. TOM TORLAKSON State Superintendent of Public Instruction NCSC Administration Model (cont.) If the student did not display a consistent and observable response: –Administer the first four items of the test; if you observe one response with any of the first four items, administer the test. –Administer the first four items of the test; if you do not observe a response, terminate the test. The test has the ability to pause and resume as many times as necessary during the test window. 16

17. TOM TORLAKSON State Superintendent of Public Instruction NCSC Administration Model (cont.) Accommodations included: –Assistive technology for presentation of items –Assistive technology for student responses to questions (e.g., alternate keyboard, switches and hub, or head mouse) –Print version of items –Scribe to complete graphic organizers or response template for writing product –Sign language interpretation 17

18. TOM TORLAKSON State Superintendent of Public Instruction NCSC Administration Model (cont.) Item types included: –Selected response –Constructed response (mathematics) –Constructed response (writing) –Open response (reading foundational items in grades four and five) 18

19. TOM TORLAKSON State Superintendent of Public Instruction Sample NCSC Item Selected response sample ELA reading passage for grade four: “We are going to read a story about a boy named Ben. After we read, you will be asked a question about the theme. Listen for the message of the story.” 19

20. TOM TORLAKSON State Superintendent of Public Instruction 20

21. TOM TORLAKSON State Superintendent of Public Instruction 21

22. TOM TORLAKSON State Superintendent of Public Instruction 22

23. TOM TORLAKSON State Superintendent of Public Instruction 23

24. TOM TORLAKSON State Superintendent of Public Instruction 24 Remember, the theme is the message of the story. What is the theme of this story? having a lot of friends showing you can take care of a pet asking your parents for something

25. TOM TORLAKSON State Superintendent of Public Instruction Phase I Pilot Results The target sample size for Pilot I was 16,000 for the consortium and 7,700 for California. The consortium had a total of 5,161 students complete the test, and California had 1,651 students complete the test (32 percent). The participation rate was significantly lower for all states because of the test window being changed from February to April. (April 15 through May 24) –Overlapped with Smarter Balanced and CAPA testing 25

26. TOM TORLAKSON State Superintendent of Public Instruction Phase I Pilot Results (cont.) A post-survey of 178 coordinators and administrators indicated that: –Training was effective (77 percent). –Sample questions were used prior to administration (81 percent). –Technology infrastructure was sufficient (81 percent), but technology issues were experienced during testing ( 79 percent). –Directions for administration were easy to follow (68 percent). –Students were engaged in testing (58 percent). 26

27. TOM TORLAKSON State Superintendent of Public Instruction Pilot I Pilot Results (cont.) NCSC Technical Advisory Committee (TAC) recommendations: –Conduct various studies of Pilot I data (e.g., sample representativeness, scrolling issues). –Further investigate tiers. –Maintain use of tiers in operational assessment. –Develop stopping rule for students who respond without intentionality. –Adjust flagging criteria for item data review. 27

28. TOM TORLAKSON State Superintendent of Public Instruction Phase I Pilot Results (cont.) TAC recommendations (cont.): –Work with special education experts to evaluate length and difficulty of the Phase II Pilot. –Use linear form and consider form length for the Phase II Pilot. –Postpone stage-adaptive assessments until after the grant ends. –Explore tension between reliability and student/teacher interest/fatigue. 28

29. TOM TORLAKSON State Superintendent of Public Instruction NCSC and states conducted a rigorous sampling effort to ensure appropriate number and range of students for Pilot II. Pilot II focused on scoring for foundational, multipart, writing, and passage items. Results will be available next year. Phase II Pilot Results (cont.) 29

30. TOM TORLAKSON State Superintendent of Public Instruction Phase II Pilot Results Testing window was October 20 through November 21, 2014. Approximately 1,500 California students completed testing. 21 test forms were assigned in either reading and writing or reading and mathematics across grades three through eight and grade eleven. 25−30 items per content area were administered. 30

31. TOM TORLAKSON State Superintendent of Public Instruction Phase II Pilot Results (cont.) 31

32. TOM TORLAKSON State Superintendent of Public Instruction New Alternate Assessment Spring 2015 alternate assessment field test for all eligible students in grades three through eight and grade eleven New computer-based items for ELA and mathematics: –Items aligned with the CCSS through CCCs –Approximately 15 items per content area The spring field test: April 15 through June 10, 2014 32

33. TOM TORLAKSON State Superintendent of Public Instruction New Alternate Assessment (cont.) To assist with the development of the 2015 field test: –An educator focus group was convened to discuss test item characteristics. –Educators provided guidance on effective item types for students with SCD. –A two day item development training occurred to guide educators in writing items to content standards. 33

34. TOM TORLAKSON State Superintendent of Public Instruction New Alternate Assessment (cont.) − The California Department of Education (CDE) examined other state item types and processes (e.g., Oregon). − CDE staff will participate in the item review process in December and February. 34

35. TOM TORLAKSON State Superintendent of Public Instruction New Alternate Assessment (cont.) The development of the 2016 operational alternate assessment will take into consideration outcomes and results from: –NCSC Phase I and II Pilots and operational assessment –California spring 2015 field test outcomes –Any other available sources 35

36. TOM TORLAKSON State Superintendent of Public Instruction Contact Information English Language Proficiency and Alternate Assessments Office Phone: 916-319-0784 Alternate Assessments e-mail: alternateassessment@cde.ca.gov Alternate Assessments Web page at: http://www.cde.ca.gov/ta/tg/ca/altassessment.asp 36