COMBINING EVIDENCE- BASED PRACTICES AND COMMON CORE REQUIREMENTS: A DESIGN EXPERIMENT LRA, Dallas, December 2013

Design Team Sharon Walpole University of Delaware Michael C. McKenna University of Virginia Dori Walk Stephanie Haskins Staunton City, Virginia, Public Schools

Theoretical Orientations Reading Development  Cognitive bias for instructional design  Constrained foundational skills can be developed quickly in small groups  Unconstrained skills must be developed continuously in authentic texts Professional Learning  Teachers must radically change instruction in order to achieve CCSS targets  Teachers may not believe such change is reasonable  Changes in instruction and in student achievement precede changes in beliefs

Assumptions about CCSS New standards for text difficulty represent a meaningful change for teaching. Volume and variety of text reading are important. Read-alouds provide access to challenging text during the period of literacy acquisition. Foundational skills are necessary but insufficient.

Instructional Vision Daily interactive read-aloud of very challenging trade books Shared reading of books aligned to CCSS difficulty requirements Small-group work on foundational skills, guided by informal assessment data

Formative Cycle Pedagogical Goal Instructional Intervention Factors that enhance or inhibit effectiveness Modifications Changes to instructional environment Unanticipated positive or negative effects

2010Fall 2011Spring 2012Fall 2012Spring 2009 Spring 2013 Controlled pilot of small-group approach in first grade Design of new placement test to facilitate assignment to groups Small-group implementation with PD support Achievement with both interventions in place Overview of Design Experiment PD for K-2 teachers in three schools Shared and interactive readings with PD support

Small-Group Pilot Test small-group skills lessons v. enhanced guided reading in first grade  Are lessons feasible in 15 minutes/day?  Daily fidelity protocols  Are lessons associated with gains in real-word reading, nonsense word decoding, spelling, or oral reading fluency?  Standardized tests pre and post

Treatments: 9 weeks Skills (n=17) Direct instruction in letter names and sounds Phonemic awareness instruction Analysis of high-frequency words Decoding of CVC words in isolation Whisper, partner, choral reading of new decodable text each day (last 3 weeks only) Guided Reading (n=18) Build background knowledge Phonemic awareness instruction for target words Choral and partner reading with leveled texts (2 texts per week) Word hunts Vocabulary and comprehension strategy discussions

Results Both treatments were feasible in 15 minutes per day SkillsGuided Reading SS Gain(SD)SS Gain(SD) Nonsense Words Real Words16.11** Spelling Oral Reading Fluency8.33* ** p<.05; * p<.10

Formative Changes Random assignment with DIBELS NWF yielded groups with large SD on standardized measures We designed a placement test (IDI) to use for grouping in the future

Small-Group Scale-Up Move to a natural environment (all three elementary schools in one district) and to K-2 classrooms (all children) Team with central office staff members, principals, school improvement specialists, reading specialists, and teachers

Inclusive Small-Group Model Traditional Guided Reading Vocabulary and Comprehension Fluency and Comprehension Word Recognition and Fluency Phonemic Awareness and Word Recognition

PD Design Formal Presentations Live and Video Demonstrations Walkthroughs and Coaching ModelingData Analysis

Student Achievement Data (PALS) We used existing data to compare results of 1 semester of the new small-group model with historical controls At Risk Spring Before Intervention At Risk Spring After Intervention Z Statistic Kindergarten (N~221) 20 (9%) 11 (5%) p=.04 First Grade (N~230) 51 (22%) 24 (10%) p=.01 Second Grade (N~188) 40 (21%) 24 (11%) p=.00

Unintended Consequences  Teachers needed to see ORF data; we designed a measure that would appear natural, but they wanted something that had been piloted and eventually began to administer DIBELS ORF.  Direct instruction revealed a small number of children who needed a true tier 3 intervention; the district did not have one.  Surveys revealed that some teachers appreciated scripted lessons and others did not; others worried about comprehension.

Formative Changes  We realized that CBMs were not sufficient for grouping and added DIBELS ORF to our assessment requirements.  We realized that we should not implement the small-group model without balancing it with authentic whole-group shared and interactive reading.  We realized that teacher leaders and principals needed tools for coaching; we designed look-for protocols to maintain fidelity to the intervention procedures.

Adding whole-group time  We selected a very small number of instructional strategies to build fluency, vocabulary, and comprehension.  We chose grade 2 texts in the CCSS band and then planned backwards for grades 1 and K.  We designed written responses to keep children engaged during small-group time.

Student Achievement Data (PALS) We used existing data to see whether there were additive gains associated with shared and interactive reading At Risk Spring Before Whole Group At Risk Spring After Whole Group Z Statistic Kindergarten (N~231) 11 (5%) 10 (5%) ns First Grade (N~230) 24 (10%) 19 (8%) ns Second Grade (N~188) 24 (11%) 30 (13%) ns

Student Achievement Data (DIBELS ORF) We used existing data to see whether there were additive gains associated with shared and interactive reading. At Benchmark after Small Group At Benchmark after adding Whole Group Z Statistic First Grade (N~225) 154 (66%) 188 (78%) p=.00 Second Grade (N~225) 135 (60%) 145 (65%) p=.09

We asked teachers in grades 1and 2 to report on effectiveness of the full curriculum Interactive Read-Alouds Teacher reportFrequency Effective in all areas8 Effective in comprehension3 Effective in vocabulary10 Impossible to judge1 Ineffective in grammar2 Shared Reading Effective in all areas7 Effective in word recognition1 Effective in fluency2 Effective in comprehension2 Effective for students at or above grade level2 Ineffective for struggling readers1

Unintended Consequences  Survey data revealed that teachers were concerned about the highest-achieving readers having to read grade-level text.  Teachers reported that some lessons were much too long.  Authentic chapter books contained themes that some teachers found objectionable (e.g., bullying)

Formative Changes  District formed a literacy committee, with representatives from all three schools, to collect feedback and redo the pacing.  A few of our books were replaced; we realized that we should set the parameters for book selection but allow teachers to select the books.  The district began collecting comprehension data.

Limitations  Our initial pilot was limited to one grade level and 9 weeks  Our scale-up efforts do not include control groups for students or systematic fidelity checks for teachers  We used existing achievement data collected by teachers rather than researchers  We do not have data on comprehension or vocabulary

Next steps  We have extensive survey data, not reported here, to use to better understand teacher efficacy and response to this rapid instructional change.  This year’s (2014, spring) achievement data will provide additional insight – the whole-group protocols have been redesigned with teacher input, and many students will have had two years of the full treatment.

Discussion  Formative design may be the only appropriate response to the CCSS requirements; schools are being forced to engage in it without the help of researchers.  A challenge this great requires teams and time; school district partnerships are essential  We have initial evidence that moving foundational skills to small-group time and using challenging shared and interactive reading for whole-group time may be associated with higher levels of achievement for students.