Interpreting data for program evaluation and planning

Literacy Coach’s Focus In Data Analysis Program Evaluation Regrouping Form needs-based groups for classroom instruction Assign children to interventions To what extent is my program keeping Benchmark children at benchmark? Choose instructional emphasis To what extent is small-group work moving strategic children to benchmark? To what extent is my program moving Intensive children to benchmark? To what extent are classroom effects apparent?

Literacy Coach’s Focus In Data Analysis Regrouping Form needs-based groups for classroom instruction Assign children to interventions Choose instructional emphasis Which DIBELS reports should I use? Do you have curriculum materials to accomplish this?

Literacy Coach’s Focus In Data Analysis Program Evaluation To what extent is my program keeping Benchmark children at benchmark? To what extent is small-group work moving strategic children to benchmark? To what extent is my program moving Intensive children to benchmark? To what extent are classroom effects apparent?

State-Level Year-End Data,

General Impressions 1.We are increasingly successful in prevention-based instruction in Kindergarten 2.We need to continue to experiment in intervention, particularly for second and third grade Consider time, focus, and explicitness for needs-based work? Consider additional intervention programs?

Cross-Sectional Analysis How well are the kindergarten children this year doing compared to last year? –Did they start out stronger or weaker? –Did they make more or less progress between fall and winter? And yes, these are different children, but the teachers are the same and the program is the same

For Kindergarten Beginning of kindergarten status includes weighted combinations of measures Middle kindergarten directs attention to initial sound fluency End of kindergarten directs attention to phoneme segmentation fluency *You have to look at your own data, considering all measures, to really evaluate your program

State K Cross-Section Fall 2003Winter 2004Spring 2004 I 31% S 43% B 26% I 30% S 40% B 30% I 24% S 34% B 42% Fall 2004Winter 2005 (ISF) Spring 2005 (PSF) I 30% S 42% B 27% I 15% S 46% B 38% I 8% S 18% B 74%

New Directions What did you decide to do differently next year when you saw these data for your school?

For first grade Beginning of first grade status includes weighted combinations of measures Middle first grade directs attention to nonsense word fluency End of first grade directs attention to oral reading fluency *You have to look at your own data, considering all measures, to really evaluate your program

State 1 Cross-Section Fall 2003Winter 2004Spring 2004 I 33% S 32% B 36% I 32% S 30% B 39% I 25% S 30% B 45% Fall 2004Winter 2005 (NWF) Spring 2005 (ORF) I 19% S 29% B 53% I 13% S 40% B 48% I 16% S 26% B 58%

New Directions What did you decide to do differently next year when you saw these data for your school?

For Second Grade Beginning of second grade status includes weighted combinations of measures Middle second grade directs attention to oral reading fluency End of second grade directs attention to oral reading fluency *You have to use the cognitive model of assessment to interpret these data

State 2 Cross-Section Fall 2003Winter 2004Spring 2004 I 27% S 32% B 42% I 32% S 18% B 50% I 38% S 22% B 40% Fall 2004Winter 2005Spring 2005 I 21% S 32% B 47% I 22% S 19% B 59% I 26% S 20% B 54%

New Directions What did you decide to do differently next year when you saw these data for your school?

For Third Grade Third grade data include only oral reading fluency *You have to use the cognitive model of assessment to interpret these data

State 3 Cross-Section Fall 2003Winter 2004Spring 2004 I 25% S 31% B 44% I 33% S 34% B 33% I 28% S 40% B 33% Fall 2004Winter 2005Spring 2005 I 26% S 35% B 40% I 27% S 31% B 42% I 20% S 38% B 41%

New Directions What did you decide to do differently next year when you saw these data for your school?

Cohort Analysis Given children’s experience at your school over time, to what extent is your instructional program actually accelerating literacy growth over time? (and you are right when you say it’s not EXACTLY the same children if your population is highly transient)

State Cohort K-1 Fall 2003Winter 2004Spring 2004 I 31% S 43% B 26% I 30% S 40% B 30% I 24% S 34% B 42% Fall 2004Winter 2005 (NWF) Spring 2005 I 19% S 29% B 53% I 13% S 40% B 48% I 16% S 26% B 58%

State Cohort 1-2 Fall 2003Winter 2004Spring 2004 I 33% S 32% B 36% I 32% S 30% B 39% I 25% S 30% B 45% Fall 2004Winter 2005 (ORF) Spring 2005 I 21% S 32% B 47% I 22% S 19% B 59% I 26% S 20% B 54%

State Cohort 2-3 Fall 2003Winter 2004Spring 2004 I 27% S 32% B 42% I 32% S 18% B 50% I 38% S 22% B 40% Fall 2004Winter 2005Spring 2005 I 26% S 35% B 39% I 27% S 31% B 42% I 20% S 38% B 41%

Interpretation To what extent have you set and communicated the plan? To what extent are teachers understanding and implementing the curriculum? How are they using time? How are they monitoring progress and adjusting their instruction and groupings? How well are they using intervention options?

If you’re not getting the results you want, you have to do something different. Start with yourself Work more closely with administration. Spend more time in classrooms. Focus your pd time on differentiation.

Next Steps Reflect on your own data; check on individual indicators in K and 1 to see if there are particular areas that are troublesome Find your most and least successful classrooms and observe so that you can learn about the curriculum and you can evaluate the effectiveness of your own professional support system Try something different