School-wide Data Analysis Oregon RtI Spring Conference May 9 th 2012

Target To develop skills in analyzing school-wide data Today we will focus on the “what” rather than the “how”

Cadre 1: Core Implementation +1.7% *K – NWF; 1 st -5 th – grade ORF School has 90-minute (or less) reading block* (interventions occur WITHIN the core) *60 minute block for ½ day kindergarten School has 90-minute (or less) reading block* (interventions occur WITHIN the core) *60 minute block for ½ day kindergarten

Cadre 1: Core Implementation +8.3% +1.7% *K – NWF; 1 st -5 th – grade ORF School has 90-minute reading block* (interventions occur OUTSIDE of the core)

Cadre 1: Core Implementation +16.5% +8.3% +1.7% *K – NWF; 1 st -5 th – grade ORF School has a 90-minute reading block* (interventions occur OUTSIDE of the core) AND Common, agreed-upon grade- level instructional strategies. School has a 90-minute reading block* (interventions occur OUTSIDE of the core) AND Common, agreed-upon grade- level instructional strategies.

Advanced Organizer Purpose of analyzing school-wide data What data are analyzed Process to analyze school-wide data

Purpose of School-wide Data Analysis 1.To determine the effectiveness of the core programming AND 2.Make necessary adjustments to the core program/instruction if it is not meeting the needs of most students

Why is it important to examine the core programming? The stronger the core programming the less support students will need through interventions.

Trust the data

School-wide Data Analysis When: 2-3 times per year (following collection of your schoolwide screening data) Who: Principal, Literacy Specialist/Title I, Counselor, Grade level team (could include Special Education teacher, ELL teacher, School Psychologist)

Use an Agenda/Guidelines

What data are used? Formative DataSummative Data CBM DIBELS DIBELS Next AIMSweb easyCBM OAKS In Program Assessments Theme Tests Weekly Tests

CBMCore Program Assessment OAKS Standardized Reliable Valid Multiple forms of same difficulty level Research based proficiency standards Availability of data reports

What skills are analyzed? Phonemic Awareness Phonics (Alphabetic Principle) Phonics (Alphabetic Principle) Oral Reading Fluency & Accuracy Oral Reading Fluency & Accuracy Reading Comprehension CBM: FSF, PSF Core Program Assessment CBM: NWF, Accuracy Core Program Assessment CBM: ORF, Accuracy Core Program Assessment CBM: ORF, Maze, Retell, Reading Comprehension Core Program Assessment OAKS Vocabulary easyCBM: vocabulary

School-Wide Analysis Process 1.Identify the effectiveness of the core instruction 2.Identify the common grade level skill needs and the desired goal for that skill/s 3.Identify a plan to make curricular and instructional changes to enhance the core

#1) Identify the effectiveness of the core instruction A.How effective is the core instruction currently? – Determine the current overall percentage of students at proficiency for the grade level B.How has the previous instruction made an impact on the core instruction? – Determine if the percentages at proficiency have changed over time

#1a) How effective is the core instruction currently? Determine the current overall proficiency percentages for each grade level Core Program Assessment OAKS CBM Screening Assessment

CBMCore Program Assessment OAKS Standardized Reliable Valid Multiple forms of same difficulty level Research based proficiency standards Availability of data reports

Proficiency Levels CBM Type of CBMProficiency as compared to….. DIBELS/NextResearch established criteria easyCBMNorms AIMSwebNorms

Benchmarks vs. Norms

General Proficiency Levels for CBMs General Proficiency Levels for CBMs Examples:Not at RiskLowSignificantly low DIBELS Next BenchmarkBelow benchmarkWell below Benchmark easyCBM*Above the 20 th percentile Between 11 th and 20 th percentile ≤10 th Percentile AIMSWEB*Above the 25 th percentile Between the 11 th and 26 th percentile ≤10 th Percentile *easyCBM & AIMSweb default percentile rank settings

Benchmarks: What does it mean to be successful? Norms: What does it mean to be like everyone else? Benchmarks vs. Norms

Benchmarks: What does it mean to be successful? Norms: What does it mean to be like everyone else? Benchmarks vs. Norms

Literacy in the United States (4 th Grade) >99 th percentile < 1 st percentile National Assessment of Educational Progress, 2011 Below Basic (33%) Advanced (8%) Proficient (26%) Basic (33%) 66% of 4 th grade students have demonstrated no more than partial mastery of prerequisite knowledge and skills that are fundamental for proficient grade-level work. In 2008, 1 out of every 4 students who started high school, did not graduate.

Core Program Assessment Proficiency as compared to…..Grade level established criterion Proficiency StandardGenerally 80% OAKS Proficiency as compared to…Norms/RIT scores Proficiency Standard“Meets” Proficiency Levels for other Assessments

#1a) How effective is our core instruction currently? 39% at or above benchmark 11% below benchmark 50% well below benchmark

Levels of Proficiency 39% 11% 27% 50%

easyCBM Example

AIMSweb Example 3 rd Grade Based on one skill (example: ORF )

Talk Time Are you currently examining the proficiency of the core at your grade level? If no……how would you move towards examining the proficiency of the core? If yes… how are you using it to change instruction in the core?

2 nd Grade Core Assessment 15/24= 63% Proficient

23/23= 100% Proficient

OAKS Does not link as well to instructional changes – Strands not technically reliable and valid to draw conclusions

What if the data are mixed? AssessmentData CBMOn level Core ProgramLow OAKSLow AssessmentData CBMLow Core ProgramLow OAKSOn Level

1a) How effective is the core instruction currently for our second grade example? How effective is the core instruction? AssessmentProficiency Level DIBELS Next39% at or above Benchmark Core Program Assessment Theme 2: 63% Proficient Not sufficient

#1b) How has the instruction made an impact on the core instruction? Determine if the proficiency percentages have changed over time – Did the percentage of students at benchmark increase? ___________ – Did the percentage of students at strategic increase or decrease? ____________ – Did the percentage of students at intensive decrease? _____________ (Might have to look at last year’s data)

#1b) Determine how overall proficiency has changed Second Grade Example

#1b) Determine how overall proficiency has changed Third Grade Example

#1b) Determine how overall proficiency has changed First Grade Example

#1b) How has the previous instruction made an impact on the core instruction for our second grade example? Core instruction is having an impact on student achievement DIBELS Next BeginningDIBELS Next Middle 39% at or above Benchmark 52% at or above Benchmark 11% Below Benchmark6 % Below Benchmark 50% Well Below Benchmark 42% Well Below Benchmark Core Program Assessment Theme 2: 63%Theme 3: 100%

School-Wide Analysis Process 1.Identify the effectiveness of the core instruction 2.Identify the common grade level skill needs and the desired goal for that skill/s 3.Identify a plan to make curricular and instructional changes to enhance the core

2) Identify the common grade level skill needs and set the desired goal for that skill/s a)Identify the common grade level skill needs – CBM – Core Program Assessments b) Set desired goal for the identified skill/s – CBM

Vocabulary Reading Comprehension (Retell/Maze) Phonemic Awareness (PSF) Phonics (Alphabetic Principle-NWF) and accuracy Oral Reading Fluency & Accuracy (ORF) Oral Reading Fluency & Accuracy (ORF) # 2a) Identify the common grade level’s skill needs within the core instruction

What skills generally should the grade levels be working on? K: Phonological Awareness (PSF) & Phonics (NWF) 1 st Grade: Fluency (Accuracy & ORF) 2 nd Grade: Fluency/Comprehension (ORF & Maze) 3 rd Grade: Fluency/Comprehension (ORF & Maze) 4 th Grade: Fluency/Comprehension (ORF & Maze) 5 th Grade: Fluency/Comprehension (ORF & Maze) * Grade level may need to work on a different priority skill that it has not “mastered” yet

Which skills have the students reached at least 80%? Start with the most critical of the Big 5 in Reading and determine if the students are at about 80% on that skill. Phonemic Awareness Phonics Fluency (Vocabulary) Comprehension #2a) Determine the most critical priority skill (most students/some students) Balance efficiency versus need

32% 43% Phonics NWF: CLS DORF: Words Correct NWF: WWR DORF: Accuracy PhonicsFluency 38% 41% 2 nd Grade DIBELS Next: Fall

1 st Grade Practice: Winter 85%41% PA Phonics

5 th Grade Practice (Winter) 62% 36% Phonics ORF

Third Grade Third Grade Practice: Fall DORF: Words Correct goal: 70 DORF: Accuracy goal: 95% Daze: Adjusted goal: 8

5 th grade: winter DORF-Accuracy DORF: Words Correct Daze-Adjusted

1 st grade Practice: Winter

1 st grade continued 1 st grade continued

Selection test example 3 rd grade

11/23=48% (comprehension) 19/23=83% (vocabulary)

#2b) Develop an overall instructional goal Look at your current reality. What is an ambitious and attainable goal? Pick a goal for each tier – At Benchmark or above to: 80% ? – Strategic to 15% ? – Intensive at 5% ?

General Guidelines for Developing an Instructional Goal 1.Increase the percentage of Benchmark students by approximately 5-30% from the current benchmark. 2.Decrease/Increase Strategic by approximately 5-15% from the current benchmark. 3. Decrease Intensive percentages by approximately 5-15% from the current benchmark.

2) Identify the common grade level skill needs and the desired goal for that skill/s for our 2 nd grade example Instructional priority: phonics goal setting Current ORF: AccuracyGoal ORF: Accuracy Benchmark41%Benchmark Below Benchmark17%Below Benchmark Well Below Benchmark 41%Well Below Benchmark 60% 15% 25% 41%

School-Wide Analysis Process 1.Identify the effectiveness of the core instruction 2.Identify the common grade level skill needs and the desired goal for that skill/s 3.Identify a plan to make curricular and instructional changes to enhance the core

3) Identify a plan to make curricular and instructional changes to enhance the core a)Instructional Strategies? – More explicit modeling – More explicit guided practice – More explicit corrective feedback – More independent practice b)Active engagement? c) Fidelity?

#3a) Instructional Strategies Tie the instructional strategy to the skill need More explicit guided practice

#3b) Active engagement of all students ExampleNon-Example Choral responding“Who can tell me?” Partner respondingStick from the can

#3c) Fidelity Why is fidelity important? Comprehensive “plan” that incorporates all components of reading The whole school has a common language, common goal, and common tools.

Focus on priority skill Instructional Strategies/engagement Evaluate Core Improving Your Core

Questions Lisa Bates Jon Potter