2 OVERVIEW OF CURRICULUM-BASED MEASUREMENT AS A GENERAL OUTCOME MEASURE Mark R. Shinn, Ph.D.Michelle M. Shinn, Ph.D.Lisa A. Langell, M.A., S.Psy.S.
3 AIMSweb in a Picture and a Sentence AIMSweb is a 3-tier Progress Monitoring System based on direct, frequent and continuous student assessment which is reported to students, parents, teachers and administrators via a web based data management and reporting system for the purpose of determining response to instruction.
4 CBM is a GOM Used for Scientific Reasons Based on Evidence Reliable and valid indicator of student achievementSimple, efficient, and of short duration to facilitate frequent administration by teachersProvides assessment information that helps teachers plan better instructionSensitive to the improvement of students’ achievement over timeEasily understood by teachers and parentsImproves achievement when used to monitor progress
5 Things to Always Remember About CBM Designed to serve as “indicators” of general reading achievement: CBM probes don’t measure everything, but measure the important things.Standardized tests to be given, scored, and interpreted in a standard wayResearched with respect to psychometric properties to ensure accurate measures of learning
6 Skill Areas Currently Assessable via AIMSweb: Early Literacy [K-1 benchmark, Progress Monitor (PM) any age]Letter Naming FluencyLetter sound fluencyPhonemic Segmentation FluencyNonsense Word FluencyEarly Numeracy (K-1 benchmark, PM any age)Oral CountingNumber identificationQuantity discriminationMissing numberOral Reading (K-8, PM any age)MAZE (Reading comprehension); (1-8, PM any age)Math Computation (1-6, PM any age)Math Facts (PM any age)Spelling (1-8, PM any age)Written Expression (1-8, PM any age)Early Literacy and Oral Reading—Spanish (K-8)
7 Big Ideas of Benchmark (Tier 1) Assessment Benchmarking allows us to add systematic Formative Evaluation to current practice.For Teachers (and Students)Early Identification of At Risk StudentsInstructional PlanningProgress MonitoringFor ParentsOpportunities for Communication/InvolvementAccountabilityFor AdministratorsResource Allocation/Planning and Support
8 Benchmark (Tier 1): 3x per year Strategic Monitoring (Tier 2): 1x per month for select students at risk for educational difficulties (Optional)Progress Monitor (Tier 3): Intensive assessment with adjustable frequency that matches need
9 Managing Data after Assessment is Easy: After completing assessment, quickly type data in system.AIMSweb instantly generates multiple reports for analysis and various decision-making purposes. A few of the many reports available appear here:SAMPLES
10 Three times per year for all students. Tier 1: BenchmarkThree times per year for all students.Fall: September Winter: January Spring: May
12 Box & Whiskers Graphs (box plots): A Brief Explanation AIMSweb commonly uses box plots to report data.This chart will help familiarize yourself with box plots:Consider bell-curve. Box plots are somewhat similar in shape and representation.outlier90th percentileAbove Average Range75th percentileMedian (50th percentile)25th percentileAverage range of population included in sample.Below Average Range10th percentile
41 Compare Your School to Various Composite Groups, By Grade:
42 Progress MonitoringStrategies for Writing Individual Goals in General Curriculum and More Frequent Formative EvaluationMark Shinn, Ph.D. Lisa A. Langell, M.A., S.Psy.S.
43 Big Ideas About Frequent Formative Evaluation Using General Outcome Measures and the Progress Monitoring ProgramOne of the most powerful interventions that schools canuse is systematic and frequent formative evaluation.Benchmark Assessment is not enough for some students because they may be in ineffective programs too long. (3 mos +)The solution is to write individualized goals and determine a feasible progress monitoring schedule.The core of frequent progress monitoring is:Survey-Level AssessmentGoal setting using logical educational practicesAnalysis of student need and resources for determining progress monitoring frequency.
44 With Very Low Performers, Not Satisfactory to Wait This Long!
45 Formative Evaluation of Vital Signs Requires Quality Tools Technical adequacy (reliability and validity);Capacity to model growth (able to represent student achievement growth within and across academic years);Treatment sensitivity (scores should change when students are learning);Independence from specific instructional techniques (instructionally eclectic so the system can be used with any type of instruction or curriculum);Capacity to inform teaching (should provide information to help teachers improve instruction);Feasibility (must be doable).
46 Thinking About A Student’s Data Sample Student:Melissa Smart3rd grade studentProgress Monitor
49 Formative Evaluation: Is data and a goal enough?
50 Formative Evaluation: Are data, goals & trends enough?
51 Formative Evaluation is Impossible without all data: Goals Make Progress Decisions Easier
52 Need Shift to Few But Important Goals Often Ineffective Goal Smorgasboard!Student will perform spelling skills at a high 3rd grade level.Student will alphabetize words by the second letter with 80% accuracy.Student will read words from the Dolch Word List with 80% accuracy.Student will master basic multiplication facts with 80% accuracy.Student will increase reading skills by progressing through Scribner with 90% accuracy as determined by teacher-made fluency and comprehension probes by October 2006.To increase reading ability by 6 months to 1 year as measured by the Woodcock Johnson.Student will make one year's growth in reading by October 2006 as measured by the Brigance.Student will be a better reader.Student will read aloud with 80% accuracy and 80% comprehension.Student will make one year's gain in general reading from K-3.Students will read 1 story per week.
53 Reduce the Number of Goals to a Few Critical Indicators Reading In (#) weeks (Student name) will read (#) Words Correctly in 1 minute from randomly selected Grade (#) passages.Spelling In (#) weeks (Student name) will write (#) Correct Letter Sequences and (#) Correct Words in 2 minutes from randomly selected Grade (#) spelling lists.Math Computation In (#) weeks (Student name) will write (#) Correct Digits in 2 minutes from randomly selected Grade (#) math problems.Written Expression In (#) weeks (Student name) will write (#) Total Words and (#) Correct Writing Sequences when presented with randomly selected Grade (#) story starters.
54 Conducting a Survey Level Assessment Students are tested in successive levels of general curriculum, beginning with their current expected grade placement, until a level at which they are successful is determined.
55 John 5th grader: 5th grade passage 26/12John 3rd grade passage 62/4John 4th grade passage 49/7Conducting a Survey Level Assessment
56 Base Goal Setting on Logical Educational Practices Example of PLEP statement:John currently reads about 26 words correctly from Grade 5 Standard Reading Assessment Passages. He reads Grade 3 reading passages successfully; 62 correct words per minute with 4 errors, which is how well beginning 3rd grade students read this material.