1. 1 FUNCTIONAL ASSESSMENT: Curriculum-Based Measurement Progress Monitoring NC Department of Public Instruction Exceptional Children Division Program Improvement & Professional Development-2008

2. 2 Today’s Agenda What is CBM? What is CBM? State norming project review State norming project review Hands on CBMs Hands on CBMs Measuring Behavior in an RtI model Measuring Behavior in an RtI model Progress Monitoring Progress Monitoring –Charting Activity –Setting Goals –Generalizing Charting

3. 3 What is the Responsiveness to Instruction Model? A student with academic delays is given one or more research-validated interventions. A student with academic delays is given one or more research-validated interventions. The student's academic progress is monitored frequently to see if those interventions are sufficient to help the student to catch up with his or her peers. The student's academic progress is monitored frequently to see if those interventions are sufficient to help the student to catch up with his or her peers. If the student fails to show significantly improved academic skills despite several well- designed and implemented interventions, this failure to 'respond to intervention' can be viewed as evidence of an underlying Learning Disability. If the student fails to show significantly improved academic skills despite several well- designed and implemented interventions, this failure to 'respond to intervention' can be viewed as evidence of an underlying Learning Disability. www.interventioncentral.com

4. Target Student Discrepancy 1: Skill Gap (Current Performance Level) Avg Classroom Academic Performance Level ‘Dual-Discrepancy’: RTI Model of Learning Disability (Fuchs 2003) Discrepancy 2: Gap in Rate of Learning (‘Slope of Improvement’)

5. 5 Why RtI? One advantage of RtI in the diagnosis of educational disabilities is that it allows schools to intervene early to meet the needs of struggling learners. One advantage of RtI in the diagnosis of educational disabilities is that it allows schools to intervene early to meet the needs of struggling learners. Another advantage is that RtI maps those specific instructional strategies found to benefit a particular student. This information can be very helpful to both teachers and parents. Another advantage is that RtI maps those specific instructional strategies found to benefit a particular student. This information can be very helpful to both teachers and parents. www.interventioncentral.com

6. What are Summative and Formative Evaluation? Summative Assessment: Culmination measure. Mastery assessment. Pass/fail type assessments which summarize the knowledge students learn. High-stakes tests (EOGs)High-stakes tests (EOGs) GRE, ACT, SAT, GMAT, etc. testsGRE, ACT, SAT, GMAT, etc. tests Driver’s license testDriver’s license test Adapted from www.aimsweb.com

7. What are Summative and Formative Evaluation? Formative Evaluation: Process of assessing student achievement during instruction to determine whether an instructional program is effective for individual students. When students are progressing, continue using your instructional programs. When tests show that students are not progressing, you can change your instructional programs in meaningful ways. Adapted from www.aimsweb.com

8. Types of Assessment PurposeMedicalEducationalScreeningTemperature, Blood Pressure, Urine samples, etc. CBMs Diagnostic Exam, history, more invasive tests R.I.O.T. Progress Monitoring Temperature, Blood Pressure, Urine samples, etc. CBMs

9. Summative Assessment Example: Weight High standard: All children will have a healthy weight by the end of third grade. High Stakes Assessment: Assessing body density. Weighing each student.Weighing each student. Immersing each student in a large tub filled with water and measuring the amount of water displaced.Immersing each student in a large tub filled with water and measuring the amount of water displaced. Divide weight by displacement and get density, a very accurate picture of physical status.Divide weight by displacement and get density, a very accurate picture of physical status. After 8-9 YEARS of growth, we would: 1.Place students who are “unhealthy” in remedial programs. 2.Create new health programs. 3.Blame the effectiveness of old health programs. 4.Blame the students (or their families) for over or under eating. Adapted from www.aimsweb.com

10. Formative Evaluation Example: Same Standard, Different Assessment High Standard: All children will have a healthy weight by the end of third grade. Universal Screening Assessment: Monitor weight directly, frequently, and continuously. From birth, measure weight frequently and continuously with a simple, albeit less precise, general outcome measure, weight in pounds, using a scale. (General Outcome Measure) Adapted from www.aimsweb.com

11. At any point in development… The child could be weighed and a decision made about healthy weight. This process is: Efficient. Efficient. Sufficiently accurate. Sufficiently accurate. Proactive. Proactive. Cost effective. Cost effective. We would know their health status before they reached the high stakes point! Formative Evaluation Example: Same Standard, Different Assessment Adapted from www.aimsweb.com

12. Characteristics of General Outcome Measures (GOMs) Powerful measures that are: Simple Simple Easier to obtain data (less time and good data) Easier to obtain data (less time and good data) Accurate Accurate Very specific data Very specific data Efficient Efficient Only a few minutes to administer Only a few minutes to administer Generalizable Generalizable Reliable Reliable Can compare and contrast student performance across school, district, country Can compare and contrast student performance across school, district, country Adapted from www.aimsweb.com

13. General Outcome Measures (GOMs) from Other Fields Medicine measures height, weight, temperature, and/or blood pressure. Federal Reserve Board measures the Consumer Price Index. Wall Street measures the Dow-Jones Industrial Average. Companies report earnings per share. McDonald’s measures how many hamburgers they sell. In Education, Curriculum Based Measurement is a General Outcome Measure Adapted from www.aimsweb.com

14. What is Curriculum Based Measurement? Curriculum Based Measurement is a form of Curriculum- Based Assessment (CBA). Curriculum Based Measurement is a form of Curriculum- Based Assessment (CBA). CBM is a method of monitoring student educational progress through direct assessment of academic skills CBM is a method of monitoring student educational progress through direct assessment of academic skills CBMs measure basic skills in reading, mathematics, spelling, written expression and readiness skills CBMs measure basic skills in reading, mathematics, spelling, written expression and readiness skills –Teacher gives the student brief, timed samples, or “probes,” made up of academic material that the student is expected to learn Early literacy skills (phonics and phonological awareness) are downward extensions of CBM. Early literacy skills (phonics and phonological awareness) are downward extensions of CBM. Because CBM probes are quick to administer and simple to score, they can be given frequently to provide continuous progress data. The results are charted and provide for timely evaluation based on hard data. Because CBM probes are quick to administer and simple to score, they can be given frequently to provide continuous progress data. The results are charted and provide for timely evaluation based on hard data. Adapted from www.aimsweb.com

15. Using Curriculum Based Measures as General Outcome Measures It’s about using General Outcome Measures (GOMs) for formative assessment/evaluation to: Inform teachingInform teachingAND ensure accountability. ensure accountability. It’s different from, but related to, summative high- stakes testing/evaluation, which: Doesn’t inform teaching. Doesn’t inform teaching. Mostly used for accountability/motivation. Mostly used for accountability/motivation. Adapted from www.aimsweb.com

16. Universal (school-wide) screening using CBMs allows us to add systematic Formative Evaluation to current practice. For Teachers (and Students) For Teachers (and Students) Early Identification of At Risk Students Early Identification of At Risk Students Instructional Planning Instructional Planning Monitoring Student Progress Monitoring Student Progress For Parents For Parents Opportunities for Communication/Involvement Opportunities for Communication/Involvement Accountability Accountability For Administrators For Administrators Resource Allocation/Planning and Support Resource Allocation/Planning and Support Accountability Accountability Using Curriculum Based Measurement as a General Outcome Measure Adapted from www.aimsweb.com

17. Using Curriculum Based Measurement as a General Outcome Measure: Research Curriculum-Based Measurement (CBM) was developed more than 20 years ago by Stanley Deno at the University of Minnesota through a federal contract to develop a reliable and valid measurement system for evaluating basic skills growth. Curriculum-Based Measurement (CBM) was developed more than 20 years ago by Stanley Deno at the University of Minnesota through a federal contract to develop a reliable and valid measurement system for evaluating basic skills growth. CBM is supported by more than 25 years of school-based research by the US Department of Education. CBM is supported by more than 25 years of school-based research by the US Department of Education. Supporting documentation can be found in 100s of articles, book chapters, and books in the professional literature describing the use of CBM to make a variety of important educational decisions. Supporting documentation can be found in 100s of articles, book chapters, and books in the professional literature describing the use of CBM to make a variety of important educational decisions. Adapted from www.aimsweb.com

18. Summary of Research Validating Curriculum Based Measurement Reliable and valid indicator of student achievement Simple, efficient, and of short duration to facilitate frequent administration by teachers Provides assessment information that helps teachers plan better instruction Sensitive to the improvement of students’ achievement over time Easily understood by teachers and parents Improves achievement when used to monitor progress Adapted from www.aimsweb.com

19. Curriculum Based Measurement: Advantages Direct measure of student performance Direct measure of student performance Helps target specific areas of instructional need for students Helps target specific areas of instructional need for students Quick to administer Quick to administer Provides visual representation (reports) of individual student progress and how classes are acquiring essential reading skills Provides visual representation (reports) of individual student progress and how classes are acquiring essential reading skills Sensitive to even small improvements in performance Sensitive to even small improvements in performance Capable of having many forms Capable of having many forms Monitoring frequently enables staff to see trends in individual and group performance—and compare those trends with targets set for their students. Monitoring frequently enables staff to see trends in individual and group performance—and compare those trends with targets set for their students. Correlates strongly with “best practices” for instruction and assessment, and research-supported methods for assessment and intervention. Correlates strongly with “best practices” for instruction and assessment, and research-supported methods for assessment and intervention. Adapted from www.aimsweb.com

20. Curriculum Based Measurement: Things to Remember Designed to serve as “indicators” of general reading achievement: CBM probes don’t measure everything, but measure the important things. Designed to serve as “indicators” of general reading achievement: CBM probes don’t measure everything, but measure the important things. Standardized tests to be given, scored, and interpreted in a standard way. Standardized tests to be given, scored, and interpreted in a standard way. Researched with respect to psychometric properties to ensure accurate measures of learning. Researched with respect to psychometric properties to ensure accurate measures of learning. Are sensitive to improvement in brief intervals of time. Are sensitive to improvement in brief intervals of time. Tell us how students earned their scores (qualitative information). Tell us how students earned their scores (qualitative information). Designed to be as short as possible to ensure “do-ability.” Designed to be as short as possible to ensure “do-ability.” Are linked to decision making for promoting positive achievement and problem-solving. Are linked to decision making for promoting positive achievement and problem-solving. Adapted from www.aimsweb.com

21. 21 Curriculum Based Measurement CBM has been shown to posses high levels of reliability CBM has been shown to posses high levels of reliability –Reliability - the extent to which the measurements of a test remain consistent over repeated tests of the same subject under identical conditions 42 one-minute CBM type assessments in reading, math, and written expression for grade K-5 were found to have reliability coefficients between.90-.99 with just three one-minute administrations (Jenkins, 2002) 42 one-minute CBM type assessments in reading, math, and written expression for grade K-5 were found to have reliability coefficients between.90-.99 with just three one-minute administrations (Jenkins, 2002)

22. 22 Curriculum Based Measurement Discriminant Validity - Does it appear to measure what it’s supposed to measure? Discriminant Validity - Does it appear to measure what it’s supposed to measure?And Doesn’t associate with constructs that shouldn’t be related. Several studies have demonstrated the ability of CBM to differentiate between students receiving special education services, students receiving Chapter 1 services, and students not receiving any of those services (Deno, Marston, Shinn, and Tindal, 1983; Marston and Deno, 1982; Shinn and Marston, 1985; and Shinn, Tindal, Spira, and Marston, 1987). Several studies have demonstrated the ability of CBM to differentiate between students receiving special education services, students receiving Chapter 1 services, and students not receiving any of those services (Deno, Marston, Shinn, and Tindal, 1983; Marston and Deno, 1982; Shinn and Marston, 1985; and Shinn, Tindal, Spira, and Marston, 1987).

23. 23 NC CBM Norming Project Completed during the 2004-05 school year Completed during the 2004-05 school year 3 days in a row in October, January, and April 3 days in a row in October, January, and April Each pilot county (New Hanover, Bertie, Harnett, Guilford, Burke) completed a project for K through 5 th grades Each pilot county (New Hanover, Bertie, Harnett, Guilford, Burke) completed a project for K through 5 th grades Random sample, 120 students per grade level Random sample, 120 students per grade level Probes were given in reading, math, and writing (Basic Skill Builders) Probes were given in reading, math, and writing (Basic Skill Builders)

24. 24 North Carolina Norms Practice Norm (mean) for a kindergartener in number identification in the fall Norm (mean) for a kindergartener in number identification in the fall Norm for a fifth grader in written expression in the spring Norm for a fifth grader in written expression in the spring Norm for a first grader in sight word reading in the winter Norm for a first grader in sight word reading in the winter Norm for a first grader in sight word reading in the spring Norm for a first grader in sight word reading in the spring

25. 25 Other types of CBMs - DIBELS Dynamic Indicators of Basic Early Literacy Skills Dynamic Indicators of Basic Early Literacy Skills Assesses similar skills as Skill Builders, a little more in depth Assesses similar skills as Skill Builders, a little more in depth Benchmark assessment data is available (national data) Benchmark assessment data is available (national data) Can use for benchmark assessments (3 or 4 times a year) as well as for progress monitoring Can use for benchmark assessments (3 or 4 times a year) as well as for progress monitoring

26. 26 Other types of CBMs - DIBELS Kindergarten – Fall Kindergarten – Fall –Initial Sound Fluency –Letter Naming Fluency Kindergarten – Winter Kindergarten – Winter –Initial Sound Fluency –Letter Naming Fluency –Phoneme Segmentation Fluency –Nonsense Word Fluency Kindergarten – Spring Kindergarten – Spring –Letter Naming Fluency –Phoneme Segmentation Fluency –Nonsense Word Fluency

27. 27 Other types of CBMs - DIBELS First – Fall First – Fall –Letter Naming Fluency –Phoneme Segmentation Fluency –Nonsense Word Fluency First – Winter First – Winter –Phoneme Segmentation Fluency –Nonsense Word Fluency –Oral Reading Fluency First – Spring First – Spring –Phoneme Segmentation Fluency –Nonsense Word Fluency –Oral Reading Fluency

28. 28 Other types of CBMs - DIBELS Second – Fall Second – Fall –Nonsense Word Fluency –Oral Reading Fluency Second – Winter Second – Winter –Oral Reading Fluency Second – Spring Second – Spring –Oral Reading Fluency

29. 29 Other types of CBMs - AIMSweb Similar reading probes to DIBELS Similar reading probes to DIBELS Adds a reading comprehension piece (MAZE) Adds a reading comprehension piece (MAZE) Also has math computation, early numeracy and writing components Also has math computation, early numeracy and writing components Product is for purchase, probes not available for free Product is for purchase, probes not available for free

30. Other types of CBMs: www.interventioncentral.org Website has many CBM probes available for free Website has many CBM probes available for free You can create multiple forms of early literacy and numeracy probes You can create multiple forms of early literacy and numeracy probes Many national norms available for comparison Many national norms available for comparison 30

31. Other types of CBMs: www.interventioncentral.org Numberfly Numberfly Create probes for Quantity Discrimination, Missing Number, Number Identification Create probes for Quantity Discrimination, Missing Number, Number Identification 31

32. ChartDog ChartDog Create charts for student progress Create charts for student progress 32 Other types of CBMs: www.interventioncentral.org

33. 33 Wrap Up of CBM Pros and Cons of each measure we discussed Pros and Cons of each measure we discussed How to use these measures as a school-wide screening tool (universal screening) How to use these measures as a school-wide screening tool (universal screening) Benefits of a using universal screening Benefits of a using universal screening Need for a good data management system Need for a good data management system Think about going paperless! Think about going paperless!