1. Data Driven Decisions & Target Interventions in the Elementary Math Classroom
Cleveland County Schools
Giancarlo Anselmo, Brian Bettis, Carrie Knotts

2. Objectives
Discuss the research behind Curriculum Based Measures (CBMs)
Advantages of CBMs
Critical Features
Reliability and Validity
Development of CBMs
Norms and Growth rates
Universal Screening
Suggestions
Team Initiated Problem Solving
Data Collection and Analysis
Progress Monitoring
Appropriate Targeted Interventions

3. Nations Report Card 2012 NAEP

4. Hierarchy of CBM Research
1st CBM reading elementary level
2nd CBM reading secondary level
3rd CBM math elementary level
4th CBM math secondary level
5th CBM for other subjects (writing, spelling, science, etc.)

5. Curriculum Based Measurement: Advantages
Direct measure of student performance
Helps target specific areas of instructional need for students
Quick to administer
Provides visual representation (reports) of individual student progress and how classes are acquiring essential reading skills
Sensitive to even small improvements in performance
Capable of having many forms
Monitoring frequently enables staff to see trends in individual and group performance—and compare those trends with targets set for their students.
Correlates strongly with “best practices” for instruction and assessment, and research-supported methods for assessment and intervention 5

6. Critical Features of CBM
Technical adequacy
Evaluation of general outcomes
Assess student progress
Stecker et al. 2005

7. Technical Adequacy Data exists for two distinct types of M-CBM
Computation
Concepts and Applications
Math concepts and applications have been shown to be distinct from computation ability (Thurber et al., 2002). Thurber’s study with colleagues used factorial analysis procedures in order to determine the best factorial model for mathematics. Their research contends that the most defensible model for math was one that displayed a two factor model of mathematics assessment where computation and applications were distinct, although highly related (r=.83) (Thurber et al., 2002).

9. Reliability of M-CBM (Basic Facts)
Foegen 2000
Alternate form=
Foegen & Deno 2001
Internal consistency=
Alternate form=
Test-Retest=
Foegen 2008
Alternate form=
Test-Retest=

10. Reliability of M-CBM Concepts and Applications
Helwig & Tindal 2002
Alternate form=
Foegen 2008
Alternate form=
Test-Retest=

11. Validity of M-CBM Concepts and Applications
Helwig, Anderson, & Tindal 2002
Criterion Validity= .80
Foegen 2008
Concurrent Validity=

12. Validity of M-CBM Basic Facts
Foegen 2000
Criterion validity=
Foegen & Deno 2001
Criterion validity=.63
Foegen 2008
Concurrent validity=

13. Different Approaches to Developing M-CBM
Curriculum sampling approach
Robust Indicators approach

14. Curriculum Sampling
Measures are developed by constructing representative samples of the year’s mathematics curriculum
Method is used with both math computation and math applications
Math concepts and applications have been shown to be distinct from computation ability (Thurber et al., 2002). Thurber’s study with colleagues used factorial analysis procedures in order to determine the best factorial model for mathematics. Their research contends that the most defensible model for math was one that displayed a two factor model of mathematics assessment where computation and applications were distinct, although highly related (r=.83) (Thurber et al., 2002).

15. Computation Curriculum
Fourth Grade Math
Computation Curriculum
Multidigit addition with regrouping
Multidigit subtraction with regrouping
Multiplication facts, factors to 9
Multiply 2-digit numbers by a 1-digit number
Multiply 2-digit numbers by a 2-digit number
Division facts, divisors to 9
Divide 2-digit numbers by a 1-digit number
Divide 3-digit numbers by a 1-digit number
Add/subtract simple fractions, like denominators
Add/subtract whole number and mixed number

16. 3rd Grade Common Core Standards

17. Robust Indicators
Measures that are not necessarily representative of a particular curriculum, but are instead characterized by the relative strength of their correlations to various overall mathematics proficiency criteria (Foegen et al., 2007)
The comparison is with ORF as that tends to be a robust indicator

18. Robust Indicators
Little research but research done shows promise for this method
Helwig & Tindal 2002
Took 11 concept grounded math problems and correlated the results the Computer Adaptive Test (state test given in Oregon)
Results suggested correlations for general education students=.80
Lower for students with learning disabilities

19. Cleveland County Math Probes
Used curriculum sampling approach
Designed our own universal screening probes using:
Math-aids.com
Math Concepts and Applications probes were adapted from Monitoring Basic Skills Progress: Basic Math Concepts and Applications
Fuchs, Hamlett, & Fuchs, (1999)
Adapted because probes gave a really good format. Very similar to Aimsweb but didn’t align with Common Core. So our curriculum people went through and tweaked questions to have them align with common core.

20. Local Norms and Growth Rates

21. M-CBM as part of a Three Tiered Model
Tier I-Universal Screening
Tier II-Progressing Monitoring
Tier III-Further assessment as part of a problem solving process

22. Universal Screening
Math assessments are generally done using one probe during universal screening
Hintze et al, 2002
Study showed that one can expect extremely high dependability with as little as one 2-minute multiple-skill math probe
Research was not done at the secondary level but considering technical characteristics are similarly an upward extension does not seem so far off the mark.

23. Universal Screening Which probes to use?
Option 1: Design your own probes
Option 2: Choose a standardized set of published probes

24. Companies that Provide Standardized M-CBM Probes
AimsWeb
Easy CBM
Yearly Progress Pro

25. AIMSweb Math measures for Computation
Mixed computation, grades 1-6
+ facts, - facts, x facts, / facts, +/-mix, mult./div. mix, all mix
Math measures for Concepts and Applications
See table for areas covered

27. AimsWeb

28. Easy CBM Math and Reading Probes from grade 1-8
Probes covering: Number and Operations, Algebra, Measurement, Geometry, and Data/Analysis
Math probes can be taken as a paper and pencil test or taken online

30. Easy CBM Sample

31. Easy CBM and AimsWeb Have established norms
Have Math probes for grades 6-8
Have alternative forms for progress monitoring
Have the capability storing data online for distribution and analysis

32. Overall Suggestions
Have a district level team select measures based on critical criteria such as reliability, validity and efficiency
Select screening measures based on the content they cover with an emphasis on critical instructional objectives for each grade level
In grades 4-8, use screening measures in combination with state testing data
Use the same screening tool across a district to enable analyzing results across schools
Clarke and Baker

33. Now What? Curriculum Based Measure has been selected
Complete Universal Screening 3 times a year. BOY, MOY, EOY
Teachers analyze data looking to answer these questions.
How are all students performing?
Why are there deficits/strengths?
Are students growing?

34. “Prismation” of Data
Multiple Data Sources: Classroom Performance, CFA, CBM, Behavior, Teacher Judgment.
Student’s Targeted Action Plan, customized to meet their individual needs.
No one data source trumps another. They work in conjunction with each other to tailor an action plan for the student.

35. Why?
Determine how well your Foundational Core instructional programs are working for all students-- proficiency and growth
Identify specific skill deficits/strengths of all students
Used as a part of an early warning system

36. Universal Screening Allows Us To….
Problem solve
The whole school
A grade level
A class
Subgroups

37. Team Initiated Problem Solving
Grade Level Teams
Analyze grade level data in conjunction with curriculum coaches
Define the problem
Answer the “why” questions
Design an action plan
Core instruction and interventions

38. Identifying Areas of Need
What are the students’ strengths? Why?
What are their deficits? Why?
Do we need to address this in core instruction?
Do you need to address this with interventions?

39. Example
4th Grade Math CBM data analysis
What would you do?

40. Progress Monitoring

41. Appropriate Targeted Interventions

42. Questions?