Incorporating Student Progress Monitoring in Teacher Education Courses Dr. Pam Fernstrom Chaney University of North Alabama

Webinar Objectives Webinar participants will be able to: Explain the differences between Student Progress Monitoring/ Curriculum-Based Measurement (SPM/CBM) and traditional assessments used in classrooms Discuss the rationale for including SPM/CBM in teacher education programs/courses Examine options for infusing SPM/CBM content & possible types of courses Identify relevant SPM/CBM content for their programs/courses Review sample in-class assignments and field experiences objectives & student artifacts Obtain materials & resources

How Are SPM/CBM Assessments Different from Traditional Assessments? Traditional assessments used in P-12 classrooms: Teacher-developed summative tests (typically have poor technical quality) Progress monitoring = 2 or 3 benchmarks across academic year Norm-referenced tests (NRTs) One time testing High-stakes tests (state-mandated) - typically group

How is SPM/CBM Different from Traditional Assessments Used? (cont.) Teachers often use mastery measurement for evaluating student performance on skill(s) being instructed. To implement mastery measurement, the teacher: Determines the sequence of skills in an instructional hierarchy. Develops, for each skill, a criterion-referenced test.

Hypothetical Fourth-Grade Math Computation Curriculum 1. Multidigit addition with regrouping 2. Multidigit subtraction with regrouping 3. Multiplication facts, factors to nine 4. Multiply two-digit numbers by a one-digit number 5. Multiply two-digit numbers by a two-digit number 6. Division facts, divisors to nine 7. Divide two-digit numbers by a one-digit number 8. Divide three-digit numbers by a one-digit number 9. Add/subtract simple fractions, like denominators 10. Add/subtract whole numbers and mixed numbers

Multidigit Addition Mastery Test

Mastery of Multidigit Addition

Hypothetical Fourth Grade Math Computation Curriculum 1. Multidigit addition with regrouping 2. Multidigit subtraction with regrouping 3. Multiplication facts, factors to nine 4. Multiply two-digit numbers by a one-digit number 5. Multiply two-digit numbers by a two-digit number 6. Division facts, divisors to nine 7. Divide two-digit numbers by a one-digit number 8. Divide three-digit numbers by a one-digit number 9. Add/subtract simple fractions, like denominators 10. Add/subtract whole numbers and mixed numbers

Multidigit Subtraction Mastery Test Name: Date: 6 5 2 1 3 7 4 9 8 Subtracting

Mastery of Multidigit Addition and Subtraction 10 8 6 4 2 WEEKS Number of problems correct in 5 minutes Multidigit Addition Multidigit Subtraction 12 14 Multiplication Facts Number of digits correct

Problems with Mastery Measurement Hierarchy of skills is logical, not empirical. Performance on single-skill assessments can be misleading. Assessment does not reflect maintenance or generalization. Assessment is designed by teachers or sold with textbooks, with unknown reliability and validity. Number of objectives mastered does not relate well to performance on high-stakes tests

In Contrast, SPM/CBM… Focuses on general outcome measures, rather than assessing only the skill(s) currently taught. Involves standardized procedures for test development, administration, scoring, and decision making. Provides a reliable and valid way for monitoring student progress at regular intervals across the year. Provides parallel and brief measures Uses data to determine goals Displays data graphically SPM/CBM correlates highly with high-stakes tests

Rationale for Including SPM/CBM in Teacher Education Programs/Courses

Rationale for Inclusion of SPM/CBM CBM is a scientifically validated form of progress monitoring Over 30 years of research support the use of CBM to… Increase student achievement (Fuchs, Deno, & Mirkin, 1984; Stecker, Fuchs, & Fuchs, 2006) Make predictions about who will succeed on high-stakes assessments (Good, Simmons, & Kameenui, 2001) Help teachers identify when instructional changes are needed (Fuchs, Fuchs, & Hamlett, 1993) Develop classroom, school, or district norms (Shinn, 2002) Increase ease of communication with parents, teachers, students, and others (Shinn, Habedank, & Good, 1993)

Rationale for Inclusion of SPM/CBM (cont.) Increased assessment knowledge base beyond NRTs & teacher-developed assessments Data-based instructional decisions for P-12 students rather than traditional approaches such as mastery measurement.

Rationale for Inclusion of SPM/CBM (cont.) SPM/CBM data may be used in multiple ways within a school and/or classroom Special education decision-making: monitoring IEP progress and eligibility for services Monitoring prereferral interventions Monitoring ALL students’ progress in the general education curriculum by student, class, grade, and/or school An alternate assessment for students unable to take state-mandated assessments School improvement plans

Rationale for Inclusion of SPM/CBM (cont.) SPM/CBM data may be used in multiple ways within a school and/or classroom (cont.) CBM provides an easy and quick method for gathering student progress Student data can be compared to teacher’s classroom or school district data Teachers can compare the efficacy of different forms of instruction and design more effective, individualized instructional programs for students with learning problems.

Rationale for Inclusion of SPM/CBM (cont.) SPM/CBM data may be used in multiple ways within a school and/or classroom (cont.) Teachers can analyze student scores and adjust student goals and instructional programs Administrators & teachers may determine Adequate Yearly Progress (AYP) Individual teachers & teams may use SPM/CBM data to determine Responsiveness to Intervention (RtI) All of the above lead to data-based decision-making rather than subjective!

Rationale for Inclusion of SPM/CBM (cont.) Students who score high on SPM/CBM are typically better at: Decoding Sight vocabulary Comprehending material Computing Problem-solving

Rationale for Inclusion of SPM/CBM (cont.) SPM/CBM benefits… ALL Students by providing More accurate identification of performance, difficulties Better instructional programs More intensive instruction General education teachers by providing More sensitive data regarding student performance Greater accountability for student performance Ability to assess effects of their instruction Special education teachers by providing Ability to monitor students’ performance towards meeting IEP goals More accurate identification

Rationale for Inclusion of SPM/CBM To help initial & advanced candidates meet national & state standards/ competencies for teacher education Examples: Initial Candidates: Interstate New Teacher Assessment and Support Consortium (INTASC) Standard 8: The teacher understands and uses formal and informal assessment strategies to evaluate and ensure the continuous intellectual, social, and physical development of the learner.

To help initial & advanced candidates meet national & state standards/ competencies for teacher education (cont.) Alabama Quality Teaching Standards (AQTS)   ALL Initial Candidates should be able to: Ability to collect and use data to plan, monitor, and improve instruction. Ability to adjust instruction in response to information gathered from ongoing monitoring of performance via formative assessment. Ability to design and use a variety of approaches to formal and informal assessment to plan instruction, monitor student understanding and progress toward learning, modify teaching and learning strategies, and measure and report student progress related to learning objectives. Have knowledge of assessment tools to monitor the acquisition of reading strategies, to improve reading instruction, and to identify students who require additional instruction. Ability to use technology to assess student progress and manage records.

To help initial & advanced candidates meet national & state standards/ competencies for teacher education (cont.) Alabama General Special Education Standards: Translate assessment information into functional long-term goals and short-term benchmarks. Implement or assist other teachers in implementing the student's individualized education program, by selecting, developing, and using appropriate instructional techniques and methods, ongoing measurement techniques, media and materials, equipment (including assistive technology devices), technological advances, and support personnel. Continuously analyze the effectiveness of the individualized education program and make appropriate modifications.

To help initial & advanced candidates meet national & state standards/ competencies for teacher education (cont.) Advanced Candidates: National Board for Professional Teaching Standards (NBPTS): NBPTS .3.4: Teachers regularly assess student progress. NBCTs know how to assess the progress of individual students as ell as the class as a whole. NBPTS Exceptional P-12.5: Knowledge of Subject Matter: Accomplished teachers of students with exceptional needs command a core body of knowledge in the disciplines and draw on that knowledge to establish curricular goals, design instruction, facilitate student learning, and assess student progress.

Questions???

SPM/CBM: A Review of the Steps

The Basic SPM/CBM Steps that Teachers Should Know and Be Able to… Step 1: Place students in a SPM/CBM task for progress monitoring by deciding on which measures to use Step 2: Identify the level of material for progress monitoring Step 3: Administer/score SPM/CBM probes and collect screening or baseline data

The Basic SPM/CBM Steps that Teachers Should Know and Be Able to… (Cont.) Step 4: Graph scores Step 5: Set ambitious long range goals, decide on short-term objective or end criteria, & the frequency of monitoring Step 6: Make instructional changes by applying decision-making rules to graphed scores to know when to revise programs & when to increase goals Step 7: Use the SPM database qualitatively to describe students’ strengths & weaknesses

How to Include SPM/CBM in Teacher Education Programs

Courses Appropriate for Integrating SPM/CBM Content General and/or special education assessment courses Reading methods and/or assessment courses Math methods and/or assessment courses Examples of Initial & Advanced Coursework: Evaluation & Remediation of Reading Disabilities Advanced Issues & Procedures in the Assessment of Students with Mild/Moderate Disabilities Assessment and Evaluation in Special Education (dual listing) Reading Instruction for Students with Mild/Moderate Disabilities (dual listing) Assessment of Students with Disabilities (graduate only) Assessment and Instruction of Reading Difficulties

SPM/CBM Content Infusion in Initial & Advanced Programs Unit of Instruction within Specific Course(s) Integrated throughout a Course Emphasis of SPM/CBM Content in Courses

SPM/CBM Content in Initial & Advanced Coursework Common Elements Across Example Courses: Administration & scoring of SPM/CBM probes Data interpretation Instructional decision-making and/or program planning

National Center on Student Progress Monitoring Suggestions for SPM/CBM Content Basics of Curriculum-Based Measurement (CBM) Definition, description of use Differentiate between CBM, standardized assessment, mastery measurement Things to think about when planning to implement SPM/CBM CBM and special populations Implementation Examples Selecting a progress monitoring tool

National Center on Student Progress Monitoring Suggestions for SPM/CBM Content (cont.) Conduct student progress monitoring using CBM Placing a student in a task and identifying the appropriate level of material Administer and score CBM Reading Letter Sound Fluency (LSF) Word Identification Fluency (WIF) Oral Reading Fluency (ORF) Maze

National Center on Student Progress Monitoring Suggestions for SPM/CBM Content (cont.) Math number identification quantity discrimination missing number computation concepts and applications Written Expression Written expression Spelling

National Center on Student Progress Monitoring Suggestions for SPM/CBM Content (cont.) Graph CBM data and use them to monitor progress Graph data points Draw trend line Calculate slope

National Center on Student Progress Monitoring Suggestions for SPM/CBM Content (cont.) Interpret and report on student progress using graphed data Setting goals, goal line by Using benchmark Using intraindividual framework Using norms for rates of improvement (no goal line) Interpreting graphed data by comparing Data points to goal line (latest 4 points method) Trend line to goal line

National Center on Student Progress Monitoring Suggestions for SPM/CBM Content (cont.) Use CBM data to make administrative and instructional decisions Decision-making protocol Raise goal Change instruction Choosing appropriate instructional modifications based on graphed data Using CBM data to set IEP goals and monitor progress Using CBM data to inform Adequate Yearly Progress Analysis

National Center on Student Progress Monitoring Suggestions for SPM/CBM Content (cont.) Using CBM in a Response to Intervention (RTI) framework: tiered decision-making Tier 1: Primary- CBM screening and prevention Identify high-risk students Monitor weekly Tier 2: Secondary Implement Tier 2 intervention Identify non-response Monitor weekly or more Tier 3: Tertiary

Examples of SPM/CBM In Class & Field Experiences Assignments in Initial & Advanced Coursework

Examples of In-Class Activities Instructor provides P-12 student responses to initial & advanced candidates to score. Administer & score SPM/CBM probes to class peers prior to working with P-12 students in field experiences. Graph P-12 student data by hand & using software, set goals, & draw trend lines. Write annual goals & benchmarks. Review data & make data-based decisions individually or as a team. Make recommendations or decisions regarding increasing goals or changes in interventions. Write instructional plans. Respond to SPM/CBM case studies individually or as a team.

Questions???

Field Experience Objectives & Artifact Example 1 Objectives: (1) Identify and administer appropriate assessment procedures to make instructional programming decisions for students with deficits in reading. (2) Identify and implement instructional strategies to improve the phonological awareness, decoding, word recognition, and vocabulary skills of students who are at-risk for or have reading disabilities. Artifact: Demonstrate the use of a data-based approach (Curriculum-based Measurement) to teach reading. Select a student that you can teach on a regular basis--at least 2 to 3 days per week--for at least 15 minutes per day. Specify a measurable goal that is potentially attainable within 8 weeks. Design procedures to measure performance on the skill that can be administered each instructional session.

Example 1 (cont.) Use the measurement procedures to contrast the student’s performance with that of his age/grade peers on that skill. Write an Instructional Plan (IP) for teaching the student the skill you have identified. Implement your IP with the student (i.e., teach the student) for a minimum of 7 weeks. Measure the student’s performance on the skill every session (2 to 3 times per week) and graph the performance data. After 3 to 5 data points, modify the IP in some clear and substantial respect that you hypothesize could be more effective. Plot the trend of the data before and after the intervention. Implement the revised plan and continue to measure and graph performance. Continue the cycle in steps i and j.

Field Experiences Objective & Artifact Example 2 Objective: Use ongoing assessment and student progress monitoring to write IEPs and account for student outcomes. Artifact: Administer at least 5 CBM progress monitoring probes in reading or math and prepare graphs reflecting at least one student’s progress. Completed probes and the graph of results will be turned to be evaluated for accuracy in scoring and recording results on the graph.

Field Experience Objectives & Artifact Example 3 Objectives: (1) Methods for monitoring progress of individuals with exceptional learning needs. (2) Develop individualized assessment strategies for instruction. (3) Demonstrate skill in using productivity tools for professional and personal use, including word processing, database, spreadsheet, graphic utilities, and drawing programs. (4) Demonstrate knowledge of uses of computers for problem solving, data collection, information management, decision making, communications, and development of presentations. Artifact: Students will complete a six-week Curriculum-Based Measurement (CBM) project in either reading or math. The CBM project will contain: (1) baseline data, (2) long-range goal line, (3) short-term objective, (4) at least one change in instruction, (5) a graph of the data, and (6) an instructional plan.

Field Experiences Objective & Artifact Example 4 Objective: develop and administer curriculum-based measures in one academic area and chart student progress making instructional changes when necessary. Artifact: Curriculum-Based Measurement Project - Students will complete a ten-week CBM project on a student in one of the following academic areas: reading, written expression, or math. The CBM project will contain: (1) baseline data, (2) long-range goal line, (3) short-term objective, (4) at least 2 intervention phases (5) a graph of the data, (6) intervention plans.

Where to Obtain Materials for Your Programs/Courses National Center on Student Progress Monitoring (NCSPM) website – Professional Development & Resources tabs http://www.studentprogress.org/default.asp

Materials (cont.)

Materials (cont.)

Materials (cont.)

Materials (cont.)

Materials (cont.) The Research Institute on Student Progress Monitoring's (RIPM) website http://www.progressmonitoring.net/ Intervention Central http://www.interventioncentral.org Numerous textbooks from various publishers

Materials (cont.) Research articles – See NCSPM’s website annotated bibliography & syllabi reading lists The National Center on Student Progress Monitoring’s Tools Chart http://www.studentprogress.org/chart/chart.asp Click on tools’ names to get detailed information and links to vendor websites, where sample probes and reports can be viewed

Materials (cont.)

Materials (cont.)

Materials (cont.)

Questions???