Presentation on theme: "Curriculum Based Measurement Grant Wood AEA 10 Austin Beer and Heather Marolf, School Psychologists August 11th, 2010 DIBELS Content Taken from the DIBELS."— Presentation transcript:
Curriculum Based Measurement Grant Wood AEA 10 Austin Beer and Heather Marolf, School Psychologists August 11th, 2010 DIBELS Content Taken from the DIBELS Assessment Manual
Why are CBMs important? Fundamental to the work we do. o Individual student level o System Level o Classroom/Teacher Level
CBM TRAINING AGENDA Overview of CBM and Norms Use of Norms in the Problem Solving process Administration and Scoring o Reading: DIBELS NEXT/Jamestown o Math o Written Language Resources
Iowa Core Curriculum and Instructional Decision Making “IDM” Screening Supplemental Intensive
Goals for Today Increase knowledge about the GW agency norming project and the appropriate use of the normative data. Increase knowledge of how CBM fits with Problem Solving and Prevention. Develop skills for administering and scoring CBM probes. Become more familiar with resources that are available for collecting CBM data. ENJOY our new learning.
Overview Please refer to “Overview” section of the CBM norming project manual for more detailed information
Curriculum Based Measurement (CBM) A set of standard simple, short- duration fluency measures of reading, written expression, and math Dynamic Indicators of Basic Skills General Outcome Measure o Measurement of the “vital signs” of student achievement o “Academic Thermometers ”
CBM - CBA - CBE Curriculum Based Assessment Curriculum Based Measurement CBM Screening & Formative Curriculum Based Evaluation CBE Diagnostic General Curriculum Based Assessment (Classroom-based)
Curriculum Based Assessment (“Umbrella” term) CBA…also includes classroom assessments such as formative assessment CBM (Shinn, Fuchs, DIBELS) o Screening & Formative o ? “Which students are likely to have a skill deficit?” o ? “What is the student’s progress given this instruction?” CBE (Howell) o Diagnostic o ? “What does this student know/not know?” o ? “What instructional strategy addresses this student’s skill deficit?”
Grant Wood AEA Norming Project students per grade level Randomly selected 30 of 33 public school districts participated, all private and parochial schools
Academic Areas Assessed, cont. Written Expression (1st -11th) o Total Words Written (TWW) o Correct Word Sequences (CWS)
Math Concepts: (K-1st) (Rote Counting, Number ID) Computation: (1st-8th) (Fuchs and Fuchs, Blackline Masters) Applications: (2nd -6th) (Fuchs and Fuchs, Blackline Masters) Applications: (7th – 11th ) Retired NAEP questions. (see p.2 & 3 of norms booklet)
Using the CBM norms Screening o Identification of students who are at-risk for academic deficits Formative/Progress Monitoring o Measuring student growth over time o Measuring student progress during supplemental or intensive instruction CBM has limited diagnostic use…CBE is used as diagnostic assessment. o DIBELS NEXT allows for more diagnostic/formative assessment
CBM and Eligibility Appropriate Use o Evidence of resistance to intervention/lack of educational progress o Evidence of significant discrepancy from peers o ALWAYS used as part of the convergence of data (ICEL/RIOT, etc.) Inappropriate Use o A “cut score” that automatically determines eligibility o In isolation, out of the context of the broader problem solving process
Orientation to Norm Tables
Norm Tables: Fall, Winter, Spring Complete listing of all scores and percentile ranks printed in a condensed format. Comparing Scores and Percentiles: o Locate raw scores in norms tables o Locate percentile in norms tables o Locate average range of grade level peers (25th percentile – 75th percentile)
Median Growth Rate per Week Chart Table with median growth rate per week Fall → Winter = 12 weeks Winter →Spring = 14 weeks
Screening Who needs additional instruction?
CBM used as screening assessment… Determine instructional effectiveness Identify the students who are at risk for basic academic deficits Prioritize which students may require additional, in-depth diagnostic assessment “Double check” teacher referral (Problem Validation)
Accessing District Screening Data All districts have screening data…we should help them use it o DIBELS, other reading fluency o Reading Inventories o ITBS/ITED, MAP (NWEA) o District-developed assessments in math or written language Continue to examine district data in conjunction with new AEA CBM norms
Comparing district data with Agency CBM norms ? My district already administers reading fluency probes…can I use AEA norms to help me interpret that data? Can be done…However… Make sure districts are using standardized administration that matches AEA CBM o Time limits, directions, probes o DIBELS likely most consistent
Screening: District/Building Level If… o LEA wants to implement supplemental instruction across grade levels or classrooms…but doesn’t know which kids should be included o Building is adding instruction to core (for example, reading fluency) and wants to examine the effects of this additional instruction
Screening: Classroom/Small Group Level If… o A teacher needs to identify which students need additional instruction in a classroom o A teacher is concerned about a small group of students and wants to prioritize additional resources o A teacher wants to determine if their core instruction is adequate (at least 80% percent of class is proficient)
Screening: Individual Level If… o a teacher is concerned about a particular student o a student has significantly low scores on district-wide assessment (ITBS/ITED, etc.) and you want to collect additional data o you want to re-screen a student who has received supplemental or intensive instruction in the past o you want to quickly screen new students who move into the building
Screening: Activity If… o Discuss at your table how you have used CBMs as part of screening, and how you used the data
Let's Look at our CBM Resources
Reading DIBELS (K-6) o First Sound Fluency o Letter Naming Fluency o Phoneme Segmentation Fluency o Nonsense Word Fluency o Oral Reading Fluency o DAZE Jamestown Timed Readings (7-11) o Oral Reading Fluency
Reading Released in 2010
DIBELS Assessment Flow Chart
DIBELS Benchmark Timeline
Components of a DIBEL's Probe Timed Directions Score Scoring Rules Reminders Wait Rule Discontinue Rule Pen or Pencil Clipboard Stop watch or Timer
Additional Components of a DIBEL's Probe Quiet Testing environment Encouragement Modeling and Practice Repeating directions Articulation and Dialect Do not audio tape
DIBEL's Performance Criteria Benchmarks o Available in August 2010 o Predictive
DIBEL's Time Requirements
First Sound Fluency(FSF) Phonological Awareness Kindergarten: Fall & Winter The assessor will say a series of words one at a time to the student and ask the student to say the first sound in the word. o students receive 2 points for saying the initial phoneme of a word or group of sounds. o Students receive 1 point for saying the initial consonant blend, consonant plus vowel, or consonant blend plus vowel. o Score is total points in 1 minute
FSF: Materials You will need: o FSF Scoring Booklet (Benchmark or Progress Monitoring) o Pencil/Pen o Clipboard o Stop Watch
FSF: Directions Example: o Listen to me say this word, “man.” The first sound that you hear in the word “man” is /mmm/. Listen. /mmm/. “Man.” What is the first sound you hear in the word “man”? o Student gives correct response: Good. /mmm/ is the first sound in “man.” o Student gives any other response:/mmm/ is the first sound you hear in the word “man.” Listen. /mmm/. “Man.” Say it with me./mmm/. Let’s try it again. What is the first sound you hear in the word“man”?
FSF: Administration After administering the 3 practice samples say, "Now I am going to say more words. You tell me the first sound you hear in the word. Proceed with standardized directions (see examiner copy) and timing.
FSF: Timing (VERY Important) After reading the directions begin your stopwatch and time the student for 1 minute. At the end of 1 minute stop presenting words to the student.
FSF: Scoring Example
FSF: Prompt & Discontinue Rule Wait Rule: Wait 3 seconds for the student to respond. Reminders: Can be given as often as needed Discontinue: Score of 0 on first 5 questions, discontinue and give a score of 0 on the cover page. Names Letter vs. saying a sound Say, "Remember to tell me the first sound in the word, not the letter name."
FSF: Additional Scoring Rules Child responds with initial blend vs. initial sound o Score as 1 Child responds with another sound in word or says whole word vs. initial sound o Score as 0 Be careful of vowels (short must be short, long must be long)
FSF: Additional Scoring Rules (Continued) Schwa sounds (/u/) added to first sound o - not counted as an error Articulation difficulty o Student is not penalized o In the case of SEVERE articulation difficulties (student is unintelligible), note and move on. Review Pronunciation Guide
FSF Scoring: General Rule When in doubt: Record student’s response verbatim and review specific rules later.
FSF Practice Activity Modeling Scoring Practice Discussion Review Together
FSF Practice Activity Practice with a partner You will have 2 minutes each.
Letter Naming Fluency Does not correspond to one of the “Big 5” ideas, but provides a measure of risk Kindergarten: Fall, Winter, Spring First Grade: Fall Students are presented with a page of upper- and lower-case letters arranged in random order and are asked to name as many letters as they can.
LNF: Directions I am going to show you some letters. I want you to point to each letter and say its name. (Put the page of letters in front of the student.) Start here (point to the first letter at the top of the page.) Go this way (sweep your finger across the first two rows of letters) and say each letter name. Put your finger under the first letter (point). Ready, begin.
LNF: Administration & Timing Start stopwatch after you say begin. Prompt: If student provides the letter sound rather than the letter name, say “Say the letter name, not its sound." (other prompts listed on pg 5 in the K scorer's booklet) Stop stopwatch at 1 minute and say “Stop”.
LNF: Scoring Score is total # of correct letters in one minute. Discontinue Rule: If the student does not get any correct letter names within the first row (10 letters), discontinue and record a score of 0. 3 second rule: If student hesitates for 3 seconds, mark a slash through the letter and say the correct letter name.
LNF: Additional Scoring Rules Self-correct within 3 seconds o Score as 1 Self- correct after 3 seconds o Score for that letter remains 0 Omissions (except for entire row) o Score as 0 Similar shaped font exception (L and i) o Score as 1
LNF: Additional Scoring Rules (Continued) Articulation and Dialect o Do not penalize Omission of entire row o Do not count row in scoring
Letter Naming Fluency Practice Activity with Partner
Phoneme Segmentation Fluency Phonological Awareness Kindergarten: Winter & Spring First Grade: Fall Examiner orally presents words of 3 or 4 phonemes. The student verbally produces the individual phonemes for the word. Materials: scoring booklet, stopwatch, pen/pencil, clipboard.
PSF: Directions "We are going to say the sounds in words. Listen to me say all the sounds in the word "fan." /f/ /a/ /n/. Listen to another word, (pause) "jump." /j/ /u/ /m/ /p/. Your turn. Say all the sounds in "soap." o Correct Response: "Very good saying all the sounds in "soap." At this point, begin testing. Any Other Response: "I said "soap" so you say /s/ /oa/ /p/. Your turn. Say all the sounds in "soap." o Correct response - say "good" and begin testing. o Incorrect response - say "okay" and begin testing.
PSF: Administration & Timing Begin testing, say: "I am going to say more words. I'll say the word and you say all the sounds in the word." After saying first word, start your stopwatch. 3 second rule: If the student does not say a sound segment after 3 seconds, give next word and score the “unsaid” segments as 0. As soon as the student finishes saying the sound, or 3 seconds have elapsed, say the next word promptly and clearly. At the end of 1 minute, stop presenting words and scoring further responses.
PSF: Scoring Discontinue Rule o If a student has not given any sound segments correctly in the first 5 words, discontinue the task and record a score of zero. General Scoring Rule o Students receive 1 point for every different, correct, part of the word. Recording Responses o Use underline for correct and slash for incorrect o If unsure, record student's response verbatim and review rules following assessment. Reminders - each allowed 1 time only o If student spells the word - "Say the sounds in the word" o if repeats the word say - "Remember to say all the sounds in the word."
PSF: Recording Errors Slash through incorrect sound segments Circle words student simple repeats Leave omitted sounds blank. Write "sc" over self corrections occuring within 3 seconds.
Phoneme Segmentation Fluency Practice Activity with Partner
Nonsense Word Fluency Alphabetic Principle (phonetic skills) Kindergarten: Winter & Spring First Grade: Fall, Winter & Spring Second Grade: Fall Student is presented a sheet of paper with randomly ordered VC and CVC nonsense words and asked to produce verbally the individual letter sound of each letter or read the whole word.
NWF: Directions We are going to read some make-believe words. Listen. the word is "sog." (Run your finger under the word as you say it.) The sounds are /s/ /o/ /g/ (point to each letter). Your turn. Read this make-believe word (point to the word "mip"). If you can't read the whole word, tell me any sounds you know.
NWF: Directions (Continued) Correct Response (Whole word): Very good reading the word "mip" Correct Response (Letter sounds): Very good /m/ /i/ /p/ (point to each letter) or "mip" (run your finger under the word as you say it.) Incorrect response: Listen. /m/ /i/ /p/ or "mip". (Run your finger under the letters as you say the sounds.) Your turn. Read this make-believe word. (Point to the word "mip") If you can't read the whole word, tell me any sounds you know.
NWF: Administration Begin testing. I would like you to read more make-believe words. Do your best reading. If you can't read the whole word, tell me any sounds you know. (Place the student copy in front of the student.) Put your finger under the first word. Ready, begin.
NWF: Timing & Discontinue Rule Start stopwatch after saying “Begin.” Stop stopwatch at 1 minute and say, “Stop” If the student does not get any sounds correct in first row, discontinue and record a score of 0.
NWF: 3 Second Rule Sound by Sound: o Score the letter sound incorrect, provide the correct letter sound. o Word by Word: Score the word incorrect, provide the correct word. If providing the correct letter-sound or word does not prompt the student to continue, say "Keep going."
NWF: Scoring Correct Letter Sounds (CLS): The student recieves credit for 1 CLS for each correct letter sound read in isolation or read as part of a make-believe word. Whole Word Read (WWR): The student receives credit for 1 WWR for each whole word read correctly without first being sounded out.
NWF: Additional Scoring Rules Partially correct words o 1 point for each correct sound Repeated sounds o Repeated sounds counted only once (2 points max for a VC word, 3 points max for a CVC word) Incorrect sound order (sound by sound) o IF student points to the correct sound as they say that sound, 1 point
NWF: Additional Scoring Rules (Continued) Sound order (word by word) o Blended letter sounds must be correct sound and in the correct place to receive credit Insertions o Ignore Articulation o Do not penalize
NWF: Additional Scoring Rules (Continued) Self-correct o Within 3 seconds, count as correct Skip a row o Do not count that row in scoring
NWF: Response Patterns: Says correct sounds out of order (sound by sound) Makes random errors Says correct sounds, does not recode Says correct sounds, recodes out of order Says correct sounds, recodes with incorrect sounds(s) Says correct sounds and correctly recodes Doesn't tract correctly Tries to turn nonsense words into real words Makes consistent errors on specific letter sounds(s)
Nonsense Word Fluency Practice Activity with Partner
Oral Reading Fluency Advanced phonics, word attack, fluent reading of connected text DIBELS: Middle of 1st grade - 6th grade Jamestown Readers: one minute probes, record median score Materials: scoring booklet, student materials, clipboard, pen/pencil, stopwatch Words per minute - fluency and accuracy.
ORAL READING FLUENCY Administration & Scoring Directions: "I would like you to read a story to me. Please do your best reading. If you do not know a word, I will read the word for you. Keep reading until I say stop." ( 1 minute per probe) DO NOT READ THE TITLE Give 3 passages. Score passages immediately after administration. Tally any errors made by the student while reading.
ORF information Correct words: Total words read - total errors (use the median score Additional information Accuracy median words correct X 100 median words correct + median errors
Important Reminders 3 second wait rule If a student stops reading and it's not a hesitation, say "keep going." If a student loses his/her place, point to the correct spot. Students are NOT penalized for differences in pronunciations due to dialect, articulation delays or impairments, or for pronunciations due to speaking a first language other than English.
Pronunciation Differences Local language norm: Example: The man washed the car. (CWR=5) read as… The man “worshed” the car. (CWR=5) Speech sound production: Example: The rabbit sniffed the carrots. (CWR=5) read as… The “wabbit” sniffed the “carwets”. (CWR=5)
Scoring Considerations: Discontinue if the student does not read any words correctly in the first line of the first passage. Record “0” on the scoring sheet. If the student reads fewer than 10 words correct on the first passage, record the score on the scoring sheet and do not administer passages 2 and 3. If student finishes passage in less than 1 minute, stop and record that score.
Approved Accommodations 1. Enlarged print 2. Use of colored overlays, filters, lighting. 3. Use of assistive technology or assistive lighting devices 4. Use of marker or ruler to focus attention
Scoring Rules and Examples
Scoring Rule 1: Leave blank any words the student reads correctly. Inserted words are not counted. To be counted as correct, words must be read as whole words and pronounced correctly for the context of the sentence.
Scoring Rule 2 - marking errors
ORF Response Patterns Reads with appropriate phrasing, intonation/expression, and observed punctuation. Self-corrects/monitors meaning Shows automaticity on re-read words Uses effective decoding strategies Errors preserve passage meaning Errors violate passage meaning Frequently omits words or letters Frequently adds words or letters Frequent errors on sight words (e.g., I, was, and, the, etc.) Frequent errors on phonetically regular words (cat, milk, etc) Frequent errors on phonetically irregular words Skips lines Other
Time to Practice Practice Sheets - brief passage - student response -scoring practice
DAZE What is Reading Comprehension? New DIBELs Assessment Measures Reading Comprehension 3rd grade through 6th grade Score is the total correct words in 3 minutes minus half the number of incorrect words. The only group adminsitered DIBELS NEXT measure Materials: Student Worksheets, Pen/Pencil, Daze Benchmark Assessment Administration Directions and Scoring Keys, and Clipboard and stopwatch.
Daze Administration Directions: Before handing out the worksheets, say, "I am going to give you a worksheet. When you get your worksheet, please write your name at the top and put your pencil down." When all of the students are ready, say, "You are going to read a story with some missing words. For each missing word there will be a box with three words. Circle the word that makes the most sense in the story. Look at Practice 1. Listen. After playing in the dirt, Sam went (pause) home, summer, was (pause) to wash her hands. You should circle the word “home” because “home” makes the most sense in the story. Listen. After playing in the dirt, Sam went home to wash her hands. Now it is your turn. Read Practice 2 silently. When you come to a box, read all the words in the box and circle the word that makes the most sense in the story. When you are done, put your pencil down.
Daze Administration: As soon as all students have their pencils down, say, "Listen. On her way home, she (pause) chair, sleep, saw (pause) an ice cream truck. You should have circled “saw” because “saw” makes the most sense in the story. Listen. On her way home, she saw an ice cream truck. When I say “begin,” turn the page over and start reading the story silently. When you come to a box, read all the words in the box and circle the word that makes the most sense in the story. Ready? Begin." Start your stopwatch after you say “begin.” Monitor students to ensure they are reading and circling the words. Use the reminders as needed. At the end of 3 minutes, stop your stopwatch and say, "Stop. Put your pencil down." Collect all of the Daze worksheet packets.
Daze Scoring: Calculate and record scores o correct and incorrect Daze Adjust Score, adjusts for guessing o Daze Adjusted Score Formula Correct Responses - Incorrect Responses ÷ 2 Then round to the nearest whole number Do not record negative numbers
DAZE Important Reminders If student read out loud say, "Remember to read the story silently." If a student is not working say, "Remember to circle the word in each box that makes the most sense int he story." Use this prompt as often as needed.
Math Math Rote Counting (K-1) Math Number Identification (K-1) Math Calculation (1-8) Math Application (2-11)
Math Rote Counting Scoring Record the last correct number said in one minute. If the student makes a mistake in counting sequence, stop the test at that time and record the last correct number stated in the sequence. Example: 1, 2, 3, 4, 5, 7, 8. The score would be 5.
Math Number Identification Materials Stopwatch Number sheet for student. Number sheet for examiner. Directions
Math Number Identification Administration Individually administered Read through directions
Math Number Identification Scoring As you are administering the probe, tally the incorrect responses. At the end of one minute how many numbers have been presented. Subtract the number of mistakes from total presented.
Math Computation: 1-8 Materials Stopwatch Probe pencil Administration Can be group administered. Times: Use chart Directions: need to look ahead for type of problem.
Math Computation: 1-8 Scoring Compare the student response with the scoring template. Only score the numbers in the students answer (NOT WORK). If the answer is correct, record the total correct digits (CD).
Math Computation: 1-8 If the answer is incorrect… o Compare each number and count any numbers correct. USE ARROWS! o When scoring addition, subtraction, and multiplication compare template answer to students response from right to left.
Math Computation: 1-8 When scoring division compare students response to the template from left to right.
Math Computation: 1-8 Example 23 Answer: 23 (2 CD) 8 184 Students Response: 21 (1 CD) 16 Students Response: 230 (2 CD) 24 Students response: 123 (0 CD) 24 0
Math Computation: 1-8 Division problems with remainders: Score division from left to right but score the remainder from right to left (because the student would likely subtract to calculate the remainder).
Math Computation: 1-8 Example Answer: 403 R 52 (5 CD) Student response: 41 R 402 (2 CD) Student response: 133 R 5 (1 CD)
Math Computation: 1-8 Decimals Start at the decimal point and work outward in both directions. Decimal points are not counted in correct digits.
Math Application: 2-11 Materials Stopwatch. Probe for student. Pencil for student. Calculator (7-11).
Math Application: 2-11 Administration Can be group administered. Time Limits: see chart. Directions: Different for 7-11 MISTAKE: 6th grade #13 answer is 15.
Math Application: 2-11 Scoring Compare the student’s response with the scoring template. Students receive one point for each blank. Answer is either right or wrong, based on the scoring template.
Math Application: 2-11 Scoring When a dollar sign is needed in the answer it must be present or it is incorrect. On the 7-11 if there are no blanks then one blank is assumed.
Calculation and Application Reversed/Rotated digits Reversed digits are counted as correct. Rotated digits with the exception of 6 and 9 are correct. Reversed $ is counted as correct.
Math Practice Exercise? Questions
Written Language MATERIALS Story starter Lined paper for student responses Stopwatch pencil
WRITTEN LANGUAGE Administration Can be group administered. Read specific directions. Time: o 1 minute to think after directions o 3 minutes to write
Scoring TWW o Count the total of number of words written during the 3 minute period. o Include words that are spelled incorrectly. o All words are counted regardless of spelling or context. The bouy runned dun the sidwalk. (TWW = 6) *If it appears to be a word unit, count it.
What is a Correct Word Sequence? Two adjacent writing units (word/word or word/punctuation) that are acceptable within the context of what is written. ^The^sky^is^blue^. CWS=5 ^The^sky^is blew. CWS=3
Scoring Rules for CWS: Rule 1. Capitalization at the beginning of a sentence. The first word of a sentence must be capitalized. Words that should not be capitalized must begin with a lower-case letter. ex: ^Let’s^go^swimming^today^. CWS=5 ex: let’s go^swimming^today^. CWS=3
Scoring Rules for CWS: Rule 2. Capitalization within a sentence. 2a. Proper nouns must be capitalized for a correct writing sequence to be scored. ex: ^She^asked^me^to^give^the^book^to^Nate^. CWS=10 ex: ^She^asked^me^to^give^the^book^to nate. CWS= 8
Scoring Rules for CWS: Rule 2. Capitalization within a sentence. 2b. Words that should not be capitalized for a correct writing sequence to be scored. ex: ^The^monkey^ate^a^banana^while ^swinging^from^the^tree^. CWS=11 ex: ^The Monkey ate^a Banana while^swinging ^from^the^tree^. CWS=8
Scoring Rules for CWS: Rule 3. Punctuation at the end of a sentence. Correct punctuation must be present at the end of a sentence for a correct writing sequence to be scored. ex: ^I^could^see^new^planets^. ^It ^would^be^fun^in^space^. CWS=13 ex: ^I^could^see^new^planets ^It^would ^be^fun^in^space CWS=11
Scoring Rules for CWS (Rule 3, cont.) ex: ^The^dog^ran^away^. ^The^boy^chased^him^. CWS=10 ex: ^The^dog^ran^away the boy^chased^him^. CWS=7
Scoring Rules for CWS: Rule 4. Punctuation within a sentence. 4a. Commas are not counted as part of the correct writing sequences total except when included in a series of nouns. When part of a series, commas must be used correctly for a correct writing sequence to be scored. ex: ^Next,^take^out^the^garbage^. CWS=6 ^Next^take^out^the^garbage^. CWS=6
Scoring Rules for CWS, cont. Rule 4a, cont. When part of a series, commas must be used correctly for a correct writing sequence to be scored. ex: ^Max^went^to^the^store^to^buy ^bread,^milk,^and^cheese^. CWS=12
Written Language Practice Exercise? Questions
RESOURCES DIBELS NEXT o Benchmarking probes o Progress Monitoring probes Fuchs &Fuchs, Monitoring Basic Skills Progress (MBSP) Jamestown Timed Readings, 4th Edition Easy CBM o Reading o Math Math Algebra Probes