Oregon Reading First DIBELS Mentor Coach Training Presented by Hank Fien Developed by Kame’enui, Simmons, Coyne & Harn (2003) University of Oregon Institute on the Development of Educational Achievement (IDEA)

Objectives Brief review of DIBELS scoring and administration rules and practice each measure. DIBELS Website Enabling Classrooms and Student Database DIBELS Data Entry Generating Data Data Collection Strategies

Assessing Each Big Idea with DIBELS DIBELS Measure Phonological Awareness Initial Sounds Fluency (ISF) Phonemic Segmentation Fluency (PSF) Alphabetic Principle Nonsense Word Fluency Measure (NWF) Fluency and Accuracy Oral Reading Fluency (ORF) Vocabulary Word Use Fluency Comprehension Oral Retell Fluency

Standard Administration For scores to be useful, we must administer the measures according to standardized administration and scoring directions. Presenting each measure: Present the directions as written Use the specific materials Timing each measure: Use a stopwatch Scoring each measure: Follow scoring rules for each measure Score immediately after completing Standardization provides each child an equal opportunity to display skills. Engage student to do his or her best

Separate Testing and Teaching Time Scores will be used to assist in making instructional decisions Therefore, we must administer the measures without: Assisting the student during the task Modifying the task, materials, or time Standardized, reliable data collection and scoring are essential!

Phonemic Segmentation Fluency (PSF) What important skill does it assess? Phonological Awareness The ability to hear and manipulate sounds in words at the phrase level What is the appropriate time and grade? Mid-year kindergarten through first grade What is the goal? How well? 35 phonemes or more By when? End of kindergarten

PSF Video

How Do We Administer and Score the PSF Measure? Materials: Examiner copy of word list with phoneme scoring columns. Student has no materials when assessing phonological awareness. Stopwatch Pencil Preparing the Student: Good testing conditions (e.g., lighting, quiet, comfortable) Provide model in standardized manner and follow correction procedures as necessary

PSF Essentials Stopwatch: Scoring: Maintaining momentum: Discontinue: Present the first word and start the stopwatch and time for 1 minute. Scoring: Underline each different, correct sound segment produced. (See specific scoring rules and examples.) Put a slash (/) through sounds produced incorrectly. Maintaining momentum: As soon as the student is finished saying the sounds, present the next word. Allow the student 3 seconds for each sound segment. Discontinue: If a student has not given any correct sound segments in the first 5 words, discontinue the task and record a score of zero (0). Ending testing: At the end of 1 minute, stop timing and calculate the number of correct phonemes per minute.

Scoring Rules for PSF Correct Segmentation: A correct sound segment is any different, correct part of the word. For example, the sound /t/ is a correct segment of "trick", as are /tr/ and /tri/ (see rule 2, following page). Examiner says "trick," student says "t...r...i...k" Examiner says "cat," student says "k...a...t" STUDENT SCORING CORRECT WORD: SAYS: PROCEDURE: SEGMENTS trick “ t...r...i...k ” /t/ /r/ /i/ /k/ 4/4 cat “ k...a...t ” /k/ /a/ /t/ 3/3

Elongating Sounds Correct Segmentation: No need for an audible pause between the sounds to receive credit. If you can hear each individual sound when the student runs them together, score each sound as correct. Use your professional judgment based on the response and your knowledge of your program. If still not sure, do not give credit STUDENT SCORING CORRECT WORD: SAYS: PROCEDURE: SEGMENTS rest “rrrreeeessssttt” /r/ /e/ /s/ /t/ 4 /4

Errors in Segmenting: No Segmentation If student repeats the entire word, no credit is given for any correct parts. Circle the word to indicate no segmented response was given. STUDENT SCORING CORRECT WORD: SAYS: PROCEDURE: SEGMENTS trick “trick” /t/ /r/ /i/ /k/ 0/4 cat “cat” /k/ /a/ /t/ 0/3

Errors in Segmenting: Incomplete Segmentation Student is given partial credit for each sound segment produced correctly, even if student has not segmented at the phoneme level. The underline indicates the size of the sound segment. For example: Examiner says “trick,” student says “tr...ick” Examiner says “cat,” student says “c...at” STUDENT SCORING CORRECT WORD: SAYS: PROCEDURE: SEGMENTS trick “ tr ... ik” /t/ /r/ /i/ /k/ 2/4 cat c … at” /k/ /a/ /t/ 2/3

Errors in Segmenting: Overlapping Sounds Student receives credit for each different, correct sound segment of the word. Underline the different sound segments produced For example: Examiner says “trick,” student says “tri...ick” Examiner says “cat,” student says “c...cat” STUDENT SCORING CORRECT WORD: SAYS: PROCEDURE: SEGMENTS trick “ tri... ick” /t/ /r/ /i/ /k/ 2/4 cat c … cat” /k/ /a/ /t/ 1/3

Errors in Segmenting: Omission of Sounds Student does not receive credit for sound segments not produced. If student provides the initial sound only, be sure to wait 3 seconds for elaboration. STUDENT SCORING CORRECT WORD: SAYS: PROCEDURE: SEGMENTS trick “ t... ik” /t/ /r/ /i/ /k/ 2/4 cat c” (3 seconds) /k/ /a/ /t/ 1/3

Errors in Segmenting: Mispronunciation of Sounds Student does not receive credit for sound segments that are mispronounced. Put a slash (/) through the incorrect sounds. For example, there is no /ks/ sound in the word "trick." STUDENT SCORING CORRECT WORD: SAYS: PROCEDURE: SEGMENTS trick “ t...r...i... ks” /t/ /r/ /i/ /k/ 3/4 cat b… a...t” /k/ /a/ /t/ 2/3

Student Characteristics Pronunciation & Dialect: Student is not penalized for imperfect pronunciation due to dialect or articulation. For example, if the student says /r/ /e/ /th/ /t/ for "rest" because of articulation difficulties, give full credit. Use professional judgment and prior knowledge of the student’s speech pattern to assess skill performance.

Student Characteristics Schwa Sounds: Schwa sounds (/u/) added to consonants are not counted as errors. STUDENT SCORING CORRECT WORD: SAYS: PROCEDURE: SEGMENTS trick “ tu ... ru ...i... ku” /t/ /r/ /i/ /k/ 4/4 cat ku ...a... tu” /k/ /a/ /t/ 3/3

Breakout Activity Form a 3-person group Assign roles: Examiner Student Observer Practice administering measure (Each person administer the measure 2 times)

Initial Sounds Fluency (ISF): What important skill does it assess? Phonological Awareness The ability to hear and manipulate sounds in words. What is the appropriate time and grade? Beginning of the year, kindergarten What is the goal? How well? 25 phonemes or more By when? Middle of kindergarten

What ISF Looks Like

How Do We Administer and Score the ISF Measure? Materials: Examiner probe Student picture pages Stopwatch Pencil Preparing the student: Good testing conditions (e.g., lighting, quiet, comfortable) Provide model in standardized manner and follow correction procedures as necessary

How Do We Administer and Score the ISF Measure? Place student copy of 4 randomized pictures in front of child. Say these specific directions to the child: “This is mouse, flowers, pillow, letters (point to each picture while saying its name). Mouse (point to mouse) begins with the sound /m/. Listen, /m/, mouse. Which one begins with the sounds /fl/?"

How Do We Administer and Score the ISF Measure? Correct Response on Sample Item: Student points to flowers, you say: “Good. Flowers begins with the sounds /fl/.” Incorrect Response: “Flowers (point to flowers) begins with the sounds /fl/. Listen, /fl/, flowers. Let's try it again. Which one begins with the sounds /fl/?”

How Do We Administer and Score the ISF Measure? "Pillow (point to pillow) begins with the sound /p/. Listen, /p/, pillow. What sound does letters (point to letters) begin with?" Correct Response: If the student says /l/ you say: “Good. Letters begins with the sound /l/.” Incorrect Response: If the student says any other response, you say: “Letters (point to letters) begins with the sound /l/. Listen, /l/, letters. Let's try it again. What sound does letters (point to letters) begin with?” Then you say: "Here are some more pictures. Listen carefully to the questions."

Maximizing Administration Time Stopwatch: Read the question, start stopwatch. After child gives response, stop stopwatch. Record the total time to answer each of the 16 questions. When the examiner is talking, the watch is not running. Scoring: Score is correct or incorrect (see specific scoring rules and examples). Maintaining momentum: Make sure to introduce each picture page. Allow student 5 seconds to answer each question. Discontinue: If a student gets no items correct in the first 5 items, discontinue the task and record a score of zero (0). Ending testing: After administering all 16 items, record the total duration of thinking/response time found on your stopwatch. Count number of items correct. Calculate final score (see formula).

Scoring Rules for ISF Identification Responses (“Which picture begins with…?”) If the child points to the correct picture or names it, score as correct. If the child names or renames the picture with a word that begins with the target sound, score as correct.

Scoring Rules for ISF Production Responses (“What sound does …. begin with?”) Correct Initial Sound or Sounds: If the word starts with an initial consonant sound, the child can respond with the first consonant or consonant-consonant blend. For example, if the word is “clock,” a correct initial sound would be /c/ or /cl/. The student must give the sound, not the letter name.

Breakout Activity Form a 3-person group Assign roles: Examiner Student Observer Practice administering measure (Each person administer the measure 2 times)

Nonsense Word Fluency (NWF): What important skill does NWF assess? Alphabetic Principle:The ability to associate sounds with letters and use these sounds to read words. What is the appropriate time and grade? Middle of the year in kindergarten and throughout first grade What is the goal? First Grade: How well? 50 letter-sounds or more By when? Middle of first grade Kindergarten: How well? 25 letter-sounds or more by end of kindergarten

What NWF Looks Like

How Do We Administer and Score the NWF Measure? Materials: Examiner probe Student pages (practice page “sim lut” and test page) Stopwatch Pencil Preparing the student: Good testing conditions (e.g., lighting, quiet, comfortable) Provide the model in standardized manner and follow correction procedures as necessary

How Do We Administer and Score the NWF Measure? Say these specific directions to the child: “Look at this word (point to the first word on the practice probe). It’s a make-believe word. Watch me read the word: (point to the letter “s”) /s/, (point to the letter “i”) /i/, (point to the letter “m”) /m/ “sim” (run your finger fast through the whole word). I can say the sounds of the letters, /s/ /i/ /m/ (point to each letter), or I can read the whole word “sim” (run your finger fast through the whole word). “Your turn to read a make-believe word. Read this word the best you can (point to the word “lut”). Make sure you say any sounds you know.”

How Do We Administer and Score the NWF Measure?

How Do We Administer and Score the NWF Measure? Place the student copy of the probe in front of the child. Here are some more make-believe words (point to the student probe). Start here (point to the first word) and go across the page (point across the page). When I say “begin,” read the words the best you can. Point to each letter and tell me the sound or read the whole word. Read the words the best you can. Put your finger on the first word. Ready, begin. Student Copy

Maximizing Administration Time Stopwatch: Start watch after tester says “Begin” and time for 1 minute. Scoring: Underline each correct letter sound produced (see specific scoring rules and examples). Slash each incorrect letter sound produced. Maintaining momentum: Allow the student 3 seconds for each letter sound. After 3 seconds, provide the sound to keep the student moving. Discontinue: If a student does not get any correct in the first row, discontinue the task and record a score of zero (0). Ending testing: At the end of 1 minute, put a bracket after the last letter-sound/word produced and calculate the total letter-sounds correct in one minute.

Scoring Rules for NWF Correct Letter Sounds A correct letter sound is scored as the most common sound in English. For example, all the vowels are scored for the short sound and the most common sound for the letter “c” is /k/. See pronunciation guide for remaining letter sounds. Marking the booklet Underline exactly the way the student completes task. For example, if the student goes sound-by-sound, underline each letter individually. If the student reads the target as a whole word, underline the entire word.

Scoring Rules for NWF Partially Correct Responses If a word is partially correct, underline the letter sounds produced correctly. Put a slash (/) through the letter if the letter sound is incorrect. For example, if stimulus word is "sim" and student says "sam," the letters "s" and "m" would be underlined because those letter sounds were produced correctly, giving a score of 2. Repeated sounds Letter sounds pronounced twice while sounding out the word are given credit only once. For example, if stimulus word is "sim" and the student says /s/ /i/ /im/, the letter "i" is underlined once and the student receives 1 point for the phoneme "i" even though the letter "i" was pronounced correctly twice (a total of 3 for the entire word).

Scoring Rules for NWF 3-second rule - sound by sound If student hesitates for 3 seconds on a letter, score the letter sound incorrect, provide the correct letter sound, point to the next letter, and say, "What sound?" This prompt may be repeated. For example, if the stimulus word is "tob" and the student says /t/ (3 seconds), prompt by saying, "/o/ (point to b) What sound?" 3-second rule - word by word If student hesitates for 3 seconds on a word, score the word incorrect, provide the correct word, point to the next word, and say, "What word?" This prompt may be repeated. For example, if the stimulus words are "tob dos et" and the student says, "tob" (3 seconds), prompt by saying "dos (point to et) What word?"

Scoring Rules for NWF Insertions Insertions are not scored as incorrect. For example, if the stimulus word is "sim" and the student says "stim," the letters "s" "i" and "m" would be underlined and full credit given for the word, with no penalty for the insertion of /t/. Skipping Rows If student skips an entire row, draw a line through the row and do not count the row in scoring. Self-corrections If student makes an error and then self-corrects within 3 seconds, write "SC" above the letter and count it as correct.

Tips for Scoring Score for the most common sounds of the letters. Short vowels: i (big), e (beg), a (bag), u (bug), o (bog) “Hard” sounds: “c” = /k/, “g” = /g/, “j” = /j/ A point for each letter, whether it is sound-by-sound or read as a whole word. Score what you hear! Underline exactly the way the student completes the task. Practice with at least 7 students before using the scores to make programming decisions. Look over words you are presenting to increase pacing.

Breakout Activity Form a 3-person group Assign roles: Examiner Student Observer Practice administering measure (Each person administer the measure 2 times)

What ORF Looks Like  A end of year first grader reading approximately 20 words correct per minute. Have participants discuss how the student read, commenting on fluency, types of errors, etc. What do we know about her future as a reader?

How Do We Administer and Score the ORF Measure? Say these specific directions to the child: “Please read this (point) out loud. If you get stuck, I will tell you the word so you can keep reading. When I say "stop," I may ask you to tell me about what you read, so do your best reading. Start here (point to the first word of the passage). Begin.” Materials: Examiner probe Student passages Stopwatch Pencil Preparing the student: Good testing conditions (e.g., lighting, quiet, comfortable)

How Do We Administer and Score the ORF Measure? Say these specific directions to the child: “Please read this (point) out loud. If you get stuck, I will tell you the word so you can keep reading. When I say "stop," I may ask you to tell me about what you read, so do your best reading. Start here (point to the first word of the passage). Begin.”

Maximizing Administration Time Stopwatch: Start watch after student says the first word and time for 1 minute. Scoring: Slash each word produced incorrectly. Maintaining momentum: Allow student 3 seconds for each word. After 3 seconds, say the word to keep the student moving. Discontinue: If student does not get any correct in the first row, discontinue the task and record a score of zero (0). If student scores less than 10 on the first passage, do not administer the other two passages. Ending testing: At the end of 1 minute, put a bracket after the last word produced and calculate the number of correct words in one minute.  In discussing the discontinue rule, remind them that during the benchmark assessments we will administer 3 passages and use the middle (median) score to represent that child’s reading skill. If on the first passage the child reads less than 10 words correct, do not administer the other 2 passages and record the score obtained on the first passage as that child’s score.

Scoring Rules for ORF: Scoring Directions are Similar to Marston, D Correctly Read Words are pronounced correctly. A word must be pronounced correctly given the context of the sentence. Example: The word “read” must be pronounced /reed/ when presented in the context of the following sentence: Ben will read the story. WRC = 5 not as: “Ben will red the story.” WRC = 4 Self-corrected Words are counted as correct. Words misread initially but corrected within 3 seconds are counted as correct. Example: Dad likes to watch sports. WRC = 5 read as: “Dad likes to watch spin...(3 seconds)…sports.” WRC = 5

Scoring Rules for ORF Repeated Words are counted as correct. Words said over again correctly are ignored. Example: I have a goldfish. WRC = 4 read as: “I have a ...have a goldfish.” WRC = 4 Dialectic variations in pronunciation that are explainable by local language norms are not errors. We took the short cut. WRC = 5 “We took the shot cut.” WRC = 5

Scoring Rules for ORF Inserted Words are ignored. When students add extra words, they are not counted as correct words nor as reading errors. Example: I ate too much. WRC = 4 read as: “I ate way too much.” WRC = 4 Mispronounced or Substituted Words are counted as incorrect. She lives in a pretty house. WRC = 6 “She lives in a pretty home.” WRC = 5

Scoring Rules for ORF Omitted/Skipped Words are counted as errors. Example: Mario climbed the old oak tree. WRC = 6 read as: “Mario climbed the tree.” WRC = 4

Words must be read in accordance with the context of the passage Scoring Rules for ORF Words must be read in accordance with the context of the passage Hyphenated Words count as two words if both parts can stand alone as individual words. Hyphenated words count as one word if either part cannot stand alone as an individual word. Numerals and Dates must be read correctly in the context of the sentence. Abbreviations must be read as pronounced in normal conversation. For example, “TV” could be read as "teevee" or "television," but “Mr.” must be read as "mister."

Tips for Scoring Student must read exactly what is on the page. Self-corrections and insertions are ignored and not counted as errors. Simply slash errors until you feel comfortable writing in the error types. Score what you hear! Practice with at least 7 students before using the scores to make programming decisions. Look over passages you are presenting to ensure pacing is efficient. Use the middle score of the three passages read to assess the student’s skill. Have student read all three passages in one sitting  Emphasize that you discontinue only if the child reads less than 10 on the first passage, otherwise all three passages are administered and the middle score is the one reported.

Breakout Activity Form a 3-person group Assign roles: Examiner Student Observer Practice administering measure (Each person administer the measure 2 times)

Letter Naming Fluency (LNF): What important skill does LNF assess? LNF not directly linked to a Big Idea: Used as a risk indicator What is the appropriate time and grade? Through kindergarten and fall of first grade What is the goal? While letter naming is a good predictor of early reading success, knowledge of letter sounds is more important to word reading. Research indicates a score of 8 or below in the beginning of kindergarten is predictive of later reading difficulty. We want participants to see how the measure looks before practicing the specific administration and scoring directions. Have participants notice how the measure provides a general assessment of letter knowledge.

What LNF Looks Like  An end of year kindergartener with solid letter-name knowledge, his score on LNF was 40. Have participants discuss instructional implications.

How Do We Administer and Score the LNF Measure? Materials: Examiner probe Student page Stopwatch Pencil Preparing the student: Good testing conditions (e.g., lighting, quiet, comfortable) Provide the model in standardized manner and follow correction procedures as necessary

How Do We Administer and Score the LNF Measure? Say these specific directions to the child: "Here are some letters" (point). "Tell me the names of as many letters as you can. When I say 'begin,' start here" (point to first letter in upper left hand corner) "and go across the page" (point). "Point to each letter and tell me the name of that letter. Try to name each letter. If you come to a letter you don't know, I'll tell it to you. Put your finger on the first letter. Ready?"

Maximizing Administration Time Stopwatch: Start watch after tester says “Begin” and time for 1 minute. Scoring: Slash each incorrect letter name produced. Maintaining momentum: Allow student 3 seconds for each letter name; after 3 seconds, say the name to keep the student moving. Discontinue: If student does not get any correct in the first row, discontinue the task and record a score of zero (0). Ending testing: At the end of 1 minute, put a bracket after the last letter-name produced and calculate the total letter-names correct in 1 minute.

Scoring Rules for LNF Correct Letter Names Student must say the correct letter name to receive credit. If the student provides the letter sound rather than the letter name, say, "Remember to tell me the letter name, not the sound it makes." This prompt may be provided only once. Self-corrections If student makes an error and self-corrects within 3 seconds, write "SC" above the letter and do not count as an error. Skipping Rows If student skips an entire row, draw a line through the row and do not count the row when scoring. Skipped or omitted letters are not counted in scoring.

Tips for Scoring Score for the letter names. If student skips a row, follow the student’s lead and keep going. Give the student 3 seconds for each letter. Score what you hear! Practice with at least 7 students before using the scores to make programming decisions.

Conducting Reliability Checks for Assessment Team It is critical that all members of the Assessment Team can get an accurate picture of student performance by administering and scoring the DIBELS measures in a reliable manner. Reliability Checks Best Scenario: Prior to Fall Benchmarking, the Assessment Team tests several students on each DIBELS measure. Mentor Coach scores alongside with each tester and compares scores. If scores are within 2 or 3 points the tester is good to go! Option #2: Same as above, except instead of testing students adults test each other. Again, Mentor Coach scores alongside with each tester and compares scores. Option #3: During the first day of Fall Benchmarking the Mentor Coach shadows members of the Assessment Team and checks for reliability.