Assessments for Secondary Students Principal’s Leadership Conference Randee Winterbottom Pat Howard Florida Center for Reading Research June 2005

Objectives To assist principals to… Understand purposes for reading assessment Know the four types of assessment Know what skills may need to be assessed Have an overview of the secondary study on progress monitoring tools Learn about oral reading passages for next year Become familiar with commonly used secondary assessments Summarize objectives for the session

Why Assess? Collect data to determine problems and make decisions about students May be formal or informal and conducted through a variety of methods: Record reviews Interviews Observations Testing One of the first questions to ask is, “Why Assess?” We assess to gather information and data that will assist us as we make instructional decisions about our students, to guide us in knowing if we are on the right track in meeting students instructional needs. Assessments may be formal or informal and conducted through a variety of methods.

Four Types of Assessments Screening First Alert – which students may need extra assistance Progress Monitoring Is the child making adequate progress towards reading goals? Diagnostic Used only if progress monitoring is not answering Questions concerning a student’s weaknesses and strengths in reading skills Outcome Did the student make progress toward reading standards There are four types of assessments that should be considered when as assessment needs are determined. What is the purpose of the assessment? (Review each type on the screen.)

Upper Grades Study - Spring 2005 Purpose: To establish the reliability of several measures and their relationship to performance on the FCAT and to determine if one of the measures would be suited for use as a state-wide progress monitoring measure. Leon and Dade – 300 randomly assigned students in grades 4, 6, 7, 8, 9, 10. Measures to be used: FCAT passages for ORF – grades 6, 7, 8, 9 - Individual FCAT maze passages – grades 4, 6, 8, 10 - Group Espin Mazes – grades 8 and 10 - Group Test of Silent Word Reading Fluency (TOSWRF) - Individual Test of Sentence Reading Efficiency (TOSRE) - Group WASI – Vocabulary and Similarities - Individual In an effort to provide TA for middle/high schools as they write their k-12 comprehensive reading plans, FCRR is working with the JRF! Office to establish valid and reliable measures for middle school progress monitoring of students scoring Level 1 or 2 on FCAT Reading. The goal is to determine which of several measures used in the study, will provide the most efficient and predictive information on how students are progressing toward year end goals and the on the FCAT. This summer the data will be analyzed in order to make some decisions related to the goals of the study. We are very interested in the results on the mazes because the maze assessment taps into vocabulary and comprehension and is group administered, requiring only about a total of 5 minutes to administer including directions. As you can see in the slide Dr. Torgesen has included several kinds of tests; both individually and group administered, and published as well as locally developed measures in the study.

Upper Grades Study - Fall 2005 cont. If one of the group administered measures shows promise as a valid and reliable predictor of FCAT performance, a progress monitoring tryout study will be conducted. School districts representative of the overall demographics of the state will participate The same students who are receiving progress monitoring with ORF probes will be administered the group measure to determine any meaningful differences - group vs. individually administered and sensitivity to individual growth In an effort to further investigate valid and reliable measures for middle school students the study will continue on into the 05-06 school year. In selected schools where students who are being administered the ORF passages they will also be administered the group administered measure to determine if there any meaningful differences between the two types of measures and sensitivity to growth.

Oral Reading Fluency Passages for 2005-2006 FCRR working with JRF! Office to make passages available for ORF for progress monitoring Optional FCAT released passages or FCAT- like passages developed for this purpose Plan to make them downloadable from a JRF! web site Will be able to enter data into the PMRN One of the measures included in Joe’s study is ORF. In an effort to provide districts with something that is free and uniformly available until the study is complete, and scientifically-based recommendations can be made; FCRR and JRF, are providing ORF measures for the 2005-2006 school year. Training for MS personnel began on May 11th. Three sites were chosen for these regional trainings. On the 19th, the last training until the fall, was conducted in Lake Placid at the Heartland Educational Consortium. These trainings; what we call facilitator trainings, (train the trainer) were for those persons who will provide training within their own district. Schools are required to do progress monitoring of their level 1 and level 2 students in middle school but they have the choice of what PM tool to use. Districts may continue to use something they already have in place. These new passages will just be an additional resource for districts/schools who choose to use them. The ORF scores from the FCAT-like passages will be enterable into the PMRN; simplifying record keeping for schools. The 6th grade DIBELS scores will not be enterable into the PMRN.

Student Practice #1 Video Clip Let’s listen to a student reading a 7th grade ORF passage. If you’d like to follow along and try to identify the errors you can find this passage in your handout.

139 - 8 131 This is how the examiner scored this students ORF. Briefly go over these errors.

This is an example of a maze passage This is an example of a maze passage. This measure requires the child to read and choose the best answer. For every 7th word, a choice of 3 words is presented with only one being the correct answer. The other 2 words are intentionally words that could not possibly be substituted for the correct response. The measure is timed for 3 minutes and can be group administered. The student’s score is simply the number of correct choices made in the three minutes. It is a measure of fluency (how much is read in three minutes and comprehension (the student would need to read and understand the text to select the correct word from the three choices).

This is the answer key for the maze passage.

What Skills Should be Assessed? Testing older students specifically on phonological awareness measures does not add power to the identification of reading and spelling difficulties (Hogan, Catts & Little, in press) Direct measures of academic skills will provide the most accurate data for planning instruction Older students will not necessarily need to be assessed on phonemic awareness but if they are not fluent readers it is important to assess decoding to determine if the student has a deficit in sound-symbol correspondence. If this is a deficit teachers will have data to assist them as they plan for instruction.

What Skills Should be Assessed? (cont.) Some of the types of assessments to consider are… Passage Reading Comprehension Oral Reading Fluency Vocabulary Knowledge Word Recognition Speed and Accuracy Knowledge of Phoneme-Grapheme (sound-symbol), Syllables, Morphology (units of meaning) and Correspondences in Ortholography (writing) Here are some of the skill areas to consider assessing.

You have a two – page handout with the complete table of listings of commonly used assessments for older students. Please understand that this list in no way should be considered an endorsement of or recommendation of any of the assessments. It is simply a compilation of measures that we know are commonly used.

DAR Diagnostic Assessments of Reading Florence Roswell and Jeanne Chall Diagnostic Assessments of Reading The Diagnostic Assessments of Reading (DAR) is one of the diagnostic assessments provided to districts by the State for free.

DAR: Characteristics Six individually administered tests of essential areas of reading and language Suitable for students of all ages functioning on pre-reading levels through end of high school Easy to administer Brief administration time The DAR is made up of a series of six individually-administered tests of reading and language. A single DAR kit can be used for students from grade one through high school and includes the assessment of pre-reading skills. The DAR is easy to administer and has a brief administration time. Because the test is adaptive in nature, the administration of the test will revolve for each student around the sub-tests and levels where the student will show mastery.

DAR: Characteristics (cont.) Easy to score Separate scores for each subtest Meaningful, easy to interpret scores Based on extensive research & experience Meets the State DOE content requirements Quality, sturdy materials The DAR is easy to score. Directions for administration and scoring are listed together with each subtest and level so that the teacher doesn’t have to go back and forth searching for directions or scoring criteria. The DAR yields separate scores for each of its subtests. When recorded in the individual interpretive profile (a report we will visit later), the scores become meaningful and easy to interpret. The DAR was authored by two very well known and respected leaders in the field of reading: Florence Roswell and Jeanne Chall.

The DAR Will Help You... Develop a constructive student-teacher relationship Identify a student’s strengths and weaknesses in reading – provides opportunities for quantitative and qualitative diagnostic information Assess the reading development/growth of a given student Prescribe meaningful individualized instruction Based on years of actual practice and proven reliability and validity. For this program, it is assumed that students having reading difficulties have been identified. Either through a formal group assessment process or screening, or through teacher observation, the State is assuming that we have identified the students who are having reading difficulties. These are the students for which the DAR will be administered. Because the DAR is administered individually, it helps teachers develop a constructive relationship with students. The data collected with the DAR are recorded in an individual Interpretive Profile that is useful to identify strengths and weaknesses in reading. The DAR can be administered at any time during the year. Pre-, interim, and post-testing is often helpful. Comparing a student’s profile from one administration to the next can help teachers determine growth in weak areas. The DAR provides opportunities for not only quantitative but qualitative diagnosis. The information provided by the DAR is based on years of actual practice and research and has proven reliability and validity. Because of this it can help teachers design meaningful instructional plans.

DAR Tests Word Recognition Word Analysis Oral Reading Silent Reading Comprehension Spelling Word Meaning We are now going to look at the six subtests that make up the DAR. As stated earlier, you will see frequent references to levels 1 through 12. Although these roughly correspond to grade levels, the DAR is considered to be appropriate for pre-reading through the end of high school. Because of the range of development during the early stages of reading, it’s difficult to assign “grade level” correspondences but an effort to provide some meaning to the “leveling,” these “approximate grade levels” were assigned. Because the DAR is an adaptive test, the test administrator will not necessarily administer all the subtests to a given student. In most cases, the progression of testing is such that an attempt is made to try to match the subtest to the student’s level of reading in order that the student experience some measure of success.

The student logs iin to the SRI and is asked, first to select readings that are of interest.

The student is given a series of brief writings and then asked to fill in the blank by selecting the correct response. The foils provided would all be sensible choices if the student had not read and understood the reading selection.

The teacher is able to gain a listing of all students in the class and to see at a glance which students had been tested and their most recent Lexile score.

These are the reports that are available to the teacher These are the reports that are available to the teacher. We will look at two on our student.

This is a report that the student receives and is duplicated for the teacher. The selections are based on the student’s initial indication of interest.

This teacher report lists all students who have participated in the SRI and their most recent test date with lexiles indicating independent, instructional and challenging texts.

GRADE Group Reading Assessment and Diagnostic Evaluation Provides data for older students to assist with instructional decision - making The Group Reading Assessment and Diagnostic Evaluation (GRADE) is appealing not only because it is group administered but also because it has good psychometrics and is valid and reliable.

Group Reading Assessment and Diagnostic Evaluation (GRADE) Can be group administered Out-of-level testing is available Students who are in need of intensive interventions may need to be assessed below their grade enrollments Level 5 is for very low 6th – 12th graders Please remember that for reporting purposes students must be assessed 4x/year using grade level materials. For ongoing progress monitoring that is conducted more frequently, it is good practice to use instructional level materials. Teachers may choose to administer only one or two of the subtests, based on students areas of need.

GRADE cont. Level 6 is for very low 7th - 12th graders Level M is for very low 10th – 12th graders Vocabulary – measures decoding and vocabulary knowledge Sentence Comprehension – measures ability to comprehend a sentence as a complete thought

GRADE cont. Listening Comprehension – measures understanding of spoken language Passage Comprehension – measures ability to comprehend extended text as a whole Test is not timed but recommendation is for two 30 – minute sessions Has 2 forms making progress monitoring an option

GRADE cont. Norm-referenced – with a variety of score types: stanines, percentiles, normal curve equivalencies, standard scores, grade equivalents Provides Diagnostic Analyses Summary for each subtest Passages include narrative and expository and vary in length

Here is an example of the vocabulary and sentence comprehension subtest.

This is what a completed individual score summary looks like This is what a completed individual score summary looks like. As you can see there is the option for deeper error analysis by indicating the types of errors the student made. This can be useful information when planning instruction. AGS has recently published a Reading Fluency measure called Reading Fluency Indicator that would be a nice companion measure to the GRADE. It measures rate, accuracy and comprehension and is individually administered.

Spellography is a spelling assessment and instructional program developed by Dr. Louisa Moats and published by Sopris West. It is designed for intermediate to upper-grade students who would benefit from word-study and spelling instruction. Students are taught multiple layers of language structure including: Phoneme-grapheme correspondence Syllable spelling patterns and combinations Affixes Parts of speech and much more

Sample Spelling Inventory Error Analysis Features Short Vowel Digraph Trigraph Blend Complex consonant VCE Vowel Teams Root Morphograph Vowel +R Total Points phone ph o-e 2 + 1 (correct) smudge u sm -dge 1 squirt squ ir throat thr oa fraction frac (correct) dream dr ea fright fr igh This is a modified version of the spelling inventory error analysis. When analyzing the types of errors made on the inventory, a teacher will have data that will indicate any deficiencies in a child’s decoding ability and can base her instruction on that analysis. Blue circles indicate incorrect spellings + 1 indicates that the whole word was spelled correctly Adapted from Lousia Moats’ Spellography Spelling Inventory

Case Study #1: Tatiana, Grade 8 (September) Skill Assessed Measure Results Reading Comprehension FCAT Reading from previous spring Level 2 Oral Reading Fluency State - Provided Passages on 8th grade reading level 112 wcpm 30th percentile (Hasbrouck and Tindal) Oral Word Reading Efficiency TOWRE – real and nonsense words Real Words 48th percentile Nonwords 35th percentile Silent Reading Comprehension Core Reading Program Placement Test 38% correct on silent reading Comprehension - beginning 6th grade Now let’s take a look at a case study.

Tatiana Where does her instructional level appear to be? Does she need instruction in decoding? Would she benefit from fluency practice? Are there other measures that should be administered? Which ones? Why? Ask for audience responses

Tatiana Where does her instructional level appear to be? Based on data provided – about 6th grade Does she need instruction in decoding? Would she benefit from fluency practice? Are there other measures that should be administered? Which ones? Why? Elicit responses from the audience

Tatiana Where does her instructional level appear to be? Based on data provided – about 6th grade Does she need instruction in decoding? Probably, since she scored in the 35%ile on the TOWRE nonword Would she benefit from fluency practice? Are there other measures that should be administered? Which ones? Why?

Tatiana Where does her instructional level appear to be? Based on data provided – about 6th grade Does she need instruction in decoding? Probably, since she scored in the 35%ile on the TOWRE nonword Would she benefit from fluency practice? Yes, she only read 112 wcpm and scored in the 30th percentile on the ORF measure. Increasing her ORF fluency would likely improve her comprehension as well Are there other measures that should be administered? Which ones? Why? Elicit responses from the audience

Tatiana Where does her instructional level appear to be? Based on data provided – about 6th grade Does she need instruction in decoding? Probably, since she scored in the 35%ile on the TOWRE nonword Would she benefit from fluency practice? Yes, she only read 112 wcpm and scored in the 30th percentile on the ORF measure. Increasing her ORF fluency would likely improve her comprehension as well Are there other measures that should be administered? Which ones? Why? Yes. A developmental spelling test. Conducting an error analysis on the spelling test would provide valuable information to the teacher about any gaps Tatiana may have with her decoding skills. Elicit responses from the audience. Be sure to discuss any other tests they suggest.

Summary Understand purposes for reading assessment Overview of the secondary progress monitoring study Be familiar with the ORF measure for 05-06 Identify assessments that will meet the needs at your school What we have tried to do today is provide an overview of reading assessments at the secondary level. We have offered compelling whys for assessing the struggling reader and possible assessments to consider for the secondary student. We hope that you will see the benefit of gathering data to better inform instruction to support students in meeting their reading goals.

Contact Information Randee Winterbottom rwinterbottom@fcrr.org Pat Howard phoward@fcrr.org 850-644-9352

Thank You