1. Florida Assessments for Instruction in Reading- Overview of the K-2 System Barbara R. Foorman, Ph.D. Florida Center for Reading Research Florida State University

2. 2 FAIR Facts  Florida Center for Reading Research (FCRR) began development in 2007 at request of FL Dept. of Ed  FCRR conducted item tryouts in 2007-08 and implementation study in 25 FL schools in 2008-09  Launched statewide in fall 2009: voluntary except for FLKRS; over 1.6 million students assessed  Administered three times a year with additional ongoing measures available  Assists Florida in its efforts to provide additional computer-based assessment in the future  Offered free of charge to Florida school districts

3. 3 K-2 Teacher Survey Results 70%+ of 150 teachers in implementation study responded 67% completed K-2 FAIR in 5 days or less 89% found instructions clear and EST easy to use 90% agreed or strongly agreed that FAIR results helpful in identifying strengths and weaknesses of students 78% said useful in forming instructional groups 80% would recommend FAIR to another teacher 86% found FAIR had comprehensive approach to reading 75% agreed results inform parents about child’s reading

4. 4 Benefits of using FAIR  Valid and reliable tool for monitoring student progress  Strong psychometric test properties form the basis for all assessments  Broad Screen tasks are directly related to Florida standards and predict to end of year outcome measures (SESAT, SAT-10, and FCAT)

5. 5 Benefits of using FAIR  Access to the Progress Monitoring and Reporting Network (PMRN) for recording student scores (by hand or scoring tool in K-2, automatically in 3-12)  Access to the PMRN reports that assist teachers in analyzing student data to better inform instruction  Links to instructional resources for teachers (PMRN reports, search tool, program specific professional development)  Passages equated for difficulty to allow for accurate progress monitoring and comparison of student performance

6. 6 FAIR Administration  Internet-based options  Teacher Administered in K-2  Electronic Scoring Tool – paper and pencil scoring also available  Computer-administered in grades 3-12  Adaptive assessment provides precise ability placement for reading comprehension tasks to monitor students at their instructional level

7. 7 Reading Priorities in Florida Six Components4 Types of Assessment High-Quality Initial Instruction Immediate Intensive Intervention Oral LanguageScreeningBackground Knowledge Extended Time Phonological Awareness Progress Monitoring MotivationFlexible Grouping PhonicsDiagnosisExplicit, SystematicAccommodations FluencyOutcomeDifferentiated, Scaffolded More frequent Progress Monitoring VocabularyReading/Writing Connection ComprehensionPrint-rich

8. 8 Converging Evidence Valid and reliable predictors of risk for reading difficulty are:  Print concepts (preK; early K)  Letter name knowledge (early K)  Phonological awareness and letter sounds (K-1)  Rapid naming of letters (end of K to early G1)  Word recognition (G1 and beyond)  Vocabulary (Fletcher et al., 2002; NELP, 2009; Scarsborough, 1998; Torgesen, 2002)

9. 9 Comprehensive Assessment System 1. Phonemic Awareness 2. Phonics 3. Fluency 4. Vocabulary 5. Text Comprehension 6. Orthographic Skills (Spelling) K-23-12 * * embedded in text comprehension

10. The K-2 “Big Picture” Map Broad Screen/Progress Monitoring Tool (BS/PMT) “All” students Letter Naming & Sounds Phonemic Awareness Word Reading Broad Diagnostic Inventory (BDI) “All” students “Some” students for vocabulary Listening Comprehension Reading Comprehension Vocabulary Spelling (2 nd grade only) Targeted Diagnostic Inventory (TDI) “Some” students K = 9 tasks 1 st = 8 tasks 2 nd = 6 tasks Ongoing Progress Monitoring (OPM) “Some” students K – 2 = TDI tasks 1 – 2 = ORF 10

11. Predictive Validity of Broad Screen

12. Reliability of Broad Screen Table 9: IRT Precision Estimates for Kindergarten and Grade 1 Broad Screen Tasks by Time of Assessment GradeTaskAP 1AP 2AP 3 KindergartenLetter Names0.51*-- Letter Sounds0.870.86- Phonological Awareness-0.860.87 Word Reading--0.86 Grade 1Word Reading0.86 *The Letter Sounds task was more reliable than Letter Names at AP 1; however, due to the restricted range for high risk, a policy decision was made to use the Letter Name task in order to better capture the floor of the distribution. Because the Broad Screen in G2 is a timed task, precision estimates are not reported; however, test-retest reliability was strong (.79-.84).

13. The K – 2 “Score” Map BS/PMTPRS = Probability of Reading Success BDI LC = Listening Comprehension Total questions correct (implicit/explicit) RC = Reading Comprehension Total questions correct (implicit/explicit), Fluency, Percent Accuracy Target Passage VOC = Vocabulary Percentile Rank SPL = Spelling Percentile Rank TDI ME = Meets Expectations BE = Below Expectations OPM ORF = Adjusted Fluency OPM TDI Tasks = ME or BE and Raw Score 13

14. 14 Probability Score – Probability of Reading Success (PRS)  Score calculated based on the BS/PMT  Probability is…the relative possibility that an event will occur or is likely to occur.  A.70 means that we expect this student to have a 70% chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10. The PMRN Reports have already converted the probability score into a percentage so you will see.70 reported as 70% 14

15. A Teacher’s View of the Class Data - Class Status Report (K-2)- ©2009 Florida Center for Reading Research

16. Student Score Detail Box (K-2) ©2009 Florida Center for Reading Research Excellent report to include in a student’s cumulative folder

17. K-2: School Status Report

18. Success Zones - Probability Scores GREEN 85% or better chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10 16-84% chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10 15% or less chance of scoring at or above the 40th percentile at the end of the year on the SESAT or SAT-10 18 YELLOW RED

19. Probability of Reading Success  What percentage of students are in each success zone at the various grade levels? High Success (Green Zone) Moderate Success (Yellow Zone)  It is very important to look at the grade summary report to see the distribution within the yellow zone  Are most of the students in the yellow zone towards the 84% chance end of the zone or closer to the 16% chance Low Success (Red Zone)  What percentage of students are in each success zone within specific teachers’ classrooms? Is there a specific class that does not seem consistent with the rest of the grade? Click on the + sign next to the grade level summary to extend and see the data on individual teachers at that grade level

20. Probability of Reading Success  Look for trends within grade levels, within teachers, and across grades.  Let’s look across grades in our sample data.  What do you notice when you compare across grades? 85% & above (Green) 16%- 84% (Yellow) 15% & below (Red) K28%61%11% 1 st 36%56%8% 2 nd 25%70%5%

21. Median Score  The median score = the middle score  The value which 50 percent of the scores fall above and 50 percent fall below  The median score will be used throughout the PMRN on percentiles and PRS to show the middle score for your group of students. On the School Status Report the Median Percentile Score is given for Vocabulary and Spelling (2 nd only)  On bar graph form it is represented by the T and in table form it is represented as a gray shaded box.

22. Percentile Rank  Percentile ranks are derived scores that are used to rank one student’s performance in relation to a specific group.  On the Florida Assessments for Instruction in Reading, the specified group was a representative sample of Florida students.  Percentile rank is NOT a percentage score identifying how many items were answered correctly.  Percentile Rank is the score given on Vocabulary and Spelling (in 2 nd grade only) 22

23. K-2: School Status Report

24. Target RC Passages for Grades 1 and 2 (BDI) ©2009 Florida Center for Reading Research

25. K-2: School Status Report

26. Target Passage – What does it tell me?  At the School Level, the Target Passage column tells you how many students were able to read at or above the Target Passage with 90% accuracy.  The calculation includes all students – even those who ended up in Listening Comprehension so you can answer the question… How many of my students are accurately decoding at the level expected of this grade level at this assessment period (AP).  To help remember this, write (Accuracy) in parentheses under or next to the column header Target Passage.

27. Target Passage  In first and second grade, this percentage can provide you with one piece of information that will help you determine if your Core Reading Program is addressing the decoding strategies required for accurate grade-level reading.  Let’s look at the sample data 1 st grade – 33.6% of my 143 first graders were able to read at or above the Target Passage which is Passage 1 (66.4% were placed in Listening Comprehension) 2 nd grade – 57.5% of my 122 second graders were able to read at or above the Target Passage which is Passage 1 (42.5% were placed in a lower 2 nd grade passage, a 1 st grade passage or 9 were placed into Listening Comprehension)

28. Total Questions Correct  How many total questions correct (out of 5) did the student answer in either a reading passage or listening comprehension. At the student level, you can get a break down of explicit (out of 3) and implicit (out of 2) questions.  The reading comprehension (RC) graph represents all students who took a RC passage not only those who were at or above the target passage, so…what does this tell you? When they read a passage with at least 90% accuracy (their instructional level), how many comprehension questions can they answer correctly.  To help remember this, write (Comprehension) in parentheses under or next to the column header Total Questions Correct.

29. K-2: School Status Report

30. Total Questions Correct - RC  Let’s look at the First Grade Sample Data: 33.6% of 143 were at or above Target Passage (Passage 1) = 48 students  4 students = Answered 0 questions correct  7 students = Answered 1 question correct  13 students = Answered 2 questions correct  8 students = Answered 3 questions correct  12 students = Answered 4 questions correct  4 students = Answered 5 questions correct

31. Total Questions Correct - LC  Let’s look at the First Grade Sample Data: 66.4% of 143 were below Target Passage (Passage 1) = 95 students  9 students = Answered 0 questions correct  13 students = Answered 1 question correct  26 students = Answered 2 questions correct  19 students = Answered 3 questions correct  14 students = Answered 4 questions correct  14 students = Answered 5 questions correct

32. Questions to Answer Using the School Status Report  Where should my resources be allocated (which grades and classes are struggling in certain areas)?  Where should professional development efforts be focused?  What percentage of students is at each success zone in each class and grade?  How many students need additional support? Are these students clustered in one grade/one teacher? What types of intervention do the students need? Where should I focus my PD? Are there any Programs that I need to purchase?

33. School Grade Summary Report (K-2)

37. Questions to answer using the School Grade Summary Report  What is the distribution of scores in a particular grade level? The School Status told us the Median Score for Vocabulary was the 24 th percentile, but now I see the full distribution and can see that the majority of my students fell within the 11 th – 30 th percentile range. This tells me that overall, the First Grade group is below average in their general vocabulary knowledge I would want to see if this is true in Kindergarten and Second Grade as well (on the School Status Report)  Kindergarten Median Vocabulary = 29 th percentile  Second Grade Median Vocabulary = 41 st  I might look to see how the Core Reading Program addresses vocabulary and if we have any additional programs addressing vocabulary (maybe we are using something different or additional at second grade?)

38. Questions to answer using the School Grade Summary Report  What is the median score in the particular grade level on particular score types? Median PRS in First Grade = 65% chance of success  What is the distribution of the ‘yellow’ zone (16- 84%)? Of my 56% of First Graders who fall in the yellow zone, are more of them at the high end or the low end of the zone?  It is a relatively even distribution across the zone until we reach 65% and then we see that 26 students out of the 80 students in the yellow zone have a PRS of 65-74% chance of success

39. Resources & Conclusions

40. 40 Web Resources  FCRR (Florida Center for Reading Research), www.fcrr.org  Just Read, Florida!, www.justreadflorida.com  LEaRN (Literacy Essentials and Reading Network), www.justreadflorida.com/LEaRN  FOR-PD (Florida Online Reading Professional Development) http://forpd.ucf.edu/strategies/archive.html  Florida Response to intervention (RtI), www.florida-rti.org  Adolescent Literacy, www.adlit.org  Center on Instruction, www.centeroninstruction.org  LD Online, www.ldonline.org  Colorin’ Colorado, www.colorincolorado.org/

43. For questions regarding the FAIR system:  Visit the resources found on the FAIR website: http://www.fcrr.org/fair/index.htm  Access the FAIR FAQ document found at the link below or accessed from the above website: http://www.fldoe.org/faq/Default.asp?Dept=4&submit=GO  Contact: FAIRhelp@fcrr.org Please note the email address is not case sensitive

44. For more detailed information regarding the PMRN:  The PMRN User’s Guides located at: www.fcrr.org/pmrn/userguides.htm There is a section that provides an Overview of Reports and a section that provides an Explanation of Score Types  Complete PowerPoints can be found on the PMRN website: www.fcrr.org/pmrn.htm Topics include:  K-2 Data Entry  K-2 Electronic Scoring Tool  3-12 Web-based Assessment Module (WAM)  School Level Users 1, 2, 3  Phone: (850) 644-0931 or 866-471-5019  Email: helpdesk@fcrr.org

45. Here is what you see if you click on Kindergarten, Phonemic Awareness, Broad Screen/Progress Monitoring Tool

46. In summary, remember…  FAIR was designed to inform instruction.  FAIR data are just one part of the puzzle. Teachers bring information from daily instruction that provides additional, important information.  As FCRR gains more experience with the data and collects data statewide, we will be able to provide guidelines on what are ‘typical’ scores at each grade level or what is a ‘good’ percentage to have in the high success zone (green zone).  We will make comparisons at the close of each assessment period to the State Median scores and percentages

47. 47 Thank you for your time and attention! Any Questions?