1 The Florida Assessments for Instruction in Reading Overview

2 Agenda Overview of Florida Assessments for Instruction in Reading (K-12) K-2 Assessments 3-12 Assessments Connecting Assessment to Instruction PMRN Reports Resources Current Steps in Implementation Study Experimental Study Looking Ahead to

3 Agenda K-12 Reading Plan District Reflection Tool K-12 District Reading Plan Revisions

13 What Will It Take to Accelerate Progress? Strong assessment system Content area literacy instruction (will not get the job done for some students) Reading interventions (will never get the job done alone) Differentiated intensity based on need “Adequate” progress is different with a struggling reader

14 Appropriate Use of Assessment  Questions to ask  Is it valid and reliable?  Does it yield new information?  Is it instructionally useful to teachers?  Is it appropriate?

15 Why Spend 30 Minutes Assessing Rather Than Teaching? 1. You can’t expect unless you inspect. 2. Diagnostic information is crucial to teaching. 3. Progress monitoring is crucial to learning.

16 Why New Reading Assessments?  Improved to include assessment of PreK, vocabulary, and comprehension  Automatic reporting without data entry  Identifies risk of reading difficulties

17 HOW does using the Florida Assessments for Instruction in Reading Support Students & Teachers? Comprehensive assessment system 1. Phonemic Awareness 2. Phonics 3. Fluency 4. Vocabulary 5. Text Comprehension 6. Orthographic Skills (Spelling) K * * embedded in text comprehension

18 Task Administration All but one of the K-2 tasks (grade 2 spelling) are individually administered and scored in a web-based application or on paper. Most 3-12 tasks are individually administered on the computer. The Informal Diagnostic Toolkit and grades 3-5 fluency probes will be individually administered by paper and pencil only.

19 HOW is Using the Florida Assessments for Instruction in Reading Beneficial? Precise ability placement for Reading Comprehension tasks  K-2: Word List placement into Reading Comprehension passages  3-12: Adaptive placement into Reading Comprehension passages Precise Internet-based scoring options  K-2: Internet-based application may be used for task administration  3-12: Screen and Targeted Diagnostic Inventory tasks on computer

20 HOW is Using the Florida Assessments for Instruction in Reading Beneficial? Access to the PMRN system for recording student scores  K-2: Scores can be entered by hand or via internet- based scoring tool  3-12: Scores are automatically sent to the PMRN via the computerized task administration Teacher website for guiding instruction will be available  Ideas for grouping students by instructional need including program-specific professional development  Links to appropriate instructional routines, based upon student score results

21 HOW is Using the Florida Assessments for Instruction in Reading Beneficial?  Broad Screen tasks are directly related to Florida Sunshine State Standards and predict end of year performance on the Stanford Achievement Test (K-2) or FCAT (3-12)  Strong psychometric test properties form the basis of all of the assessments  All reading comprehension and oral reading fluency tasks use equating methods to put passages on the same scale so that fluency scores are meaningful.

22 K-2 Web-Based Administration Neither Palm nor handheld scoring will be available May be administered on laptops, mini- laptops, NOT desktop computers Tentative plan to provide mini-laptops for elementary schools in pilot, Reading First schools, and current Palm users (estimated one laptop for every two K-2 teachers)-Based on availability of funding and results of bidding process District-wide implementation required to receive hardware

23 Why Web-Based Versus Palms or Handhelds? Confident that assessments can be administered in a reliable, valid, and efficient manner Long range technology solution Teachers more comfortable with laptop format Broader availability and greater functionality (multifunction versus single) Broader access to technical assistance and maintenance No reprogramming required for updates

24 The Role of DIBELS in Schools can choose which assessments they want to use if they are not in Reading First. DIBELS will not be supported by the PMRN. DIBELS will not be a part of the FLKRS.

25 Florida Assessments Frequency of Administration Assessment Period 1 – Fall/September (Instructional Days ) Assessment Period 2 – Winter/February (Instructional Days ) Assessment Period 3 – Spring/April (Instructional Days )

26 Florida Assessments: K - 2 Most assessments include practice items.

27 How Long Will it Take to Administer the Grades K-2 Tasks? Broad Screen Broad Diagnostic Inventory Targeted Diagnostic Inventory 3- 5 minutes minutes minutes

28 WHAT are the Florida Assessments for Instruction in Reading? Broad Screen/Progress Monitoring Tool (K-2) K-2: 3-5 minute set of tasks designed to indicate the probability of success on the SESAT (K) or SAT-10 (1, 2) Kindergarten oLetter (Naming) Sounds: Assessment Periods 1 and 2 oPhonemic Awareness: Assessment Periods 1, 2, and 3 oWord Reading: Assessment Period 3 Grades 1 & 2 oWord Reading

29 What is the Probability of Success in Reading Score? A score that tells the probability that a student will be on grade level at the end of the year. [at or above the 40%ile on the Stanford Achievement Test (SAT 10)] Zones of Success Green Success Zone: the student has an 85% or greater chance of being at or above grade level Yellow Success Zone: the student has a 16-84% chance of being at or above grade level Red Success Zone: the student has 0-15% chance of being at or above grade level

30 WHAT are the Florida Assessments for Instruction in Reading? Broad Diagnostic Inventory (K-2) Set of tasks designed to indicate other key areas of instructional need related to reading success o Listening Comprehension o Reading Comprehension o Vocabulary o Spelling (Grade 2 only)

31 WHAT are the Florida Assessments for Instruction in Reading? Targeted Diagnostic Inventory (K-2) Set of tasks designed to more precisely indicate the areas of instructional need based upon Screen performance o Print Awareness o Phoneme Blending/Deletion o Letter Names o Letter Sound Knowledge o Letter Sound Correlations o Word Building o Multisyllabic Words

32 Florida Assessments: K - 2 Most assessments include practice items.

33 Broad Screen/Progress Monitoring Tasks

34 Letter Naming and Sounds Kindergarten Teacher Directions: I’m going to show you some letters of the alphabet. Tell me the name of the letter and the sound it makes. Ready?

35 Phonemic Awareness Kindergarten Teacher Directions : 1. What would the word be if I say: /l/ /ake/ (lake) /l/ /ock/ (lock) /sh/ /ip/ (ship) /s/ /o/ (so) /h/ /all/ (hall) /c/ /oa/ /t (coat) For the first assessment, the majority of the items are onset and rime.

36 Word Reading Kindergarten Grade K Word Reading Task Card Student reads word lists Ten items

37 Broad Screen Task Example

38 First Grade Probability of Success Chart

39 Broad Diagnostic Inventory

40 Florida Assessments Broad Diagnostic Inventory Assessment Periods 1 & 2 All receive Listening Comprehension/Reading Comprehension Vocabulary is only for students scoring in yellow/red Success Zones /Reading Comprehension

41 Broad Diagnostic Task Examples

K-2 Assessment Sample Internet-based Administration

43 K-2 Assessment Sample Internet-based Administration Teacher is guided through assessment Easy to use When synced, scores are sent directly to PMRN Immediate display of raw score performance

44 Vocabulary Teacher Directions: What is this?

45 Vocabulary Teacher Directions: What do you call this outdoor activity?

46 The Targeted Diagnostic Inventory A set of tasks that cover the range of developmental reading skills at each grade Students in the yellow and red zones of Success work through these tasks. A criterion of 80% accuracy is required to move on to the next task.

Targeted Diagnostic Inventory * Optional

Targeted Diagnostic Task Example Phoneme Blending Administer items using the key phrase “What would the word be if I say /n/ /ew/ ?”

49 Targeted Diagnostic Task Example Word Building Tasks are given at each grade level. Grade 1 Vowels. Student moves letters to make new words like dig, dug, pug, pig, and pit. Practice and Task Items u p t a e i dog

Targeted Diagnostic Task Example Multisyllabic Word Reading – Grade 2

Medial Vowels Targeted Diagnostic Inventory AP 1, 2, & 3 Only given to K-2 students in yellow or red zones Focus is on more specific skills May be given on a different day

The “Big Picture” Map (K-2) Broad Screen/Progress Monitoring Tool (BS/PMT) “All” students Letter Naming Letter Sounds Phonemic Awareness Word Reading Broad Diagnostic Inventory (BDI) “Most” students Listening Comprehension Reading Comprehension Vocabulary Spelling (2 nd grade only) Targeted Diagnostic Inventory (TDI) “Some” students K = 9 tasks 1 st = 8 tasks 2 nd = 2 tasks

The “Score” Map (K-2) BS/PMTPRS = Probability of Reading Success BDILC = Listening Comprehension Total questions correct (implicit/explicit) RC = Reading Comprehension Total questions correct (implicit/explicit), Adjusted Fluency, Percent Accuracy VOC = Vocabulary Percentile Rank SPL = Spelling Percentile Rank TDIME = Meets Expectations BE = Below Expectations

Grades 3-12 Assessments Model Broad Screen/Progress Monitoring Tool Reading Comprehension Task ( 3 Times a Year ) Targeted Diagnostic Inventory Maze & Word Analysis Tasks If necessary Diagnostic Toolkit ( As Needed ) Ongoing Progress Monitoring ( As Needed ) Scores are automatically entered into the PMRN to generate student reports

55 Reading Concepts Assessed Reading Comprehension ( Broad Screen ) Fluency ( Maze task, Targeted Diagnostic Inventory) Orthographic Processing/Spelling Knowledge (Word Analysis task, Targeted Diagnostic Inventory)

56 WHAT are the Florida Assessments for Instruction in Reading? Broad Screen/Progress Monitoring Tool 3-12: Computer-based, adaptive reading comprehension assessment designed to indicate the probability of success on FCAT Reading

57 WHAT are the Florida Assessments for Instruction in Reading? Broad Screen/Progress Monitoring Tool (3-12) Reading Comprehension Screen This will meet the requirement for reporting progress monitoring scores for students scoring at Level 1 and 2 on FCAT Reading at grades 4-12 and students with reading deficiencies at grade 3.

58 The Broad Screen/Progress Monitoring Tool minute computer-adaptive task of reading comprehension Predicts student’s “probability of success in reading” on the FCAT

59 Reading Comprehension Task Demonstration Click here to start Click here to start

Sample RC Screen Task Item The student reads a passage and answers 7-9 questions to assess reading comprehension. Once all questions are completed, the student clicks the “Submit Answers” button. Submit Answers Switch between full-and split- screen views

61 RC Screen Feedback After the answers are submitted for a passage, the student receives performance feedback. Depending upon his/her score, the student will either be administered an additional (easier or harder) passage or be finished with the task.

62 WHAT are the Florida Assessments for Instruction in Reading? Targeted Diagnostic Inventory (3-12) Set of tasks designed to more precisely indicate the areas of instructional need based upon Broad Screen performance Online MAZE and Word Analysis

63 TDI: The Maze Task Two 3-minute passages Combines the assessment of reading accuracy and speed, or fluency, and gist – level comprehension; results indicate whether probability of success in reading is hampered by difficulties with fluency Administered to all implementation study students in

64 Maze Task Demonstration Click here to start Click here to start

Sample Maze Task Item

66 Maze Feedback After the answers are submitted for a passage, the student receives performance feedback. Students are administered 2 passages and then proceed to Word Analysis.

67 TDI: The Word Analysis Task A 5-15 minute computer-adaptive spelling task that assesses a student’s understanding of letter/sound correspondence Results indicate whether probability of success in reading is hampered by difficulties with word-level skills. Administered to all implementation study students in

68 Word Analysis Task Demonstration Click here to start Click here to start

69 The Word Analysis TDI Student clicks on the ear to hear the word Student types spelling of word in the box Student clicks DONE when finished spelling box

FCAT Success Probability (FSP) Score A prediction score from the Broad Screen that indicates whether a student needs TDI assessment and additional instructional support in reading The probability score will fall within 1 of 3 Success Zones: GREEN YELLOW RED

Success Zone Probabilities 85% or better probability of grade-level or above success on the FCAT 16-84% probability of grade- level or above success on the FCAT 15% or less probability of grade-level or above success on the FCAT GREEN YELLOW RED

72 The TDI Logic Moderate or Low Success Probability on FCAT= Administer TDI Word Analysis TDI Intervention with word recognition skills? Intervention with reasoning skills, prior knowledge, and/or vocabulary? Maze TDI Intervention with fluency/ comprehension skills?

The “Big Picture” Map Type of Assessment Name of Assessment Information Given Broad Screen/Progress Monitoring Tool (BS/PMT) – “All” students Reading Comprehension (RC) Identifies which students we need more information on before planning instruction Targeted Diagnostic Inventory (TDI) – “Some”students Maze Word Analysis (WA) Differentiates between what type of intervention is needed Comprehension and Fluency Deeper Word Level Skills

The 3-12 “Score” Map Broad Screen/ Progress Monitoring Tool FCAT Success Probability (FSP)* Color- coded Percentile Standard Score Ability Score Maze – Targeted Diagnostic Inventory Percent Correct* Percentile (look up table) Standard Score Monitoring Level Word Analysis – Targeted Diagnostic Inventory Percentile* Standard Score Ability Score (WAAS) * = Scores reported from Assessment Period 1

75 Matching Reader to Text Scaffold discussion. Verify that comprehension is at least at 75% correct.

76 FCAT Test Design Cognitive Complexity (Webb’s Depth of Knowledge) Content Categories for Reading - Words & phrases in context - Main idea, plot, & author’s purpose - Comparison; cause/effect - Reference & Research – locate, organize, interpret, synthesize, & evaluate information

77 Purpose of Each 3-12 Assessment Reading Comprehension Screen Assists in identifying students who may not be able to meet the grade level literacy standards as assessed by the FCAT. Mazes Assists in determining whether a student has more fundamental reading problems in the area of fluency and low level reading comprehension. Word Analysis Helps us learn more about a student's fundamental literacy skills--particularly those required to decode unfamiliar words and read accurately.

78 Information Gained from Results Reading Comprehension Screen If a student has a relatively low probability of success, it means they will need extra support for reading growth during the year, and the TDI tests are given as a follow up to get more information about the type of help that may be needed. Maze If a student does well on this test, but has a low probability of success from the RC screen, this means that they are likely to need support in the comprehension strategies, vocabulary, and thinking skills, but likely do not need intensive work to increase their basic reading abilities. Word Analysis Students who perform low on both mazes and word analysis may need intensive interventions to improve their basic reading skills.

79 Ongoing Progress Monitoring (3-12) Oral Reading Fluency (3-5) on paper Computer-based Mazes (3-12)

80 Informal Diagnostic Toolkit (3-12) Phonics Inventory An inventory designed to help you gather information on students’ phonology, orthography, and morphology skills for grades 3-5 and grades Sight Word Grade Level Inventory Word lists for grades 3-12 designed to help monitor the students’ knowledge of grade level vocabulary words. Scaffolding Reading Passages…Coming Soon Short reading passages with multiple choice questions. Within each set of passages, scaffolds will be embedded to help students determine the correct responses (e.g., think and search, right there, author and you.)

81 Accessing the 3-12 Florida Assessments for Instruction in Reading 1.Make sure the computers meet the minimum technology requirements for use: - Internet Explorer 6.0 or later, Safari 2.0 or later, or Firefox 1.5 or later web browser enabled (tasks can be taken on both PC and Mac) -Screen resolution of 1024 x 768 pixels in use -Flash Player or later -JavaScript and Cookies enabled -Headphones for administration in use (the task instructions have audio) -Placement in a well-lit, non-distracting, quiet area 2. Assessments can be accessed at the following URL:

82 Accessing the 3-12 Florida Assessments for Instruction in Reading Scenarios for testing: Computer lab setting Classroom setting

3-12 Assessment Sample Login Student logs into the computer application using: District Name School Name Grade Student Name Birth Date

Assessment Sample Login Using the drop-down menu options, the student completes the login process. A confirmation page is displayed prior to the start of the task. Florida Assessments for Instruction in Reading

85 Connecting Assessment to Instruction PMRN Reports Instructional Resources

86 PMRN Reports FCRR reports (e.g., school or student change) State & Region reports (state status; success zone; historical) School report (status; success zone; historical; student 3+ progress) Teacher reports (status; grade; historical; year- end outcome; success zone; progress) Class reports (also TDI report and error analysis) Student report (OPM; cumulative; historical; score detail; parent letter)

87 K – 2 PMRN Reports Reviewed Class Status Report Student Score Detail Box Class Targeted Diagnostic Inventory Report Class Detail Targeted Diagnostic Inventory Report Class Summary Error Analysis Report Class Detail Error Analysis Report

88 3 – 12 Reports Reviewed Class Status Report Student Score Detail Box Class Summary Error Analysis Report Student Ongoing Progress Monitoring Report Class Year-End Outcome Test Report

94

97 Other resources available online: Florida Department of Education Just Read! Florida LEaRN (Literacy Essentials and Reading Network) Partnership for Reading (National Institute for Literacy) Reading Rockets Colorin’ Colorado

98 The Florida Assessments for Instruction in Reading: Current Steps

99 Implementation Study schools in 4 counties (Broward, Manatee, Leon, & Alachua) administer new tests during 3 windows Teachers individually administer the K-2 assessments (except grade 2 spelling) Reading teachers in grades 3-12 take students to computer lab or Internet-connected computers Teachers give feedback on tests, links to instruction, and toolkit

100 Psychometric Work Testers administer (in Manatee, Leon, & Alachua): Additional Phonemic Awareness & Word Recognition items for Broad Screen: K-1 Additional vocab items in fall; validation in spring Additional passage tryouts at all grades, with links to word placement list. Equating study for Ongoing Progress Monitoring Maze & grades 3-5 fluency Comparison: DIBELS with Broad Screen in K; FORF with Maze; Word Analysis with reading.

101 Continued Goals Helping teachers understand the value of doing their own assessment Streamlining assessment burden in schools Translating assessment results to instruction Logistics of getting assessment done in real school time

102 Looking Ahead: The Florida Assessments for Instruction in Reading will be available to all public schools free of charge Scores from these assessments will be used to meet Reading First reporting requirements Professional development will be provided in regionally during spring and summer 2009 and at the JRF! Leadership conference.

103 Professional Development on the New Florida Reading Assessments December and January: FCRR trains RFPD coordinators and FLARE trainers Winter, 2009: RFPD and FLaRE Coordinators train reading coaches and developers in all districts choosing to use new assessments Spring, 2009: FCRR conducts regional trainings to develop district Master Trainers Summer Academies and Leadership Conference: All attendees trained (teachers, leadership) Elementary Train the Trainer- 3 days Secondary Train the Trainer – 2 days

Professional Development Opportunities TrainingAudienceWhenLocation Elementary/Secondary Train the Trainer: How to Administer Assessment Linking to Instruction PMRN Elementary and Secondary District Trainers Spring/Summer (Provided by FCRR) Regional Elementary Train the Trainer: How to Administer Assessment Linking to Instruction PMRN Elementary (K-5) Reading Coaches and District Trainers RFPD QuarterliesDistrict Secondary Train the Trainer: How to Administer Assessment Linking to Instruction PMRN Secondary (6-12) Reading Coaches and District Trainers FLaRE QuarterliesDistrict Elementary/Secondary How to Administer Assessment Linking to Instruction PMRN Grades K-12 TeachersSummer Professional Development (provided by Just Read, Florida!, FLaRE, RFPD, FCRR) Regional Elementary/Secondary Assessment Overview Elementary and Secondary Reading Leadership Team members Reading Leadership Team Institutes (Provided by FLaRE and RFPD) Regional Elementary/Secondary How to Administer Assessment Linking to Instruction PMRN Elementary and Secondary teachers, coaches, and principals Just Read, Florida! Leadership Conference (audience-specific session tracks) (provided by Just Read, Florida!, FLaRE, RFPD, FCRR) Orlando

105 K-12 District Reading Plan Revisions Embedded RtI language Professional Development Charts will be pre-populated. Added question(s) about PD for Florida Assessments for Instruction in Reading to the PD section Charts C, F, and I will be pre-populated Separated assessment information on Charts C, F, and I (not pre-populated)

106 What Can Be Gained From the Florida Assessments for Instruction in Reading? Use of an efficient Broad Screen to predict K-12 student success in reading Access to free assessment tools designed to guide instruction Linking of expanded curricular supports to assessment results Successful Readers!

107 Additional information regarding the Florida Reading Assessments for Instruction in Reading is available at: For more information contact: Questions?

108 K-12 Reading Plan

109 K-12 Reading Plan District Reflection Tool Think-Pair-Share In terms of K-12 District Reading Plan implementation: Discuss what is in place and working Discuss the top priority areas to develop Discuss the general strategies needed to address top priorities Discuss anticipated challenges and how the challenges will be addressed

110 K-12 Reading Plan District Reflection Tool How can Just Read, Florida! help? Is there an area within the K-12 Reading Plan that should be enhanced?

111 Contact Information Executive Director, Just Read, Florida! – Leadership Contact Deputy Director, Just Read, Florida! – Professional Development Contact Director of Reading First – Elementary School Contact

112 Contact Information Elementary Reading Specialist – Elementary Contact Middle School Reading Specialist – Middle School Contact High School Reading Specialist – High School Contact Call us at…. 850/