1. Florida Association of Science Supervisors Presentation November 3, 2005 Orlando, Florida

2. 2November 3, 2005Florida Department of Education Presentation Topics FCAT Update – Science Setting FCAT Science Achievement Levels Grade 11 Student Motivation Discussion Interpretive Products Release of FCAT Tests 2006 Testing Schedule FCAT Update – Other Content Areas Cognitive Complexity

3. 3November 3, 2005Florida Department of Education FCAT Science Update FCAT Science Standard Setting meeting for educators representing Grades 5, 8, and 11 held in January and September 2005 DOE plans to report student scores and achievement levels beginning with the 2006 test Science to be included in school grade calculations beginning with the 2007 test

4. 4November 3, 2005Florida Department of Education Modifications to Appendix B Grade 5 B.1.2.2 – now also assesses B.1.2.3, B.1.2.5, and B.1.2.6 C.2.2.4 – now also assess C.2.2.3 D.2.2.1 – now a CS also assessing B.2.2.2 and B.2.2.3 (was with G.2.2.3)

5. 5November 3, 2005Florida Department of Education Modifications to Appendix B Grade 8 B.1.3.1 – now also assesses B.1.3.4 F.2.3.3.- now an Annually Assessed also assessing F.2.3.4 and G.1.3.2 G.2.3.2 – now an Annually Assessed assessing D.2.3.2, G.2.3.3, and G.2.3.4 (previously the AA ‘parent’ was G.2.3.4

6. 6November 3, 2005Florida Department of Education Modifications to Appendix B Grade 11 A.1.4.3 – now Annually Assessed also assessing B.1.4.3 C.1.4.1 – now Annually Assessed also assessing C.1.4.2 and C.2.4.6 H.1.4.2 – now also assesses H.1.4.5

7. 7November 3, 2005Florida Department of Education Upcoming FCAT Science Meetings Item Review – 3 weeks, 5 different groups October 31, November 7, and January 9 (review of item development for field testing in 2007) Rangefinder Review November 30 through December 9 (review of scoring criteria for 2006 operational performance task items) Rangefinding Wave 1 April and May Wave 2 September (setting of scoring criteria for performance tasks field tested in 2006)

8. 8November 3, 2005Florida Department of Education Process for Nominating Science Educators Nomination formsNomination forms Sent to SuperintendentsSent to Superintendents Given to District Science Supervisors and educators serving on committeesGiven to District Science Supervisors and educators serving on committees More diversity needed in Science databaseMore diversity needed in Science database GenderGender EthnicityEthnicity Science content areasScience content areas

9. 9November 3, 2005Florida Department of Education Next Version of FCAT Science Item Specifications On hold pending the revision to the Sunshine State StandardsOn hold pending the revision to the Sunshine State Standards 2002 Draft of Item Specifications on website2002 Draft of Item Specifications on website PDF Appendix B (dated October 2004)sent to District Assessment Coordinators also available via email request to mePDF Appendix B (dated October 2004)sent to District Assessment Coordinators also available via email request to me Sample Test Materials have current version of the reference sheets and scoring rubricsSample Test Materials have current version of the reference sheets and scoring rubrics

10. Setting Achievement Levels for FCAT Science Summary of Process and Timeline

11. 11November 3, 2005Florida Department of Education What is Standard Setting? Process of defining the frame of reference for the interpretation of test scores in criterion- referenced assessments, or The matching of achievement levels to test scores Involves both quantitative and qualitative judgment of test items and expected student performance

12. 12November 3, 2005Florida Department of Education Standard Setting Process Performance review meeting - January 2004 Grades 5 and 8 FCAT Science Standard Setting meeting held with educators - January 2005 Grade 11 FCAT Science Standard Setting meeting held with educators – September 2005 Educator committees represented schools and districts from across the state

13. 13November 3, 2005Florida Department of Education Standard Setting Process Table and Group Discussions focused on: Desired student content knowledge and skills of the SSS benchmarks How much content knowledge and skills are necessary for each of the achievement levels Materials used during the meeting included: Performance descriptions from previous meeting FCAT Achievement Level descriptions Test items ordered by difficulty (based on student performance)

14. 14November 3, 2005Florida Department of Education Setting of Achievement Levels: What Happens Next FCAT Science Achievement Level Timeline Business Panel meeting – November 2 Staff review of input and recommendations Draft of State Board of Education Rule Regional workshops – November Rule development workshop Workshop with State Board of Education Advertisement of Rule State Board of Education Rule Adoption - February Reporting of student scores by Achievement Level - May 2006

15. 15November 3, 2005Florida Department of Education Student Motivation Previous discussions on Grade 11 student motivation included: Requiring a specific score on FCAT Science for graduation Requiring a specific score on FCAT Science OR four science courses for graduation Science endorsement for students receiving a specific score on FCAT Science

16. 16November 3, 2005Florida Department of Education InterpretiveProducts

17. 17November 3, 2005Florida Department of Education FCAT Interpretive Products FCAT Handbook— A Resource for Educators Program-level information about the FCAT Posted to DOE website Copies are now in districts

18. 18November 3, 2005Florida Department of Education FCAT Resources Department of Education FCAT Web Addresses http://www.firn.edu/doe/sas/fcat.htm or http://www.fldoe.org FCAT Publications (General Public) FCAT Publications (Educator) FCAT Scores and Reports Content Focus Reports for 2004 and 2005 tests (2006 Report to be posted in May)

19. 19November 3, 2005Florida Department of Education Some FCAT Publications Available Test Design Summary   Details about the structure of the test, including number of items, time limits, and session breakdowns   See handout provided or check the DOE website Content Focus Reports For each content or reporting cluster More specific description of benchmark content assessed in each reporting cluster, each year

20. 20November 3, 2005Florida Department of Education FCAT Interpretive Products Spring Deliveries Mid May 2006 Florida Inquires! (Grades 5, 8, 11) Florida Reads! (Grades 4, 8, 10) Florida Solves! (Grades 5, 8, 10) Florida Writes! (Grades 4, 8, 10) FCAT Results Folder for parents to keep track of various reports See 2005 – 2006 FCAT Interpretive Products Shipment and Delivery Schedule handout for more information

21. Released FCAT Tests What is available now? What are the plans for future release? Where to find more information?

22. 22November 3, 2005Florida Department of Education Tests Released in 2005 Reading and Mathematics Grade 4 and 8 2005 Operational Tests Grade 10 Reading March 2004 Retake version Grade 10 Mathematics 2004 Operational version

23. 23November 3, 2005Florida Department of Education Future plans (if resources are available) Tentative Schedule for FCAT Releases Fall 05 Fall 06 Fall 07 Fall 08 Reading 4, 8, and 10 3, 7, and 9 5 and 6 Release the same tests as in 2005 Mathematics 4, 8, and 10 3, 7, and 9 5 and 6 ScienceNoNo 5 and 8 11 Note: For the fall 2005 release, the Grade 10 tests were used in 2003-04 and the Grade 4 and 8 tests were used in 2004-05.

24. 24November 3, 2005Florida Department of Education Where to Find More Information Additional information about released tests is available at www.fldoe.orgwww.fldoe.org Release includes: Fact Sheet – “How to use Released Tests” and how not to use Test questions formatted like the FCAT Includes all test items that contribute to a student’s score Answer Key Frequently Asked Questions (and answers) And more

25. 25November 3, 2005Florida Department of Education 2005-06 Testing Schedules FCAT Writing+: February 7-8 FCAT SSS (Reading, Mathematics, Science): February 27-March 10 FCAT NRT: same period as FCAT SSS Summer Reading and Mathematics Retakes: June 19-23

26. FCAT Update What is new for Writing+, Reading, and Mathematics?

27. 27November 3, 2005Florida Department of Education FCAT Writing + February 2006 Prompted Essay Test with Multiple-Choice (MC) Items

28. 28November 3, 2005Florida Department of Education FCAT Writing + Writing and Item Types Cloze Selection Conventions 3-option MC Stand-alone Structure o Conventions º 3-option MC Writing PlanWriting Plan º Focus and Organization º 4-option MC Writing SampleWriting Sample º Focus, Organization, Support º 4-option MC

29. 29November 3, 2005Florida Department of Education FCAT Writing + Standard Setting no earlier than Fall 2006 Standard Setting no earlier than Fall 2006 FCAT Writing+ will count for graduation requirement beginning with the graduating class of 2010 FCAT Writing+ will count for graduation requirement beginning with the graduating class of 2010

30. 30November 3, 2005Florida Department of Education Reading and Mathematics Reading Test Item Specifications on hold until the SSS revisions complete Mathematics Test Item Specifications Released on DOE Website in October

31. 31November 3, 2005Florida Department of Education Cognitive Complexity Prior to 2004: based upon Bloom’s Taxonomy Level I and Level II Considers instructional background and students’ approach to a problem Movement to “Cognitive Complexity” to classify test items. Low, moderate, and high Based upon Webb’s depth of knowledge and NAEP system Considers the cognitive demand of a test item

32. 32November 3, 2005Florida Department of Education Based on recall and recognition of previously learned concepts and principles Examples include : Identifying common examples or recognizing a concept Recognizing a standard scientific representation of a simple phenomenon Retrieving information from a chart, table, diagram, or graph Calculating or completing a familiar single-step procedure or equation using a reference sheet Low Complexity Items

33. 33November 3, 2005Florida Department of Education Require students to decide what to do using informal methods of reasoning and problem- solving strategies from various domains Examples include: Applying or inferring relationships among facts, terms, properties, or variables Describing examples and non-examples of scientific processes or concepts Predicting the next logical step Moderate Complexity Items

34. 34November 3, 2005Florida Department of Education High Complexity Items Require students to solve non-routine or unique problems, require abstract reasoning, analysis, judgment, or creative thought Example include : Generalizing or drawing conclusions Analyzing an experiment to identify a flaw and propose a method for correcting it

35. 35November 3, 2005Florida Department of Education FAST Conference Presentation Florida Inquires! and the Grade 5 FCAT Released Performance Task Presented by Roberta McDonald, FCAT Science Development Team Saturday, November 5 – 9:00 a.m.

36. 36November 3, 2005Florida Department of Education QUESTIONS?? Contact Information: Linda Oravetz FCAT Science Coordinator Test Development Center Florida Department of Education oravetzl@mail.leon.k12.fl.us