1. Data Driven Decisions Moving from 3D to D 3

2. Data Driven Decisions Moving from 3D to D 3 Malcolm Thomas Director, Evaluation Services Escambia School District

3. Presentation Agenda 3D Data Driven Decisions - Review Power of Data Analysis Moving to D 3 - Formative Assessments Review of 3D Resources

4. Malcolm Thomas mthomas@escambia.k12.fl.us (850) 469-5386 www.escambia.k12.fl.us/adminoff/eval

5. How much water is impacted?

6. “Insanity is doing the same thing over and over expecting a different result.” Albert Einstein

7. One of the first steps: Develop a common vocabulary that will lead to common understanding. Digging In the Data

8. NRT CRT Scale Score DVSS NCE Stanine Raw Score Percentile NPR FCAT

9. 1. Norm Referenced Test (NRT) 2.Criterion Referenced Test (SSS) Administered to all students in Grades 3 - 10

10. NRT Score Types Scale Scores –Use to compare year to year and to interpret trends in performance National Percentile Rank (NPR) –% in national sample scoring at or above student (ranks) NOT equal interval Normal Curve Equivalent (NCE) –Equal interval scale from 1 to 99 with average of 50 IS equal interval Stanine –Scores divided into 9 equal groups –Scores of 4 – 6 are considered average

11. Normal Distribution Curve “A Band of Scores” 68% 0 -1  11 -2  -3  22 33 95% 99% MEAN  = Standard Deviation

12.  what test takers can do and what to others  report how well students are doing relative to a pre-determined performance level on a specified set of educational goals or outcomes

13. FCAT CRT Scores (SSS) Scale Scores 100 - 500

14. Scale scores have no meaning in isolation Scale Scores are used to determine achievement levels FCAT CRT (SSS) Scale score range is 100 - 500 FCAT SSS Scale Scores

15. FCAT Developmental Scale Scores (DVSS) Created to help understand students' year-to-year progress. DVSS range from 86 to 3008. Students should receive higher scores as they move from grade-to-grade according to their increased achievement. Use DVSS to monitor academic progress each year.

16. DVSS Linear Progression

17. Let’s Review Scores By Types by Test NRT Raw Score Scale Score NPR NCE Stanine CRT (SSS) Raw Score Scale Score DVSS Achievement Level

18. Learning Gains Measure individual student progress from year to year Uses the Developmental Scale Score and Achievement Level Compares current scores to scores of previous year

19. Making a Learning Gain… 1.Improve FCAT at least one achievement level (example move from Level 1 to Level 2 or Level 3 to Level 5) 2.Maintain satisfactory achievement levels (Levels 3, 4, 5) 3.Demonstrate more than one year’s growth within Level 1 or Level 2 (using DVSS Gains Table) Retained students MUST increase 1 achievement level or maintain satisfactory achievement level (cannot use DVSS method for retained students)

20. Developmental Scale Gain GradeReadingMath 3 to 4230162 4 to 5166119 5 to 613395 6 to 711078 7 to 89264 8 to 97754 9 to 107748 Developmental Scale Scores must be at least one point greater than the above cut-off scores for a student to make a Learning Gain. This method of Learning Gain calculation only applies to promoted students maintaining achievement 1 or 2.

21. Developmental Scale Scores (DVSS) for consecutive Years NRT NCE scores for consecutive years Learning Gains for consecutive years Monitoring Student Progress …

22. Content Clusters The way the FCAT SSS breaks down the subject area tests. The number of items varies across disciplines and grade levels.

23. SSS Scores are used in School Grade Calculations Changes are coming to School Grade calculations in 2007

24. School grade will consist of a total of eight components in 2007 Four (4) Proficiency components: –Reading –Math –Writing –Science **NEW** Four (4) Learning Gain Components: –Reading Learning Gain –Lowest 25% Reading Learning Gain –Math Learning Gain –Lowest 25% Math Learning Gain **NEW**

25. Revised School Grading Scale Grade 2006 Points Required (Points possible = 600) 2007 Points Required (Points possible = 800) A410+525+ B380-409495-524 C320-379435-494 D280-319395-434 F<280<395

27. Assess the current and future needs of students Decide what to change Determine if goals are being met Engage in continuous school improvement Identify root causes of problems Promote accountability

28. Leadership is the key … Source: Brandeis University

29. BARRIERS TO DATA ANALYSIS Lack of training in data use No uniform data collection Lack of leadership at the school and district level Outdated technology Unclear priorities Distrust of data use

30. Elements for Effective Use of Data Create collaborative / trusting relationships Engage in data driven dialogue and collaborative inquiry Learn what you can from standardized tests Use multiple measures Examine data in variety of ways

31. Collaborative Inquiry

32. Questions usually lead to ….... more questions and more data collection.

33. When analyzing data A Picture is Worth …

35. Collect Multiples Measures Example: –Grade 3 Reading FCAT Scale Scores –Grade 3 Reading National Percentile Ranks –Or Grade 3 Reading FCAT Achievement Level Percentages for multiple years

36. Multiple Years of Data

37. Comparing FCAT SSS Reading to Reading NRT

38. Other Examples of Multiple Measures Compare Student Reading or Math Scores with: Averages of State, District or School Attendance Data Achievement Scores And Discipline Referrals GPA Content Areas By Grade Level

39. FCAT Myths to Avoid Performance for individual student assessments should remain the same or improve FCAT Proficiency equals grade level FCAT Proficiency levels are the same for each grade level All data are accurate

40. Moving... From 3D... Data Driven Decisions to... D 3 Data Driven Decisions

41. Summative Assessments Summative assessments attempt to summarize student learning at a specific point in time –Most standardized tests are summative –Excellent tool for shaping district / school or course goals Major disadvantages – time lag between assessment and results –lack of specific information that can be used by teachers for instruction

42. Formative Assessments Occurs when teachers provide information to students in ways that enable the student to learn better, or when students can engage in a similar, self- reflective process Primary purpose support high quality learning Most effective for lower performing students; although all students benefit

43. Formative Assessments Major Advantages –Can occur in real time –Maximize teachable moments in the classroom –Both teachers and students receive information necessary for improvement

44. Profound Mistake Students NOT included as active participants in the assessment process in order for achievement to improve

45. Remedies Create a balanced system of summative assessments of learning and formative assessments for learning

46. A Balanced Approach Assessment of Learning How much have students learned as of a particular point in time? Assessment for Learning How can we use assessment to help students learn more?

47. High Quality Classroom Assessment Teachers should be asking: Why am I assessing? What am I assessing? What is the best assessment method? How do I communicate the assessment results?

48. Assessments Tools, strategies, and techniques to analyze each student’s demonstration of accomplishments of specific goals and criteria. Paper-pencil tests Exhibits Interviews Surveys Observations Other measures Good assessment requires variety to ensure opportunities for success for all students. Would you rather be at the beach?

49. Traditional Assessment Methods Selected response Extended written response Performance assessment Personal oral communication

50. Selected Response Students select an answer from a list or generate a very brief answer. Examples: Multiple Choice True / False Matching Fill in the blank Label a diagram

51. Advantages of Multiple Choice When there is only one correct answer with several plausible alternatives to the correct answer, MC is an effective tool to: Cover a variety of material effectively Be scored easily Provide formative information

52. Multiple Choice – D 3 Provide one, and only one, correct answer Include plausible options that demonstrate a student’s level of understanding Maintain a homogeneous feel in style, length, and visual display Colleague review

53. School Bus - Item A teacher is planning a field trip and will need school buses to transport students. A school bus holds 36 students. If 1,128 students will be transported, how many buses are needed? A. 31 C. 32 B. 31.33D. 36

54. School Bus - Item A teacher is planning a field trip and will need school buses to transport students. A school bus holds 36 students. If 1,128 students will be transported, how many buses are needed? Explain your answer. A. 31 C. 32 B. 31.33D. 36

55. Begin with the end in mind... MC assessment could be designed so that each stem presents a different level or skill set Subsequent assignments could be generated based on the students response to the MC item

56. Formative Assessments Cautions –Must be as rigorous as summative assessment (FCAT) –Data must be collected and analyzed –Data must be used to create future instruction

57. Which Type of assessment has the greatest potential to alter classroom instruction? Which assessment will produce data driven decisions in the classroom? D 3 = Effective Formative Assessment System

58. Moving to D 3 : LESSONS LEARNED It takes time. It has to start at the top. Progress has to be measurable. Summative models are starting points. Data–driven decision making can be a powerful tool in changing student outcomes and promoting continuous improvement.

59. 70 Years ago …. H. L. Menchen said: “ For every complex problem there is an answer that is clear, simple and ……. wrong.”

60. Tools for Data Analysis Evaluation Services Web Site –Tools –Documents –Summative data and charts –Student level disaggregation - secure access (password required)

61. Malcolm Thomas mthomas@escambia.k12.fl.us (850) 469-5386 www.escambia.k12.fl.us/adminoff/eval