Presentation on theme: "Data Driven Decisions Moving from 3D to D 3. Data Driven Decisions Moving from 3D to D 3 Malcolm Thomas Director, Evaluation Services Escambia School."— Presentation transcript:
“Insanity is doing the same thing over and over expecting a different result.” Albert Einstein
One of the first steps: Develop a common vocabulary that will lead to common understanding. Digging In the Data
NRT CRT Scale Score DVSS NCE Stanine Raw Score Percentile NPR FCAT
1. Norm Referenced Test (NRT) 2.Criterion Referenced Test (SSS) Administered to all students in Grades 3 - 10
NRT Score Types Scale Scores –Use to compare year to year and to interpret trends in performance National Percentile Rank (NPR) –% in national sample scoring at or above student (ranks) NOT equal interval Normal Curve Equivalent (NCE) –Equal interval scale from 1 to 99 with average of 50 IS equal interval Stanine –Scores divided into 9 equal groups –Scores of 4 – 6 are considered average
Normal Distribution Curve “A Band of Scores” 68% 0 -1 11 -2 -3 22 33 95% 99% MEAN = Standard Deviation
what test takers can do and what to others report how well students are doing relative to a pre-determined performance level on a specified set of educational goals or outcomes
Scale scores have no meaning in isolation Scale Scores are used to determine achievement levels FCAT CRT (SSS) Scale score range is 100 - 500 FCAT SSS Scale Scores
FCAT Developmental Scale Scores (DVSS) Created to help understand students' year-to-year progress. DVSS range from 86 to 3008. Students should receive higher scores as they move from grade-to-grade according to their increased achievement. Use DVSS to monitor academic progress each year.
Let’s Review Scores By Types by Test NRT Raw Score Scale Score NPR NCE Stanine CRT (SSS) Raw Score Scale Score DVSS Achievement Level
Learning Gains Measure individual student progress from year to year Uses the Developmental Scale Score and Achievement Level Compares current scores to scores of previous year
Making a Learning Gain… 1.Improve FCAT at least one achievement level (example move from Level 1 to Level 2 or Level 3 to Level 5) 2.Maintain satisfactory achievement levels (Levels 3, 4, 5) 3.Demonstrate more than one year’s growth within Level 1 or Level 2 (using DVSS Gains Table) Retained students MUST increase 1 achievement level or maintain satisfactory achievement level (cannot use DVSS method for retained students)
Developmental Scale Gain GradeReadingMath 3 to 4230162 4 to 5166119 5 to 613395 6 to 711078 7 to 89264 8 to 97754 9 to 107748 Developmental Scale Scores must be at least one point greater than the above cut-off scores for a student to make a Learning Gain. This method of Learning Gain calculation only applies to promoted students maintaining achievement 1 or 2.
Developmental Scale Scores (DVSS) for consecutive Years NRT NCE scores for consecutive years Learning Gains for consecutive years Monitoring Student Progress …
Content Clusters The way the FCAT SSS breaks down the subject area tests. The number of items varies across disciplines and grade levels.
SSS Scores are used in School Grade Calculations Changes are coming to School Grade calculations in 2007
School grade will consist of a total of eight components in 2007 Four (4) Proficiency components: –Reading –Math –Writing –Science **NEW** Four (4) Learning Gain Components: –Reading Learning Gain –Lowest 25% Reading Learning Gain –Math Learning Gain –Lowest 25% Math Learning Gain **NEW**
Revised School Grading Scale Grade 2006 Points Required (Points possible = 600) 2007 Points Required (Points possible = 800) A410+525+ B380-409495-524 C320-379435-494 D280-319395-434 F<280<395
Assess the current and future needs of students Decide what to change Determine if goals are being met Engage in continuous school improvement Identify root causes of problems Promote accountability
Leadership is the key … Source: Brandeis University
BARRIERS TO DATA ANALYSIS Lack of training in data use No uniform data collection Lack of leadership at the school and district level Outdated technology Unclear priorities Distrust of data use
Elements for Effective Use of Data Create collaborative / trusting relationships Engage in data driven dialogue and collaborative inquiry Learn what you can from standardized tests Use multiple measures Examine data in variety of ways
Other Examples of Multiple Measures Compare Student Reading or Math Scores with: Averages of State, District or School Attendance Data Achievement Scores And Discipline Referrals GPA Content Areas By Grade Level
FCAT Myths to Avoid Performance for individual student assessments should remain the same or improve FCAT Proficiency equals grade level FCAT Proficiency levels are the same for each grade level All data are accurate
Moving... From 3D... Data Driven Decisions to... D 3 Data Driven Decisions
Summative Assessments Summative assessments attempt to summarize student learning at a specific point in time –Most standardized tests are summative –Excellent tool for shaping district / school or course goals Major disadvantages – time lag between assessment and results –lack of specific information that can be used by teachers for instruction
Formative Assessments Occurs when teachers provide information to students in ways that enable the student to learn better, or when students can engage in a similar, self- reflective process Primary purpose support high quality learning Most effective for lower performing students; although all students benefit
Formative Assessments Major Advantages –Can occur in real time –Maximize teachable moments in the classroom –Both teachers and students receive information necessary for improvement
Profound Mistake Students NOT included as active participants in the assessment process in order for achievement to improve
Remedies Create a balanced system of summative assessments of learning and formative assessments for learning
A Balanced Approach Assessment of Learning How much have students learned as of a particular point in time? Assessment for Learning How can we use assessment to help students learn more?
High Quality Classroom Assessment Teachers should be asking: Why am I assessing? What am I assessing? What is the best assessment method? How do I communicate the assessment results?
Assessments Tools, strategies, and techniques to analyze each student’s demonstration of accomplishments of specific goals and criteria. Paper-pencil tests Exhibits Interviews Surveys Observations Other measures Good assessment requires variety to ensure opportunities for success for all students. Would you rather be at the beach?
Traditional Assessment Methods Selected response Extended written response Performance assessment Personal oral communication
Selected Response Students select an answer from a list or generate a very brief answer. Examples: Multiple Choice True / False Matching Fill in the blank Label a diagram
Advantages of Multiple Choice When there is only one correct answer with several plausible alternatives to the correct answer, MC is an effective tool to: Cover a variety of material effectively Be scored easily Provide formative information
Multiple Choice – D 3 Provide one, and only one, correct answer Include plausible options that demonstrate a student’s level of understanding Maintain a homogeneous feel in style, length, and visual display Colleague review
School Bus - Item A teacher is planning a field trip and will need school buses to transport students. A school bus holds 36 students. If 1,128 students will be transported, how many buses are needed? A. 31 C. 32 B. 31.33D. 36
School Bus - Item A teacher is planning a field trip and will need school buses to transport students. A school bus holds 36 students. If 1,128 students will be transported, how many buses are needed? Explain your answer. A. 31 C. 32 B. 31.33D. 36
Begin with the end in mind... MC assessment could be designed so that each stem presents a different level or skill set Subsequent assignments could be generated based on the students response to the MC item
Formative Assessments Cautions –Must be as rigorous as summative assessment (FCAT) –Data must be collected and analyzed –Data must be used to create future instruction
Which Type of assessment has the greatest potential to alter classroom instruction? Which assessment will produce data driven decisions in the classroom? D 3 = Effective Formative Assessment System
Moving to D 3 : LESSONS LEARNED It takes time. It has to start at the top. Progress has to be measurable. Summative models are starting points. Data–driven decision making can be a powerful tool in changing student outcomes and promoting continuous improvement.
70 Years ago …. H. L. Menchen said: “ For every complex problem there is an answer that is clear, simple and ……. wrong.”
Tools for Data Analysis Evaluation Services Web Site –Tools –Documents –Summative data and charts –Student level disaggregation - secure access (password required)
Malcolm Thomas firstname.lastname@example.org (850) 469-5386 www.escambia.k12.fl.us/adminoff/eval