1. A Model Description By Benjamin Ditkowsky, Ph.D. Student Growth Models for Principal and Student Evaluation

2. Its Not Always Easy To See What Our Students Can And Can Not Do

3. In Addition, Sometimes Some Things Just Dont Match

4. Can We Just Use The Same Test At Two Points In Time (Use Gain Scores)? There are some problems with gain scores All tests are made up of a true score and measurement error x i = x i TRUE + Error is always present, but we dont know how much error is present When we subtract one score from another (i.e., a gain score) error in the final score is greater than in either of the tests from which the error originates x gain = x 2 TRUE – x 1 TRUE + + (true differences are subtracted but error propagates)

5. Error! There Are Many Reasons Other Than The Test That Student Scores Vary From One Test To Another

6. Complex Solutions Exist, But Do Complex Solutions improve teaching and learning? Do Complex Solutions maintain a focus on the connection on standards? Do Complex Solutions applied to our data violate basic statistical assumptions? Do Complex Solutions focus on details or big ideas? The overly complex statistical models have not been shown to be substantially more effective than less complex models with complexity comes other concerns

7. Accountability Should Be Trustworthy For a solution to be trustworthy it should meet all of the assumptions upon which it is based Useable For a solution to be useable, it should help teachers and principals adjust instruction and intervention in real time Accessible For a solution to be accessible it should be transparent

8. We Will Use Student Data For Evaluation

9. We Have Lots Of Data Complex solutions have complex algorithms to estimate missing information. For most students… we have enough information we can infer the rest. Although, we dont necessarily have the same data for all students…

10. Complex Statistical Analysis Is Unnecessary Given sufficient evidence, complex analysis is unnecessary looks like a duck walks like a duck swims like a duck quacks like a duck it probably is a duck If it …

11. We Know What We Expect Cut Scores are Based on Expected Performance Test performance is categorized as Exceeding Standards, Meeting Standards, Below Standards, Academic Warning Progress is defined by our assignment of the value or worth of improvement in the categorical performance of test scores from one time to the next We Use Cut Scores To Categorize Level of Achievement, and Progress

12. Cut Scores Separate Performance Levels ISAT Scores are broken down into performance designations (Academic Warning, Below Standards, Meets Standards, Exceeds Standards) Typical Growth in Language Development on ACCESS has been established (W.I.D.A. Consortium, 2009)is calculated based on entry performance level and change in composite score (typical low, and high range growth) R-CBM Scores reliably predict state test scores (Silberglitt & Hintze, 2005), cut scores predicting outcomes have been established by aimsweb and within Illinois (i.e., Below Basic, Questionable, Proficient, confidently proficient) M-CAP scores predict State Test scores, and typical performance is established across the township Benchmark assessments can divide score into performance categories

13. We Are Explicit About Our Values Exceptional Growth (2) as well as maintenance of exceptional performance is highly valued, thus any score-pair that moves up two or more categories or ending in the highest performance category is worth two points. Proficient Growth (1) is defined as any score-pair that moves up one category or is maintained i.e., scores growth at a rate commensurate with the increasing expectations of the Meeting Expectations category. Inadequate Growth (-1) is defined as a score-pair in which the performance category at time 1 is higher than the performance category at time 2, or growth from time 1 to time 2 is not sufficient to move a student from the below expectations category into the meeting expectations category. Unsatisfactory Growth (-2) is defined as a score-pair in which the performance category drops by two categories from time 1 to time 2 or ends in the Academic Warning Category. Growth is defined as a change in performance category from Time 1 to Time 2

14. A Value Table Weights Growth

15. Miguel Is A 2nd Grade Student The Data we have for him include: Type I ACCESS from grade K and 1 Aimsweb - Fall and Winter RCBM MCAP (not in example) Type II Vocabulary Matching Fall and Winter Avenues Pre-Post Assessments for ELLs Type III Pre-Post Classroom Assessments

16. Each ACCESS pair is assigned a growth value based on the comparison of scores and the expectations for growth (WIDA, March 2009) Unsatisfactory Needs Improvement Proficient growth Excellent growth Miguel is in second grade. Baseline: In K his language proficiency level was 1.8 Expectations: The low to high range for growth was 44 to 90 Scores: 297 – 209 = 88 His gain was 88, less than 90 Category designation: His growth level is considered in the proficient range Miguel Is A 2nd Grade Student

17. In Winter, Miguels performance was in the proficient range Fall performance was Below Basic Expectations Miguel is in second grade. Baseline: In Fall his score 15 WRC indicated Below Basic Performance Expectations: Winter Proficient Score is 65 WRC Growth: In Winter, Miguel scored in the proficient range, his movement up two categories (Below to Proficient) is considered excellent growth Category designation: His growth level is considered in the excellent range MeasuredEffects.com, 2010 ISAT Cut Scores Miguel Is A 2nd Grade Student

18. Relatively low scores are in the proficient range in the fall, Miguels score of 2 is considered proficient In Winter, expectations for VM are higher, and Miguels score increased sufficiently to remain on target. Local Normative Values Miguel is in second grade. Baseline: In Fall his score 2 WRC indicated his performance was typical in the Meets category Expectations: Winter Proficient range is from 7 to 10 Growth: In Winter, Miguel scored in the proficient range, his performance indicated that his growth was consistent with expectations for his grade level (Meets to Meets) Category designation: His growth level is considered in the proficient range Miguel Is A 2nd Grade Student

19. The Avenues pre test is broken into 6 levels designated in 3 levels (Beginning, Intermediate and Advanced) Miguel is in second grade. Baseline: In Fall his proficiency level was 1 indicating early beginning language proficiency Expectations: Winter Intermediate language range is from 3 to 4 Growth: In Winter, Miguels language level increased from level 1 to 2, though not up to the Intermediate range, his scores demonstrated growth Category designation: His growth level is considered in the proficient range Miguel Is A 2nd Grade Student

20. A Demonstration Of Proficient Growth Each score, for each second grade student in the targeted ELL subgroup has been examined and categorized based on Cut Scores. Each available score-pair has been reviewed and growth has been categorized and weighted. Individual ratings were calculated The overall group rating was calculated j m Miguel = m[1,2,1,1] = 1 Rating = m[1, -1, 1, 2, 1] = 1 1 Proficient Growth

21. Alternate Illustration: Why Use More Than Two Administrations Of The Same Test From One Source j It is possible that different tests indicate different patterns, without changing the overall rating The differences in patterns by test may be diagnostically important, but insufficient for high stakes evaluative purposes English Language proficiency is growing at a rate above what is expected Automaticity with basic skills in Reading may need to improve Automaticity with basic skills in Math may be on track Instructional Vocabulary may be insufficient for continued growth in grade level material Growth English Language Proficiency is increasing at an adequate rate While there may be some areas where progress is worth further investigation, for the purposes of evaluating and categorizing; overall, across measures, academic growth is occurring at an acceptable rate.

22. Timmy Is A 4th Grade Student The data we might have for him include: Type I ISAT from grade 3 and 4 Aimsweb - Fall and Winter RCBM MCAP Type II Vocabulary Matching Type III Pre-Post Classroom Assessments What Data Might We Use For Him?

23. Gertrude Is A 7th Grade Student The data we might have for her include: Type I ISAT from grade 3 and 4 Aimsweb - Fall and Winter RCBM MCAP Type II Vocabulary Matching Prentice Hall Benchmark Assessments Type III Pre-Post Classroom Assessments What Data Might We Use For Her?

24. Setting Student Data Goals For Principal Evaluation By February 201x, given available Type I and Type II assessments administered at two points in time school wide, students at school name will demonstrate an increase in the proportion making adequate progress from 35% to 40%. Goals can be set to increase the amount of progress made

25. Setting Student Data Goals For Principal Evaluation By February 201x, given available Type I and Type II assessments administered at two points in time school wide, students identified as define cohort at school name will demonstrate an increase in the proportion making adequate progress from 68% to 80%. Goals can be set to increase the proportion of students making adequate progress made for a particular subgroup

26. Setting Student Data Goals For Principal Evaluation By February 201x, given available Type I and Type II assessments administered at two points in time school wide, students identified as define cohort at school name will demonstrate a decrease in the proportion making unsatisfactory progress from 8% to 4%. Goals can be set to decrease the proportion of students not making adequate progress for a particular subgroup

27. Setting Student Data Goals For Principal Evaluation By February 201x, given available Type I and Type II assessments administered at two points in time school wide, students at school name will demonstrate proficient or excellent growth as defined by the convergence and magnitude of data classified with district defined value tables. Goals can be set to achieve an overall rating for the demonstration of student growth