1. Value Added Model Value Added Model

2. New Standard for Teacher EvaluationsNew Standard for Teacher Evaluations Performance of Students. At least 50% of a performance evaluation must be based upon data and indicators of student learning growth assessed annually and measured by statewide assessments or, for subjects and grade levels not measured by statewide assessments, by district assessments as provided in s. 1008.22(8), F.S. -Section 1012.34(3)(a)1., Florida Statutes

3. Florida’s Value-Added Model Developed by Florida Educators  The Department convened a committee of stakeholders (Student Growth Implementation Committee –or SGIC) to identify the type of model and the factors that should be accounted for in Florida’s value-added models  To provide technical expertise, the Department contracted with the American Institutes for Research (AIR) to help the SGIC develop the recommended model that was adopted.

4.  After exploring eight different types of value-added models, the SGIC recommended a model from the class of covariate adjustment models  This model begins by establishing expected growth for each student:  Based on historical data each year  Represents the typical growth seen among students who have earned similar test scores the past two years and share the other characteristics identified by the committee Florida’s Value-Added Model Developed by Florida Educators

5. Value-Added ModelsValue-Added Models  “Level the playing field” by accounting for differences in the proficiency and characteristics of students assigned to teachers; teachers do not have advantages or disadvantages simply as a result of the students who attend a school and are assigned to a class.  To isolate the impact of the teacher on student learning growth, the model developed by the SGIC and approved by the Commissioner accounts for:  Student Characteristics  Classroom Characteristics  School Characteristics

6. Factors Identified by the SGIC to “Level the Playing Field”  Student Characteristics:  Up to two prior years of achievement scores  The number of subject-relevant courses in which the student is enrolled  Students with Disabilities (SWD) status  English Language Learner (ELL) status  Gifted status  Attendance  Mobility (number of transitions)  Difference from modal age in grade  Classroom characteristics:  Class size  Homogeneity of students’ entering test scores in the class

7. Quick RecapQuick Recap  Florida’s Model (FCAT Reading and Math grades 4+) is a covariate adjustment model  Covariate adjustment is a fancy way to say we use prior test scores and identified student characteristics as predictor variables that influence the outcome variable (what a student scores)

8. It Looks Like This…It Looks Like This… Outcome Variable How it is calculated

9. It Looks Like This…for Each StudentIt Looks Like This…for Each Student

11. We decide on the following scale: 0-23 inchesLevel 1 23-46 inchesLevel 2 47-70 inchesLevel 3 71-94 inchesLevel 4 Above 94 inchesLevel 5 Sound familiar? This is analogous to using a Proficiency Model like for school grades. Based on this model, which gardener was more successful?

17. What Happens Next?What Happens Next?  A VAM Score is calculated for each FCAT subject teacher by aggregating student data for each year, grade, and subject  This is done for three years (where available) and may include scores for multiple grade levels and subjects  For example:  A fifth grade teacher may have 6 data sets and scores:  3 years each of math and reading  A first-year, seventh grade math teacher would only have one data set and score

18. What Happens Next?What Happens Next?  Last year, each district was allowed to model the data based on their state-approved plan.  Common metric, weighted averages, comparison to state percentiles. etc.  Proposed state board rule would standardize how final categories are assigned.  Let’s take a look at a couple of models…

19. What Happens Next? Weighted Averages Model  A Performance Category is determined for each FCAT Subject teacher using a Weighted Averages Model  This model deals with differences by classifying independently by year, grade, and subject then sums with weighted values to determine a final category  Didn’t follow that? Let’s look at an example…

20. Weighted Averages ExampleWeighted Averages Example Chuck Norris teaches 9 th and 10 th grade math and has for the last three years. The color coding for the VAM Estimate is determined by positive or negative. This indicates a cut point of zero. Zero is used because it represents expected performance. In other words, above zero is above expectation and below zero is below expectation.

21. Weighted Averages ExampleWeighted Averages Example Next we take into account the Standard Error associated with each calculation through the use of a confidence interval (K). This helps us refine a performance category with more statistical certainty. Performance categories: Highly Effective, Effective, Needs Improvement (Developing), Unsatisfactory

22. Weighted Averages ExampleWeighted Averages Example Once we determine a performance category for each data set, it is weighted by the number of students and aggregated across years, grades, and subjects for a final score and category.

23. Common Metric ExampleCommon Metric Example Common metric does most of the work up front by putting all data on the same scale. This is done be dividing by a year’s growth, so the metric becomes a “proportion of a year’s growth above or below expectation.” It has the same interpretation across subjects and years. agg_vam_read agg_vam_read _se agg_vam_mat h agg_vam_mat h_se agg_vam_com bined agg_vam_com bined_se K=0 (intial cut method)K=.5K=1 Final Rating for K=.5 Final Rating for K=1 -0.044870.074913-0.179450.124325-0.108270.077521NI/UN-0.06951-0.03074UN Common Metric is the model proposed for the state.

27. Achievement/Proficiency Growth/VAM Teacher Observation/Feedback ? ? ? !