Presentation is loading. Please wait.

Presentation is loading. Please wait.

TEACHER VALUE-ADDED REPORTS STATE BOARD OF EDUCATION JANUARY 14, 2014 Jamie Meade Managing Director, Strategic Measures Battelle for Kids.

Similar presentations


Presentation on theme: "TEACHER VALUE-ADDED REPORTS STATE BOARD OF EDUCATION JANUARY 14, 2014 Jamie Meade Managing Director, Strategic Measures Battelle for Kids."— Presentation transcript:

1 TEACHER VALUE-ADDED REPORTS STATE BOARD OF EDUCATION JANUARY 14, 2014 Jamie Meade Managing Director, Strategic Measures Battelle for Kids

2 © 2013, Battelle for Kids. All Rights Reserved. Session Objectives  Provide value-added information relevant to the practitioner’s perspective.  What should teachers and administrators know and understand about value-added reporting?  How may educators use value-added information to improve professional practice and impact student academic achievement and progress?

3 © 2013, Battelle for Kids. All Rights Reserved. Value-Added in Ohio  For over 10 years, Battelle for Kids has provided support for professional learning and fostered collegial dialogue for understanding and using value-added measures.  Battelle for Kids continues to advocate for the use of value-added measures, in combination with other educational measures, to improve practice and accelerate student academic progress.

4 © 2013, Battelle for Kids. All Rights Reserved. Ohio’s Value-Added History  2002: Battelle for Kids’ SOAR Collaborative School and District Value-Added Reports  2006: Battelle for Kids’ TCAP Project Teacher Value-Added Reports  2007: ODE Value-Added on Local Report Card  2009: Battelle for Kids’ Ohio VA High Schools  2011: RttT: 30% Teachers, VA Reports  2012: RttT: 60% Teachers, VA Reports  2013: 4-8 Reading & Math Teachers, VA Reports 80 RttT Districts, K-3 R & M Teachers, VA Reports

5 © 2013, Battelle for Kids. All Rights Reserved. Value-Added in Ohio  Educational Value Added Assessment System (EVAAS)  SAS Analytics  Customers in 135 countries  More than 65,000 business, government and university sites  SAS customers or their affiliates represent 90 of the top 100 companies on the 2012 FORTUNE Global 500 ® list.

6 © 2013, Battelle for Kids. All Rights Reserved. Value-Added Information in Practice: Building Awareness  Understanding the Difference  Achievement Measures  Progress / Growth Measures

7 © 2013, Battelle for Kids. All Rights Reserved. Achievement & Growth: Understanding the Difference Achievement Growth

8 © 2013, Battelle for Kids. All Rights Reserved. Measuring Growth is Important for ALL Students Proficient Grad e 3 rd 4 th 5 th 6 th 7 th 8 th Jacob Adam

9 © 2013, Battelle for Kids. All Rights Reserved. “Why can’t we simply compare OAA scaled scores from one year to the next to measure growth?” Level3 rd 4 th 5 th 6 th 7 th 8 th Advanced Accelerated Proficient Basic Limited OAA Math Scaled Score Ranges May 2013

10 © 2013, Battelle for Kids. All Rights Reserved. Questions?

11 © 2013, Battelle for Kids. All Rights Reserved. Value-Added Information in Practice: SAS EVAAS MRM Model  Mutlivariate Response Model (MRM)  Grades 4–8 Reading and Math VA Reports  Uses Ohio Achievement Assessments

12 © 2013, Battelle for Kids. All Rights Reserved. SAS EVAAS Value-Added MRM Model*  Uses Grades 3 – 8 Reading and Math OAAs.  Compares the average growth of students in the most recent year to the average growth of students in 2010 (state’s baseline year)*  Growth expectation is defined as maintaining placement in the distribution of NCE scores from one year to the next* *conceptual definition

13 © 2013, Battelle for Kids. All Rights Reserved. Raw Score Scaled Score NCE Sample Raw Score Range Sample Scaled Score Range Normal Curve Equivalent NCE

14 © 2013, Battelle for Kids. All Rights Reserved. Scaled Scores Converted to NCEs in State’s 2010 Baseline Year Example 2010 Scaled Scores Rank Ordered Normal Curve Equivalent (NCE) 99 1 Conversion Values Are Fixed / Frozen

15 © 2013, Battelle for Kids. All Rights Reserved. Value-Added Terminology Normal Curve Equivalent (NCE)  The NCE is similar to a percentile rank in that scores are derived from scaled scores and ranked based upon performance.  A significant difference between percentile rank and NCE is that an NCE scale is an equal interval scale.

16 © 2013, Battelle for Kids. All Rights Reserved. Normal Curve Equivalent (NCE) Distribution of Scores Normal Curve Equivalents Percentile Equivalents

17 © 2013, Battelle for Kids. All Rights Reserved. Value-Added Terminology  Baseline Score  Group of students’ prior year mean NCE  Example: Spring 2012 OAA mean NCE  Observed Score  Group of students’ new/most recent mean NCE  Example: Spring 2013 OAA mean NCE

18 © 2013, Battelle for Kids. All Rights Reserved. SAS EVAAS MRM Model Basic, Conceptual Example Baseline OAA th NCE Student 2 55 NCE Student 3 92 NCE Student 1 20 NCE Observed OAA th NCE Student 2 67 NCE (+12) Student 1 20 NCE (0) Expected Growth Maintain Placement in Distribution of Scores For example, a student at the 20 th NCE must “at least” stay at the 20 th NCE Student 3 89 NCE (-3)

19 © 2013, Battelle for Kids. All Rights Reserved. Basic, Conceptual Example  Scaled scores are converted to NCEs 394 = 46Student 1430 = = 50Student 2417 = = 42Student 3400 = = 46Student 4390 = = 52Student 5425 = 57 Mean Baseline = 47.2 Grade 6 Baseline Grade 7 Observed Mean Observed = 52.6 A basic measure of the growth for this group is 5.4 NCEs Growth = Mean Observed – Mean Baseline Growth = 52.6 – 47.2 = 5.4 (Mean NCE Gain)

20 © 2013, Battelle for Kids. All Rights Reserved. Questions?

21 © 2013, Battelle for Kids. All Rights Reserved. Levels of Value-Added Effects Students are making substantially more progress than the state growth standard. Students are making more progress than the state growth standard. Students are making about the same amount of progress as the state growth standard. Students are making less progress than the state growth standard. Students are making substantially less progress than the state growth standard.

22 © 2013, Battelle for Kids. All Rights Reserved.

23 Standard Error  A measure of the uncertainty  All measures of student learning contain error.  In the EVAAS teacher value-added report, the size of the standard error is influenced by  N size (size of the student group).  Missing scores.

24 © 2013, Battelle for Kids. All Rights Reserved. Year 1 Estimate = True value lies somewhere within the range of the standard error Estimate is Most likely value Y1 Y2Y3 Multi-yr. Ave.

25 © 2013, Battelle for Kids. All Rights Reserved. Teacher Value-Added Report Note: Battelle for Kids is utilizing visual representations of copyrighted EVAAS ® Web reporting software from SAS in this presentation for instructional purposes.

26 © 2013, Battelle for Kids. All Rights Reserved. Value-Added Terminology  Growth Index  Since the size of the standard error (degree of certainty) will vary across teachers, their estimated gain must be standardized to include both the estimate and the degree of certainty (standard error).  Divides a teacher’s estimated gain by the associated standard error.

27 © 2013, Battelle for Kids. All Rights Reserved. The Teacher Value-Added Report

28 © 2013, Battelle for Kids. All Rights Reserved. The Teacher Progress Table

29 © 2013, Battelle for Kids. All Rights Reserved. Levels of Teacher Value-Added Effects Most Effective Teacher's index: 2 or greater Students are making substantially more progress than the state growth standard. Above Average Teacher's index: equal to or greater than 1, but less than 2 Students are making more progress than the state growth standard. Average Teacher's index: equal to or greater than -1, but less than 1 Students are making about the same amount of progress as the state growth standard. Approaching Average Teacher's index: equal to or greater than -2, but less than -1 Students are making less progress than the state growth standard. Least Effective Teacher's index: less than -2 Students are making substantially less progress than the state growth standard.

30 © 2013, Battelle for Kids. All Rights Reserved. Foundational Statistics GR Ohio’s Academic Content Standards Mathematics: Statistical Methods 10 Interpret the relationship between two variables using multiple graphical displays and statistical measures (e.g., box-and-whisker plots and measures of center and spread). 8 Describe how the relative size of a sample compared to the target population affects the validity of predictions. Explain the mean’s sensitivity to extremes… 7 …describe how the inclusion and exclusion of outliers affect those measures. 6 Understand the different information provided by measures of center (mean, mode, median) and measures of spread (range). 1 Describe the likelihood of simple events as possible/impossible and more likely/less likely.

31 © 2013, Battelle for Kids. All Rights Reserved. Key Considerations:  Systemic Programs, Delivery Models, Structures, Services, etc.  Professional Learning  Curriculum Alignment What I teach  Assessment How I measure/monitor learning along the way  Instruction How I teach Using Value-Added to Inform Practice

32 © 2013, Battelle for Kids. All Rights Reserved. Combining Measures to Inform Practice  All measures of student learning contain error.  No single measure can capture the complexity of learning and teaching.  There’s an important distinction between a flawed measure and a flawed assumption based upon a single measure.

33 @BattelleforKids facebook.com/battelleforkidsorg youtube.com/battelleforkids BattelleforKids.org Questions? Thank You!


Download ppt "TEACHER VALUE-ADDED REPORTS STATE BOARD OF EDUCATION JANUARY 14, 2014 Jamie Meade Managing Director, Strategic Measures Battelle for Kids."

Similar presentations


Ads by Google