NCLB AND VALUE-ADDED APPROACHES ECS State Leader Forum on Educational Accountability June 4, 2004 Stanley Rabinowitz, Ph.D. WestEd

Slides:

Advertisements

Similar presentations

Using Growth Models to improve quality of school accountability systems October 22, 2010.

Advertisements

Administering K-2 WESTELL. Overview l Purpose and Nature of WESTELL K-2 l Language Acquisition in Young Children l What You Will Need l Administration.

Assessment and Accountability at the State Level NAEP NRT (Iowa Tests) Core CRTs DWA UAA UBSCT NCLB U-PASS Alphabet Soup.

VALUE – ADDED 101 Ken Bernacki and Denise Brewster.

Issues of Technical Adequacy in Measuring Student Growth for Educator Effectiveness Stanley Rabinowitz, Ph.D. Director, Assessment & Standards Development.

OVERVIEW OF CHANGES TO EDUCATORS’ EVALUATION IN THE COMMONWEALTH Compiled by the MOU Evaluation Subcommittee September, 2011 The DESE oversees the educators’

Forum on Educational Accountability Assessment and ESEA Reauthorization.

Tracking Test Scores to Address Educational Accountability Standards in a Public School System Faculty Sponsor: George Tremblay, Ph.D. Antioch University.

Robert L. Linn CRESST, University of Colorado at Boulder Paper presented at a symposium sponsored by the National Association of Test Directors entitled.

MCAS-Alt: Alternate Assessment in Massachusetts Technical Challenges and Approaches to Validity Daniel J. Wiener, Administrator of Inclusive Assessment.

New Hampshire Enhanced Assessment Initiative: Technical Documentation for Alternate Assessments Standard Setting Inclusive Assessment Seminar Marianne.

Robert L. Linn CRESST, University of Colorado at Boulder Paper presented at a symposium sponsored entitled “Accountability: Measurement and Value-Added.

Using Growth Models for Accountability Pete Goldschmidt, Ph.D. Assistant Professor California State University Northridge Senior Researcher National Center.

Consistency/Reliability

Meeting NCLB Act: Students with Disabilities Who Are Caught in the Gap Martha Thurlow Ross Moen Jane Minnema National Center on Educational Outcomes

1 Standards, Curriculum, and Research Mathematically Connected Communities (MC 2 ) Adapted from a PowerPoint by Barbara A. Austin, Ph.D.

New Hampshire Enhanced Assessment Initiative: Technical Documentation for Alternate Assessments Alignment Inclusive Assessment Seminar Brian Gong Claudia.

San Juan High School Intermediate Intervention/Under-performing School Program By Mike Peebles, Teacher Partial fulfillment of ED251 Instructors: Duane.

1 The New York State Education Department New York State’s Student Reporting and Accountability System.

Title IIA: Connecting Professional Development with Educator Evaluation June 1, 2015 Craig Waterman.

Office of Institutional Research, Planning and Assessment January 24, 2011 UNDERSTANDING THE DIAGNOSTIC GUIDE.

John Cronin, Ph.D. Director The Kingsbury NWEA Measuring and Modeling Growth in a High Stakes Environment.

Educator Evaluations: Growth Models Presentation to Sand Creek Schools June 13, 2011.

Addressing Student Growth In Performance Evaluation For Teachers and Administrators Patricia Reeves.

NEXT GENERATION BALANCED ASSESSMENT SYSTEMS ALIGNED TO THE CCSS Stanley Rabinowitz, Ph.D. WestEd CORE Summer Design Institute June 19,

+ Equity Audit & Root Cause Analysis University of Mount Union.

SB : The Great Teachers and Leaders Act State-wide definition of “effective” teacher and principal in Colorado Academic growth, using multiple measures.

Fall Testing Update David Abrams Assistant Commissioner for Standards, Assessment, & Reporting Middle Level Liaisons & Support Schools Network November.

Qatar Comprehensive Educational Assessment (QCEA) 2007: Summary of Results.

Carlos Martínez Group Leader – Standards and Assessments U.S. Department of Education.

CCSSO Criteria for High-Quality Assessments Technical Issues and Practical Application of Assessment Quality Criteria.

Measured Progress ©2012 Student Growth in the Non-Tested Subjects and Grades: Options for Teacher Evaluators Elena Diaz-Bilello, Center for Assessment.

Project on Educator Effectiveness & Quality Chancellor Summit September 27, 2011 Cynthia Osborne, Ph.D.

Issues in Assessment Design, Vertical Alignment, and Data Management : Working with Growth Models Pete Goldschmidt UCLA Graduate School of Education &

Growth Model for District “X” Why Use Growth Models? Showing progress over time is a more fair way of evaluating It is not just a “snap shot” in time.

PROGRESS & & ACHIEVEMENT Pennsylvania Value-Added Assessment System (PVAAS) The Power of Derry Township School District School Board Presentation Sept.

© 2007 Board of Regents of the University of Wisconsin System, on behalf of the WIDA Consortium WIDA Focus on Growth H Gary Cook, Ph.D. WIDA.

The hybrid success model: Theory and practice G. Gage Kingsbury Martha S. McCall Northwest Evaluation Association A paper presented to the Seminar on longitudinal.

Copyright © 2009 National Comprehensive Center for Teacher Quality. All rights reserved. Evaluating Teacher Effectiveness: Some Models to Consider Laura.

“Value added” measures of teacher quality: use and policy validity Sean P. Corcoran New York University NYU Abu Dhabi Conference January 22, 2009.

Gary W. Phillips American Institutes for Research United States Department of Education Public Hearings December 1, 2009, Denver, Colorado.

Michigan School Report Card Update Michigan Department of Education.

Assessment: Reliability & Validity. Reliability Refers to the repeatability of a given testing instrument The extent to which a student would be expected.

Student Learning and Growth Goals Foundations 1. Outcomes Understand purpose and requirements of Student Learning and Growth (SLG) goals Review achievement.

ANNOOR ISLAMIC SCHOOL AdvancEd Survey PURPOSE AND DIRECTION.

Gary W. Phillips American Institutes for Research CCSSO 2014 National Conference on Student Assessment (NCSA) New Orleans June 25-27, 2014 Multi State.

1 Scoring Provincial Large-Scale Assessments María Elena Oliveri, University of British Columbia Britta Gundersen-Bryden, British Columbia Ministry of.

Educator Effectiveness Summit School District’s Recommendation for the School Year.

Application of Growth and Value-Added Models to WASL A Summary of Issues, Developments and Plans for Washington WERA Symposium on Achievement Growth Models.

Teacher Evaluation Systems 2.0: What Have We Learned? EdWeek Webinar March 14, 2013 Laura Goe, Ph.D. Research Scientist, ETS Sr. Research and Technical.

Aligning Assessments to Monitor Growth in Math Achievement: A Validity Study Jack B. Monpas-Huber, Ph.D. Director of Assessment & Student Information Washington.

No Child Left Behind Impact on Gwinnett County Public Schools’ Students and Schools.

Office of Service Quality

Introduction to the Pennsylvania Kindergarten Entry Inventory.

Weighting components of teacher evaluation models Laura Goe, Ph.D. Research Scientist, ETS Principal Investigator for Research and Dissemination, The National.

C R E S S T / CU University of Colorado at Boulder National Center for Research on Evaluation, Standards, and Student Testing Measuring Adequate Yearly.

LISA A. KELLER UNIVERSITY OF MASSACHUSETTS AMHERST Statistical Issues in Growth Modeling.

VAM Training. Florida’s value-added model developed by Florida educators  The Department convened a committee of stakeholders (Student Growth Implementation.

Forum on Evaluating Educator Effectiveness: Critical Considerations for Including Students with Disabilities Lynn Holdheide Vanderbilt University, National.

Value Added Model Value Added Model. New Standard for Teacher EvaluationsNew Standard for Teacher Evaluations Performance of Students. At least 50% of.

Lesson Thirteen Standardized Test. Contents Components of a Standardized test Reasons for the Name “Standardized” Reasons for Using a Standardized Test.

 Mark D. Reckase.  Student achievement is a result of the interaction of the student and the educational environment including each teacher.  Teachers.

Take-Home Message: Principles Unique to Alternate Assessments

Smarter Balanced Assessment Results

2015 PARCC Results for R.I: Work to do, focus on teaching and learning

Interim Assessments: Do you know what you are buying and why

Presentation at CCSSO NCSA Conference

Driving Through the California Dashboard

AACC Mini Conference June 8-9, 2011

Jennifer Rodriguez TB610-60

Presentation transcript:

NCLB AND VALUE-ADDED APPROACHES ECS State Leader Forum on Educational Accountability June 4, 2004 Stanley Rabinowitz, Ph.D. WestEd

GUIDING QUESTIONS 1.What are the goals/assumptions/values of Value-Added accountability models? 2.What are the pros and cons of Value-Added accountability models? 3.What are the requirements of Value-Added accountability models (data, technical, political)? 4.Are there alternatives that provide “similar” information?

ACCOUNTABILITY MODELS Status Cohort Same students (unmatched) Same students (matched)—“Value Added” All models are defensible (assumptions, values) All models can be interpreted as “growth”

ACCOUNTABILITY MODELS (cont.) Multiple (Simultaneous) Models - two tier (funnel into state system) - federal and state system (side-by-side) - combine status and growth - formal and informal systems (program improvement) Advantages of multiple systems - differing assumptions may be appropriate for different audiences and purposes - approval Disadvantages of multiple systems - need to administer multiple systems - interpret/explain differing results

CHALLENGES OF A VALUE-ADDED SYSTEM Values Assessment - content - technical Data systems Political

CHALLENGES OF A VALUE-ADDED SYSTEM: VALUES Concept of “one-year’s growth” (across grades, content areas, students) Individual student growth most important measure of school effectiveness (student by student) Classroom/teacher effect most important factor in student success (vs. school, district, home, community) Student test scores are a proxy for effective teaching Changes in standardized test scores are what is valued

CHALLENGES OF A VALUE-ADDED SYSTEM: ASSESSMENT Test questions of varying difficulty in order to adequately discriminate among the range of achievers typically found in a classroom, school, district, state Sufficient validity and reliability of the underlying instrument(s) (reliability of difference scores) A set of scaled tests, vertically equated, aligned to the curriculum NRT vs. CRT

Longitudinally merged student data base (unique student identifiers) Student data need to be traceable back to the focus of accountability (teacher, classroom, school, district…) Data to be controlled for other factors (e.g., socioeconomic indicators, disabilities) Unmatched students? Mobility across state and across schools? Multiple teachers? CHALLENGES OF A VALUE-ADDED SYSTEM: DATA SYSTEMS

NCLB status Black box vs. transparency of system Specific attribution of blame/responsibility Expected opposition Large year-to-year swings in value-added scores that administrators could not explain Assumption that all learning takes place in the classroom, although the model "has not provided adequate evidence to support this contention" CHALLENGES OF A VALUE-ADDED SYSTEM: POLITICAL

Fully aligned content standards Fully aligned performance standards Sufficient variability within performance categories (e.g., break out basic into low, medium, high basic) Accountability based on growth of individual students or groups (within and across categories) Assign point values for different types of growth - value of reaching proficient - value at other points (advanced, at least basic) - value of lowest category (0?) Focus attention at performance labels, especially Proficient Lower technical requirements Easier political discussion PSEUDO VALUE-ADDED MODEL