1. NCLB AND VALUE-ADDED APPROACHES ECS State Leader Forum on Educational Accountability June 4, 2004 Stanley Rabinowitz, Ph.D. WestEd srabino@wested.org

2. GUIDING QUESTIONS 1.What are the goals/assumptions/values of Value-Added accountability models? 2.What are the pros and cons of Value-Added accountability models? 3.What are the requirements of Value-Added accountability models (data, technical, political)? 4.Are there alternatives that provide “similar” information?

3. ACCOUNTABILITY MODELS Status Cohort Same students (unmatched) Same students (matched)—“Value Added” ------------------------------------------------------ All models are defensible (assumptions, values) All models can be interpreted as “growth”

4. ACCOUNTABILITY MODELS (cont.) Multiple (Simultaneous) Models - two tier (funnel into state system) - federal and state system (side-by-side) - combine status and growth - formal and informal systems (program improvement) Advantages of multiple systems - differing assumptions may be appropriate for different audiences and purposes - approval Disadvantages of multiple systems - need to administer multiple systems - interpret/explain differing results

5. CHALLENGES OF A VALUE-ADDED SYSTEM Values Assessment - content - technical Data systems Political

6. CHALLENGES OF A VALUE-ADDED SYSTEM: VALUES Concept of “one-year’s growth” (across grades, content areas, students) Individual student growth most important measure of school effectiveness (student by student) Classroom/teacher effect most important factor in student success (vs. school, district, home, community) Student test scores are a proxy for effective teaching Changes in standardized test scores are what is valued

7. CHALLENGES OF A VALUE-ADDED SYSTEM: ASSESSMENT Test questions of varying difficulty in order to adequately discriminate among the range of achievers typically found in a classroom, school, district, state Sufficient validity and reliability of the underlying instrument(s) (reliability of difference scores) A set of scaled tests, vertically equated, aligned to the curriculum NRT vs. CRT

8. Longitudinally merged student data base (unique student identifiers) Student data need to be traceable back to the focus of accountability (teacher, classroom, school, district…) Data to be controlled for other factors (e.g., socioeconomic indicators, disabilities) Unmatched students? Mobility across state and across schools? Multiple teachers? CHALLENGES OF A VALUE-ADDED SYSTEM: DATA SYSTEMS

9. NCLB status Black box vs. transparency of system Specific attribution of blame/responsibility Expected opposition Large year-to-year swings in value-added scores that administrators could not explain Assumption that all learning takes place in the classroom, although the model "has not provided adequate evidence to support this contention" CHALLENGES OF A VALUE-ADDED SYSTEM: POLITICAL

10. Fully aligned content standards Fully aligned performance standards Sufficient variability within performance categories (e.g., break out basic into low, medium, high basic) Accountability based on growth of individual students or groups (within and across categories) Assign point values for different types of growth - value of reaching proficient - value at other points (advanced, at least basic) - value of lowest category (0?) Focus attention at performance labels, especially Proficient Lower technical requirements Easier political discussion PSEUDO VALUE-ADDED MODEL