1. SCOTT MARION, CENTER FOR ASSESSMENT PRESENTATION AT CCSSO NCSA AS PART OF THE SYMPOSIUM ON: STUDENT GROWTH IN THE NON-TESTED SUBJECTS AND GRADES: OPTIONS FOR TEACHER EVALUATORS JUNE 25, 2014 A Body of Evidence for Documenting Student Learning and Growth

2. Common “Assignment” Project Marion. CAS-Educator Evaluation. CCSSO_2014 2 More properly, Common “Unit” Study… R & D project in which teachers collaborate across districts and states to develop and implement high-quality “replacement” curricular units to ensure student learning of meaningful disciplinary content and skills

3. “Replacing a Unit of Study” requiring deeper thinking and learning A “Replacement Unit” of Study… June Unit of Study August Unit of Study … allows us to take a previous unit of study and “replace” it with a NEW unit of study *that is aligned to the CCSS/NGSS. Unit of Study Marion. CAS-Educator Evaluation. CCSSO_2014 3

4. Common Assignment Research Strands Marion. CAS-Educator Evaluation. CCSSO_2014 4 Strand 1 Design & Use Strand 2 Resources & Support Strand 3 Teacher Evaluation Common Assignment Project

5. Theory of Action as Design & Evaluation Marion. CAS-Educator Evaluation. CCSSO_2014 5 Theories of action can (should) be used both as purposeful design and as a framework for validity evaluation We use a theory of action to identify the intended program activities and outcomes to help prioritize the evaluation efforts We also think it can be a helpful check on our design activities

6. Strands of Research & Evaluation Assumptions Problem Statement What are effective and efficient approaches for supporting the deeper learning and teaching of important knowledge and skills Units foster collaboration and frame teacher expectations Design, implementation, and use of common units Units are appropriate for influencing change Resources and supports necessary to build capacity Program resources and supports will be available to teachers Feasibility for educator evaluation An Overview of the Theory of Action Marion. CAS-Educator Evaluation. CCSSO_2014 6

7. Data examined to determine extent it is useful for making inferences about effectiveness Evaluation systems are modified Educator evaluation is based on common student work Determining feasibility of using common units as part of teacher evaluation Data from common units provides a more valid indicator of educator quality than assessment data alone Student work from common units will be collected as sources of evidence Activities Ultimate Effects Intermediate Effects Identify opportunities and challenges associated with using student work from common units Use of data from common units increases educator buy-in Strand 3: Feasibility for Educator Evaluation Marion. CAS-Educator Evaluation. CCSSO_2014 7

8. Prioritized Research Questions (Strand 3) Marion. CAS-Educator Evaluation. CCSSO_2014 8 1. What information is available to support inferences about teachers’ contributions to student learning? 2. Do the data gathered throughout the units add useful information to inferences about teacher quality drawn solely from the end-of-year assessments? 3. Do teacher find the information from multiple Common Units administered throughout the year more credible for evaluation than single EOY assessments?

9. What information is available to support inferences about teachers’ contributions to student learning? Marion. CAS-Educator Evaluation. CCSSO_2014 9 Analytic approach  Student work from common assessments will be closely analyzed within and across (fall-spring) units to characterize differences in student learning at a point in time as well as over time  Use “scores,” but need to move to finer-grain info  How much does the evidence differ across units by content and grade level? Major challenge: If we can find and document changes in student learning, at least descriptively, can we quantify it for use in large-scale evaluation systems?

10. Do the data gathered throughout the units add useful information to inferences drawn solely from the end-of-year assessments? Marion. CAS-Educator Evaluation. CCSSO_2014 10 Analytic approach  Comparison of “model-data fit” between end-of-year and common unit data  Determine whether utility of unit data differs by content or grade level  Determine if there are features of units that provide more useful information for making inferences about teacher quality  Comparison of inferences about teacher performance with and without CAS data

11. Next Design Steps Marion. CAS-Educator Evaluation. CCSSO_2014 11 This was a “proof of concept” year to ensure that teachers across districts and states could collaborate to develop high quality units and associated assessments The next two years will focus the design of the two units around a common “big idea” of the discipline  Like the way that the learning goal SLO should be tied to a major idea of the discipline Increased focus around designing high quality assessments to produce comparable scores and evidence

12. SLO as a Framework Marion. CAS-Educator Evaluation. CCSSO_2014 12 SLOs can provide the structural and analytic frame for analyzing the work associated with units tied to a common big idea We have seen too many cases where a big idea (learning goal of the SLO) is evaluated with a single assessment If it is truly a big idea, it will most certainly require evidence from multiple assessments The common units provide multiple and rich sources of evidence tied to big idea

13. Why Should We Care? Marion. CAS-Educator Evaluation. CCSSO_2014 13 People welcome accountability when they feel they have some control of the processes and the ability to influence the outcomes  Includes understanding what “improvement” means Need ways to document teacher contributions to student learning that are genuinely integrated with the real work of teaching and learning We do not need more “drop from the sky” assessments Our goal is to reduce the temptation to corrupt the processes and results