1. Forum - 1 Assessments for Learning: A Briefing on Performance-Based Assessments Eva L. Baker Director National Center for Research on Evaluation, Standards, & Student Testing Graduate School of Education & Information Studies University of California, Los Angeles The Forum for Education and Democracy October 20, 2008 © Regents of the University of California

2. Forum - 2 Performance Assessment Why now? What’s different? What about quality, fairness and cost? © Regents of the University of California

3. Forum - 3 © Regents of the University of California

4. Forum - 4 Why Performance Assessment? 21 st Century Skills! Adaptive problem-solving Situation awareness and risk assessment Decision-making Self-management Teamwork Learning to learn Communication Application of knowledge to new settings and situations © Regents of the University of California

5. Forum - 5 What’s Changed in Performance Assessments Correlations = ExchangeabilityLearning requires active patterns and performance SubjectivityRaters can be trained to agree almost perfectly Arbitrary and mindless, Not serious content Map to detailed specs of content, criteria Not fair to minoritiesBecause of sensitivity to instruction, evidence that all children can learn complex outcomes High costTechnology-based design reduces cost more than order of magnitude Myth Truth © Regents of the University of California

6. Forum - 6 How? Collaborate on detailed domains (ontologies) underlying only key standards Focus on important outcomes rather than covering a thin layer of standards Connect during up-front design to instruction and curriculum as well as measures Include specific measures of transfer and adaptation as well as information for improvement © Regents of the University of California

7. Forum - 7 Algebra Ontology © Regents of the University of California

8. Forum - 8 Physics Ontology © Regents of the University of California

9. Forum - 9 Novice Level 1 Flooding Ontology Situation Awareness and Communications © Regents of the University of California

10. Forum - 10 Collaborate on Careful Specifications of Domain Assure coverage of the key standards Provide transparent map of relationships for teaching, learning, and testing Guide professional development Allow task sampling for test to be systematic and increasing complex up the grade levels Allows weighting of tasks by importance, complexity, skill level so the test reflects proficiency © Regents of the University of California

11. Forum - 11 Technology’s Role Design reusable task components, cost sensitive formats, scoring criteria, and reporting Many performance assessments use computers for scoring and instant feedback Some approaches use well-trained raters to score, and some use mixed models Psychometric methods can make predictions about who needs what during instruction to make teaching more individual and adaptive Technology is repeatable and easily improved © Regents of the University of California

12. Forum - 12 Validity of Measures and Accountability Systems: What Evidence is Needed? Transparent and fair to all Multiple measures of learning 21 st century focus Sensitivity to good teaching Measures of classroom practice Accurate for all purposes Valid classification cuts Progress and improvement Technology and cost © Regents of the University of California

13. Forum - 13 Changing Educational Practices Diminishing attention to tests, more to problem solving, and evidence of actual performance in universities and businesses International competitors, e.g., South Korea, Singapore, are concerned with innovation, adaptation, and creativity, areas where U.S. students have historically been stronger © Regents of the University of California

14. Forum - 14 © Regents of the University of California 1.We cannot tolerate accountability systems that do not push our students to serious performance needed for satisfaction and for our own society’s competitive position in the world 2.Nor weak and indefensible low achievement in middle and secondary schools 3.Start here by changing NCLB measures to focus on applied as well as academic skills at the secondary level Where to Start?

15. Forum - 15 http://www.cse.ucla.edu Eva L. Baker 310.206.1530 310.267.0152 baker@cse.ucla.edu voice fax email © Regents of the University of California

16. Forum - 16 Back Up © Regents of the University of California

17. Forum - 17 POWERSOURCE © Pre-Algebra Learning-based assessment Content and cognitive ontology Tasks Schema or pattern development Explanation Transfer and application Formative and certification use Supportive instruction and PD Small investment, big change © Regents of the University of California

18. Forum - 18 CRESST PA Experiences 1.Ages of students using range from 4-year-olds to post graduates and military 2.Special efforts with English Language Learners and underperforming students to assure fairness 3.Evidence of scale in multiyear performance assessment in Los Angeles (300K+ students annually) 4.Now preparing performance assessments in procedural and complex decision-making in military for critical tasks 5.Presently conducting studies in pre-algebra and algebra with positive early effects 6.Engaged in simulation and gaming forms of assessment 7.Roadmap for the design of performance assessments for accountability and other purposes © Regents of the University of California