Forum - 1 Assessments for Learning: A Briefing on Performance-Based Assessments Eva L. Baker Director National Center for Research on Evaluation, Standards, & Student Testing Graduate School of Education & Information Studies University of California, Los Angeles The Forum for Education and Democracy October 20, 2008 © Regents of the University of California

Forum - 2 Performance Assessment Why now? What’s different? What about quality, fairness and cost? © Regents of the University of California

Forum - 3 © Regents of the University of California

Forum - 4 Why Performance Assessment? 21 st Century Skills! Adaptive problem-solving Situation awareness and risk assessment Decision-making Self-management Teamwork Learning to learn Communication Application of knowledge to new settings and situations © Regents of the University of California

Forum - 5 What’s Changed in Performance Assessments Correlations = ExchangeabilityLearning requires active patterns and performance SubjectivityRaters can be trained to agree almost perfectly Arbitrary and mindless, Not serious content Map to detailed specs of content, criteria Not fair to minoritiesBecause of sensitivity to instruction, evidence that all children can learn complex outcomes High costTechnology-based design reduces cost more than order of magnitude Myth Truth © Regents of the University of California

Forum - 6 How? Collaborate on detailed domains (ontologies) underlying only key standards Focus on important outcomes rather than covering a thin layer of standards Connect during up-front design to instruction and curriculum as well as measures Include specific measures of transfer and adaptation as well as information for improvement © Regents of the University of California

Forum - 7 Algebra Ontology © Regents of the University of California

Forum - 8 Physics Ontology © Regents of the University of California

Forum - 9 Novice Level 1 Flooding Ontology Situation Awareness and Communications © Regents of the University of California

Forum - 10 Collaborate on Careful Specifications of Domain Assure coverage of the key standards Provide transparent map of relationships for teaching, learning, and testing Guide professional development Allow task sampling for test to be systematic and increasing complex up the grade levels Allows weighting of tasks by importance, complexity, skill level so the test reflects proficiency © Regents of the University of California

Forum - 11 Technology’s Role Design reusable task components, cost sensitive formats, scoring criteria, and reporting Many performance assessments use computers for scoring and instant feedback Some approaches use well-trained raters to score, and some use mixed models Psychometric methods can make predictions about who needs what during instruction to make teaching more individual and adaptive Technology is repeatable and easily improved © Regents of the University of California

Forum - 12 Validity of Measures and Accountability Systems: What Evidence is Needed? Transparent and fair to all Multiple measures of learning 21 st century focus Sensitivity to good teaching Measures of classroom practice Accurate for all purposes Valid classification cuts Progress and improvement Technology and cost © Regents of the University of California

Forum - 13 Changing Educational Practices Diminishing attention to tests, more to problem solving, and evidence of actual performance in universities and businesses International competitors, e.g., South Korea, Singapore, are concerned with innovation, adaptation, and creativity, areas where U.S. students have historically been stronger © Regents of the University of California

Forum - 14 © Regents of the University of California 1.We cannot tolerate accountability systems that do not push our students to serious performance needed for satisfaction and for our own society’s competitive position in the world 2.Nor weak and indefensible low achievement in middle and secondary schools 3.Start here by changing NCLB measures to focus on applied as well as academic skills at the secondary level Where to Start?

Forum Eva L. Baker voice fax © Regents of the University of California

Forum - 16 Back Up © Regents of the University of California

Forum - 17 POWERSOURCE © Pre-Algebra Learning-based assessment Content and cognitive ontology Tasks Schema or pattern development Explanation Transfer and application Formative and certification use Supportive instruction and PD Small investment, big change © Regents of the University of California

Forum - 18 CRESST PA Experiences 1.Ages of students using range from 4-year-olds to post graduates and military 2.Special efforts with English Language Learners and underperforming students to assure fairness 3.Evidence of scale in multiyear performance assessment in Los Angeles (300K+ students annually) 4.Now preparing performance assessments in procedural and complex decision-making in military for critical tasks 5.Presently conducting studies in pre-algebra and algebra with positive early effects 6.Engaged in simulation and gaming forms of assessment 7.Roadmap for the design of performance assessments for accountability and other purposes © Regents of the University of California