1. HP Technology for Teaching Worldwide Higher Education Conference Portola Plaza HotelMonterey, CA February 3, 2006 Measuring the Impact Daryl E. Chubin, Ph.D. American Association for the Advancement of Science www.aaascapacity.org

2. 2006 HP Technology for Teaching Conference Faculty Hold the Key While faculty members may try to do the best they can in class using familiar methods of instruction, they seldom work systematically at improving the methods themselves. Few faculties... assess how much their students are learning, identify deficiencies, develop and test possible remedies, and ultimately adopt those approaches that prove most successful. Derek Bok, The Chronicle of Higher Education, 12/16/05

3. 2006 HP Technology for Teaching Conference On Learning At bottom, a students real opportunity to learn is a function of his or her relation to a teacher in a classroom in a complex but specific learning environment. The best indicator of this opportunity... is student learningthe achievement of high levels of skills, knowledge, and understandings in challenging subject areas... Shirley Malcom, Promises to Keep, Report to the National Education Goals Panel, 11/15/93

4. 2006 HP Technology for Teaching Conference To Tell Your Story... Commit to keeping scoreand project evaluation Determine what matters and whats measurablequantify when possible Establish a baseline or pre condition Focus on value-added assessmentdifference between capability on entry and on exit (post) Find/adapt/use templates Put numbers into contextqualitative, descriptive, interpretive Reflect, and learn, from the experience!

5. 2006 HP Technology for Teaching Conference Project Evaluation: Analog to Program Evaluation Chief evaluation prototypes: Formative: ongoing, process, feedback Summative: gauging impact, attributing to intervention, claiming success See BEST Rating Form for Assessing Education Programs/ Practices, A Bridge for All (2004), www.bestworkforce.orgwww.bestworkforce.org

6. 2006 HP Technology for Teaching Conference

7. Core Data Elements Class Composition (Learners): The Study Population Program Character: Student activities Your behavior Learning Outcomes & Impacts: Short-term Longer-term Your adjustments/modifications

8. 2006 HP Technology for Teaching Conference Modes of Measurement Direct observation Analysis of documented student performance (assignments, major project, exams) Comparison of class performance with other course, college, or national disciplinary data Interviews & focus groups Consultation with other faculty familiar with the student, the course content, and/or the technology

9. 2006 HP Technology for Teaching Conference Guiding Principle Measure what you valueinstead of valuing what you measure...

10. 2006 HP Technology for Teaching Conference Other Sources of Inspiration, Practical Advice, & Existence Proofs National Center for Academic Transformation, The Roadmap to Redesign http://www.thencat.org/PlanRes/R2R_PrinCR.htm http://www.thencat.org/PlanRes/R2R_PrinCR.htm Project Kaleidoscope, What Works Visions http://www.pkal.org/collections/WhatWorks.cfm

11. 2006 HP Technology for Teaching Conference On Change "People hate change when it is someone else's plan, when it is imposed on them, when they are told what to do and exactly how they must do it, when they are threatened with punishment if they don't do it. People love change when it is shaped by them, when they are in control of it, when it is their chance to make a difference. In fact, then they don't even call it 'change' it's a project, a venture, a dream come to life. It's their passion turned into a professional pursuit." Rosabeth Moss Kanter, 2004 May we all catch this spirit!

12. 2006 HP Technology for Teaching Conference Breakout Groups will... Engage in table conversations/ brainstorming Start to fill in the project-specific worksheet Report out, one per table, illustrative variables that fit one or more project examples Be better prepared to report/confer with team back home

13. 2006 HP Technology for Teaching Conference Then the Conversation Continues... Email me: dchubin@aaas.orgdchubin@aaas.org Call me: 202-326-6785 Consult the web:www.aaascapacity.orgwww.aaascapacity.org