Good Evaluation

Good Evaluation Should … be simple be fair be purposeful be related to the curriculum assess skills & strategies set priorities use multiple methods go from general to specific analyze errors substantiate findings record & report improve continuously

4 Attributes of a Good Evaluation Program utility: produce data to meet the information needs of its users feasibility: is realistic; prudent, diplomatic, and frugal

4 Attributes of a Good Evaluation Program propriety: is legal, ethical, providing for the welfare of those being evaluated & the users of the information Accuracy: reveals & technically conveys adequate information about the features that determine the worth or merit of the program being evaluated

Examining Your Own Beliefs What are the most essential things learners should know and do at the end of the program? Should learners be helped to create their own knowledge, or to master the knowledge that you and others deliver to them? Which is more important: collaborative or individualized learning? Should the instructor control the content and process of learning or should learners have more control?

Examining Your Own Beliefs Does learning happen mostly in systematic steps, mastering one skill or understanding one piece of knowledge before moving to the next? Or is learning mostly holistic, unpredictable, complex, and idiosyncratic [unique to each learner]? Does learning happen more in Ah-ha! insights, or through gradual evolutionary change? Should learners be able to demonstrate what they have learned in your program upon its conclusion, or will much of the learning be visible only at a later time?

Practices of Evaluation Do your methods for evaluating learners really show whether they have mastered the essentials they need in order to develop? Does your evaluation process allow students to reveal the knowledge they have personally constructed, or does it mostly compel students to display knowledge that you have constructed and they have memorized?

Practices of Evaluation Does the type of activity students must use during final evaluation match the type of activity they have learned during instruction? [Does your evaluation ever use procedures that learners have not been taught to use?] Does the language of the evaluation match the language used during instruction?

Practices of Evaluation Do your final evaluation procedures provide opportunities for students to really show what they know? [Do you ever try to trap students, or use certain evaluation methods simply because they will keep an acceptable spread in the marks?] How much do learners participate in the evaluation methods of your program?

Practices of Evaluation Do your evaluation methods match your instructional methods? Do your evaluation methods embody your deepest beliefs about teaching and learning? Can you justify each assignment or method of evaluation you use as a valid way of assessing your learners progress towards developing the most important elements in your program?

Helpful Evaluations Clarity: criteria & methods need to be crystal clear to all concerned Immediacy: make evaluation immediate so people get feedback right away Regularity: evaluation a regular part of instruction – more info more immediately equates to improved learning & teaching

Helpful Evaluations Accessibility: be available to provide clarification to learners Personalized: personalize the detail, but focus on the learners actions – a judgment about their work/efforts, not them Future-oriented: clear & specific suggestions for future actions re improving

Helpful Evaluations Justifiable: show reasons for the criticism, matching learner performance to the criteria required – show criticisms spring from a basic concern for students learning & a desire to assist them in reaching their own goals – what is correct/accepted practice not your own personal preference

Helpful Evaluations Educative: it is our responsibility to help students learn. Comments that are warm and sympathetic may make learners feel better, but are they educative. How can they improve their performance is key! Selective: avoid too much information. Focus on a few areas that will pay the highest dividends. One step at a time!

Helpful Evaluations - Summary These characteristics will ensure that the evaluation process is honest and helpful, while being as sensitive as possible to adult learner needs and the delicate learner-teacher relationship. Progress is enhanced when evaluation attends to learners feelings.

Planning for Evaluation Why should the evaluation take place? What should be evaluated? What do you want learners to know? What does the institution want to know? What do you want to know? What approaches should be used? How can you ensure validity & reliability? How much time/other resources do you have?

What Do You Want to Know? what the learners are thinking & feeling about what happens in the course whether the learners remembered what you taught in the last class or previous classes how adept learners are at test-taking and essay-writing, using techniques used in college/school/course

What Do You Want to Know what questions learners have or which parts of the content are confusing for them what kinds of associations learners are making between class experiences and their own life experiences how much time and effort learners are putting into the course work outside of class

What Do You Want to Know understand how the learner thinks whether learners can perform a certain skill effectively by themselves if the learners did the assigned class work what activity to do next, the one that will be most helpful what learners think about the learning activities