Evaluation of Student Learning Third Summer Leadership Institute Amherst, MA Center for School Counseling Outcome Research

Some Preliminary Evaluation Questions ► How do we know that students have learned what we intended or hoped they would? ► What types of learning are you looking for on Bloom’s taxonomy? ► What will you accept as evidence of learning? ► What is evidence of mastery? ► Do you want all students to reach mastery? Some percentage?

Purposes for Evaluation of Learning ► Feedback to students ► Feedback to teachers ► Feedback to parents ► Information for selection and certification ► Information for accountability ► Information to increase student effort

Increasing Student Effort ► Evaluation must matter or be important to students (parents care or college admission) ► Evaluation must be tied to actual performance (honest, objective measures) ► Consistent standards (fair and equal) ► Clear criteria (how to get a good grade) ► Reliable interpretations of evaluations ► Frequent evaluations (quizzes rather than finals) ► Challenging evaluations (hard, but impossible for none; compare students to own past performance)

What tools do we have? 1. Selected-response-format (multiple choice, true-false) tests and quizzes 2. Written responses to academic prompts (short-answer format) 3. Extended written products (essays, papers, lab reports) 4. Visual products (PowerPoint, mural) 5. Oral performances (oral report, world language dialogue, debate)

What tools do we have? 6. Student demonstrations (athletic skills, music performance, role plays) 7. Long-term authentic assessment projects (exhibit) 8. Portfolios 9. Reflective journals or learning logs 10. Informal observations of students

What tools do we have? 11. Formal observations of students using observable indicators 12. Student self-assessment 13. Peer reviews and peer response groups 14. Questions in class-- learning probes

Assessment Resources: Understanding by Design (UbD) ► Encouraging Self-Evaluation: p. 223 ► Questioning for Understanding: p. 156 ► Performance Task Scenarios: p. 172 ► Possible Products and Performances: p. 174 ► Generic Rubric for Understanding: p. 193

Assessment Resources ► Rubrics for Generic Information, Generic Procedures, Problem Solving, Decision Making: pp , 215, 226, Handbook for Classroom Instruction ► Model for Decision Making: p. 221, Handbook for Classroom Instruction

Assessment of Learning Start with what you want students to know and to be able to do at the end of the lesson. Start with what you want students to know and to be able to do at the end of the lesson. These are the learning or instructional objectives, and the assessment needs to be directly linked to them.

Knowledge and Skills ► What we want students to know: Vocabulary, Terminology, Definitions Key factual information Critical details SequencesConcepts

Knowledge and Skills ► What do we want students to be able to do: Basic skills – math, reading Communication skills – listening, speaking, writing Thinking skills – compare, infer, analyze, interpret Research, investigation skills Study skills Interpersonal and group skills

Assessment of Learning Caveat ► Assessment and evaluation of learning is a snapshot of what is learned (except alphabet and math facts) – there is no way to fully represent or measure all the learning that has occurred. SO, of all the things learned, what do we want to demonstrate to selves, students, parents was the outcome?

Assessment of Learning: Prior to Implementation Need to start with identification of what students already know. What are their existing understandings, skills, interests? Need to start with identification of what students already know. What are their existing understandings, skills, interests? Can ask, “Why do you think this is important to know?” “Why do you think we are studying this?” “What do you already know about this?”

Diagnostic Assessment Diagnostic assessment precedes instruction, and identifies: ► students’ prior knowledge ► misconceptions ► interests ► learning-style preferences Examples: pre-tests, student surveys, skills check

Formative Assessment Formative Assessment is ongoing, informal checks for understanding during the curriculum implementation, and provides: ► Information to guide teaching ► Feedback for pacing ► Information about how to improve student performance Resource: p. 234 Understanding by Design

Summative Assessment Summative assessment follows instruction, and identifies: ► Students’ level of mastery ► Student proficiency Examples: post-tests, performance task, culminating project, portfolio

Effective Assessment Effective Assessment ► Performance goals and standards are clear ► Diagnostic assessments check for prior knowledge, skill level, and misconceptions ► Students demonstrate their understanding through real-world applications ► Assessment methods are matched to achievement targets

Effective Assessment Effective Assessment ► Assessment is ongoing and timely ► Learners have opportunities for trial and error, reflection and revision ► Self-assessment is expected

Returning to Clarifying Curriculum Content Priorities ► Worth being familiar with  Assess using traditional quizzes and tests ► Important to know and do  Assess using both traditional methods and performance tasks and projects ► Big Ideas and Enduring Understandings  Assess using performance tasks and projects  Complex, open-ended, authentic assessment

Simple Pre-test/ Post-test Evaluation

Authentic Assessment ► Authentic or Performance Assessment: Students are asked to demonstrate that they can do something real with the information and skills they have learned. ► Portfolios ► Write letters to the editor or a school newspaper ► Write and illustrate a book for the classroom ► Build scale models ► Perform for an audience ► Perform an experiment

Scoring Rubrics for Performance Assessments ► Specify in advance the type of performance expected ► Need to ensure that the scoring system is clear to students ► See rubric examples from UbD

Research on Authentic Assessment ► Performance assessments are challenging to evaluate objectively ► Evaluation outcomes seem to depend on the type of performance as much as the skills of the learner ► Student evaluation scores have been more related to student aptitude than to what students were taught ► Scoring rubrics can enhance achievement

Assessment Example: Returning to Backwards Design 1. Identify desired results: If we want the learners to understand that: ► Friendship demands honesty ► True friendship is often revealed during hard times ► It is sometimes hard to know who your true friends are

Backwards Design 2. Identify acceptable evidence of knowledge: What must students be able to explain, justify, support, or answer about their work for us to infer genuine understanding? What would enable us to infer students’ understanding of what they have learned?

Assessments Linked to Desired Results Possible assessment of understanding of friendship (elementary level): ► Order a friend from a catalog: what qualities should your friend have? ► Dear Abby: Give advice to someone who lied to a friend. ► Develop a brochure for younger students to help them know who their true friends are. ► Explain who your friends are and why they are your friends.

Assessments Linked to Desired Results Possible assessment of understanding of friendship (elementary level): ► Describe the qualities of a true friend. Justify the qualities you selected. ► Create a comic strip or book to illustrate friendship qualities and friendly behaviors. ► Tell or draw a story showing what happens when two friends don’t see eye-to-eye. ► Respond to quotes about friendship (e.g. “A friend in need is a friend indeed”)

Small Group Activity ► Using the pedagogical cycle already developed for SSS, identify several ways you could assess student learning. Revise your learning objectives if necessary. ► What authentic assessments could you use? ► What rubric(s) would you use? ► What are 3 short, key pre-test/post-test questions you could ask?

National Center for School Counseling Outcome Research Thank You