Assessing Your Assessments: The Authentic Assessment Challenge Dr. Diane King Director, Curriculum Development School of Computer & Engineering Technologies March 13, 2009

SESSION OBJECTIVES How to identify and define learning outcomes How to align assessment with learning outcomes How to assess learning outcomes

What is assessment? Why do we assess? What do we assess? Is it working?

What is Assessment? Assessment is the systematic gathering and analyzing of information (excluding course grades) to inform and improve student learning or programs of student learning in light of goal-oriented expectations Effective Grading: A Tool for Learning and Assessment Barbara Walvoord and Virginia Johnson Anderson Jossey-Bass, 1998

What is Assessment? “Techniques used to analyze student accomplishment against specific goals and criteria” UW-Madison Assessment Manual

Authentic Assessment …simulate or replicate important real- world challenges (Wiggins and McTighe) …measures a student’s ability to perform a real world task (Northern Illinois University)

Authentic Assessment A form of assessment in which students are asked to perform real-world tasks that demonstrate meaningful application of essential knowledge and skills. Student performance on a task is typically scored on a rubric to determine how successfully the student has met specific standards. (Mueller)

Why Do We Assess? To know if students can apply what they have learned in authentic situations (Mueller)

Why Do We Assess? To Improve To Inform To Prove UW-Madison Assessment Manual: Using Assessment for Academic Program Improvement April 2000 Program Improvement Revised April 2000

What Do We Assess? How do we know what to assess?

Where To Start? At the END!!! …with the OUTCOME!!!!

Outcome Specific accomplishments to be achieved (Hatfield)

LEARNING OUTCOME What do I want my students to KNOW as a result of taking this course? = KNOWLEDGE What do I want my students to be able to DO as a result of taking this course? = SKILLS

Learning Outcome HOW are my students going to APPLY what they have learned in here, out there? = PERFORMANCE

Learning Outcomes “As a result of this program/course, students will be able to:….” > College wide > Discipline specific > Program level > Course level

Institutional Outcomes

Backward Design 1. Identify desired results 2. Determine acceptable evidence 3. Plan learning experiences and instruction Wiggins and McTighe

Desired Result Participants will identify one key learning outcome for a course they teach Participants will design one authentic assessment activity to measure

Performance Task Real world application of knowledge or skill

Measurable Performance What criteria will you use to measure how the student performed the task?

Rubric A scoring scale used to assess student performance along a task-specific set of criteria – Mueller

Rubric

Types of Assessment Methods > Portfolio - Digital/electronic/web- based > Special projects/capstones > Journals/learning logs/digital learning records > Conferences/interviews > Oral examinations > Self-/peer assessments > Collaborative project > Performances > Experiments/research studies/visual representations > Case studies > Service learning > Internships – logs/journals/reflections

Types of Assessment Methods > Anecdotal observations > Student generated items > Industry certifications that show competencies > Conferences/interviews > Education plan > Faculty critiques > Documentation of service learning experiences

Direct Assessment > Comprehensive Exam > Major Project > Student Portfolios > Pre-test/Post-test > Embedded questions > Performance assessment > Senior Portfolio > Embedded questions in exams > Senior seminar

INDIRECT ASSESSMENT > Internship Evaluation > Alumni, Employer, Graduate Exit survey > Student scholarly achievement > Examination of information contained in department's own database > Student Satisfaction Survey > Student Course Evaluation > Community perception of program > Student graduation/retention rate > Focus group discussions

Classroom Assessment Techniques Informal/Immediate Feedback The One-Minute Paper The minute paper is a short exercise in which you ask students to write for one minute on two questions: What was the most important thing you learned today? and, what question still remains in your mind after today's class? The Muddiest Point This assessment method is similar to the minute paper. Students write a one-minute essay on the muddiest point that remains in their minds after a lecture, demonstration, or presentation. The One-Sentence Summary In this method, students write and then discuss a one-sentence summary that describes the content covered in class. Directed Paraphrasing In directed paraphrasing, students summarize a concept or procedure in two or three sentences. Applications Cards Here, the instructor asks students to think of real-world applications of topics discussed in class.

IS IT WORKING?

ASSESSING THE ASSESSMENT

ASSESSMENT VALIDATION 1. Are the following elements present in the instrument? > Learning Outcome > Product or performance-based assessment > Real-world relevance > Application of knowledge > Alignment with criteria on rubric

ASSESSMENT VALIDATION 2. Will the assessment task elicit the learning outcome(s) being assessed? 3. Will the assessment task elicit a full expression of ability at a level appropriate to the students’ general education learning experience? 4. Does the assessment task require students to demonstrate proficiency of the learning outcome (understanding and ability)?

ASSESSMENT VALIDATION 5. Does the assessment task integrate knowledge and skills gained throughout the students’ general education learning experience? 6. Does the assessment task permit students some individual difference in meeting the performance criteria? 7. On a scale (from disconnected to fully integrated), does the assessment task encourage students to integrate competencies with each other?

ASSESSMENT VALIDATION 8. Does the assessment task assess both knowledge and ability? 9. Is the assessment task authentic; that is, does it involve students in issues they see as vital concerns or engage them with problems related to the real world? 10. Will the assessment task produce results that can provide diagnostic, structured feedback on students’ attainment of the targeted learning outcome?

Workshop Applied Learning Tasks Introduction to Computers/Computer Literacy 1. Identify One Learning Outcome 2. Develop one authentic assessment task to measure that outcome

THANK YOU!