Designing Assessment: How much is enough?

Assessment practice is surrounded by uncertainty (Allen 1998) Academics design tasks, award grades and provide feedback that,  They feel comfortable with,  They believe the student will feel is fair,  Can with stand robust scrutiny by colleagues. They make decisions that lead to preferred and expected outcomes using their experience to predict the best and worst outcomes!

“Good practice” in assessment underpins fairness Clear descriptions of grading criteria & grade related performance standards Timely feedback that linked to the assessment criteria Appropriate tasks that encourage learning Decision-making guided by posted criteria and standards when grading (adapted from Carroll 2004)

Scholarship on Learning & Assessment  Students require risk and ambiguity to learn  Students require challenge to engage in their learning  Students need support in their learning  Students who are meta-cognitive are more likely to succeed  Students learn for the longer term when they have to DO SOMETHING with the knowledge and concepts they learn  Much assessment is taken up with surface learning and trying to ‘spin out a grade’ for the institution.  Ultimately assessment is a fallible and human activity.

Meta-cognitive is knowing what you know and don’t know and how you learn and how to go about finding out what you don’t know. is awareness of your own approach to learning and the current status of your learning.

Assessment Design Informs & Shapes Learning  Tells students what you, the teacher/assessor, think is important  Relevance  Relatedness Dictates students’ learning activities  Formative (Assessment as Learning) Defines students to themselves as learners  Effort  Ability  Persistence

What Informs Assessment Design?  Traditions (essays and exams)  Conventions & common practice (Britzman 2003)  Wisdom of experience (Shulman 1979 & beyond)  Contemporary trends (e.g. portfolio assessment)  Expedience and practicalities  Scholarship?

About assessment design!  It is the least understood aspect of curriculum and teaching.  It has educational, technical and ethical dimensions that encompass: 1. Purposes: Why assess? 2. Goals: What to assess? 3. ProcessesHow to assess? 4. InterpretationHow to grade? 5. ResponseHow to feedback & report? (Rowntree, 1977)

Focus : Assessment Design 1. Purposes: Why assess? 2. Goals: What to assess? 3. ProcessesHow to assess? 4. InterpretationHow to grade? 5. ResponseHow to feedback & report? Impacts on, should consider, and be influenced by the four other dimensions.

Key Issues in designing assessment?  What is the most important for learning goals for students in this unit?  How to engage students from the start?  What will be appropriate for the unit level?  How to build on, & develop existing capabilities?  How to encourage Higher Order Thinking (HOT)?  How to support students with special needs?  How to provide feedback opportunities?  How to design tasks that will generate evidence of students learning achievements?

Important for learning goals Assessment designs have 3 functions:  Define for students what is important to learn  Shape students approach to learning  Provide a means of assurance of learning for the university and society. Clear learning goals Tasks that align with them

What attributes would you like to engender in your graduates?  Knowledgeable disciplinary specialists?  Ethical, efficient & trust worthy professionals?  Collaborative, reliable colleagues and team players?  Community & corporate citizens?  Effective problem solvers?  Creative & critical reasoning thinkers?  Effective verbal & written communicators?  Culturally literate?  Technologically literate?

Important for learning goals Identify a Unit you will teach next session. Unit……….. Level:………… Learning goals:

Engaging students from the start What capabilities do you assume students will bring to the Unit?  Terminology  Conceptual understanding  Capabilities  Dispositions How will you confirm this?  Pre-tests/Online tests/ reflective logs/  Hurdle assessment, work required What will signal the learning priorities and approaches to students?

Engaging students from the start

Appropriate for the unit level Assessment design should alter in terms of complexity across the years. First year:  Acquire discipline language, terminology  Develop research analysis, communication & psychomotor skills  Identify relevance to the big picture  How experts proceed & facts become facts Later years  Less tasks, less testing, greater complexity.  Application of the information to novel contexts  Increased research led learning  Learn the values of the discipline

Appropriate for the unit level

Build on existing capabilities Often units are taught and assessed in isolation which increases the burden on each unit to ‘cover’ and ‘assess’. Assessment designs need to:  account for their own contribution to the overall degree program  capitalize on and develop what has gone before and is currently being taught Programmatic approaches to assessment assist students to:  synthesize, effectively utilize multi-source feedback  become more metacogntively aware  become responsible for their own learning

Build on existing capabilities What are the concurrent and previous units that can contribute to student performance in this unit? What would be the essential learning that this unit contribute to a programmatic approach?

Develop Higher Order Thinking The essence of a university education is higher order thinking.

Blooms Taxonomy CreatingEvaluatingAnalyzingApplying Comprehension Understanding Knowledge Remembering Higher Order thinking

Biggs (1992) Pre-structural Uni-structural Multi-structural Relational Extended Abstract

Biggs (1992) Pre-structural Uni-structural Multi-structural Relational Extended Abstract

Biggs (1992) Pre-structural Uni-structural Multi-structural Relational Extended Abstract

Scholarship to Inform Standards Reasoning Perry’ Ethico Moral Reasoning in College Students Black & white thinking Multiplicity + search for truth. Relativism but no commitment Commitment Conditional Commitment

Develop Higher Order Thinking Consider a good performance by a student in your unit.  What was the student able to demonstrate that lead you to assess it as higher order performance?  How might the assessment tasks encourage that kind of performance?

Supporting students with special needs Students have diverse needs across their course of study. It is important to anticipate challenges for students It is important that academics don’t take on issues as their sole responsibility.

Supporting students with special needs What flexibility is in your unit to accommodate unpredictable circumstances & needs? What resources can academics call upon at UWA?

Provide feedback opportunities  Students NEVER feel they get enough feedback  Students rarely go back and respond to the feedback with which they they have been provided  Student ought to be active participants in the feedback process  Self and peer feedback processes increase student metacognition  Feedback and the response to feedback needs to be designed into the tasks with successful performance and grades contingent on students engagement with feedback.

Provide feedback opportunities  How might students be engaged in the feedback process in your unit?

Generating evidence of learning achievement The most valuable assets students can graduate with are:  a sound understanding of what they have learned and achieved in the degree program  validated evidence to support their new capabilities  E.g. Portfolios, projects, research

Evidence of achievement What processes will assessment in this unit assist students to gain an accurate awareness of their own emerging capabilities? What tasks can provide students with validated evidence of achievement and capability?

What designs best produce desired learning and graduate capabilities? Authentic Tasks: Assessment as Learning  E.g.. Library search (critical selection & team activity)  Forensic role play (investigation/problem solving)  Observation (noticing)  Case presentations (theory/practice application)  Debates and position statement (evidence-based, opinion formation)  Mastery learning (Skill development)

How much assessment is enough?  Enough to engage them from the start and keep them engaged for 12 weeks.  Enough to extend their existing abilities and develop higher order thinking.  Nothing that is there because its university and we always do it at university.  Enough to produce sound evidence of achievement.

Some Design examples  First year technical (accounting, economics)  Regular (weekly/) work assessed in class, presented in class, collected, worked examples and a sample of 3-4 assessed at the end of semester.  Plus an end of semester exam.  First year, expressive areas  Comparison of popular media treatment(web?) of a topic with academic literature, (Individual or group)  Presentation of findings  Website design to illustrate findings  Final year:  Proposal, plan or contract of work to be don  Work in progress self peer and expert formative evaluation  Completed project e