1. Margaret Macleod Keithia Wilson Griffith University
Quality Improvement of First Year Assessment: A process and template for the review & enhancement of assessment design & management
Margaret Macleod
Keithia Wilson
Griffith University

2. Acknowledgment to Country
In the Spirit of Reconciliation Following on from Sorry Day I would like to acknowledge & honour the Traditional Custodians of the land we are meeting on today, Turrbal and the Jagera People, and pay respect to their Elders past & present And acknowledge the contribution of our First Nation Peoples to Higher Education & learning
Keithia Wilson GU - May 2012

3. Introductions Margaret Macleod:
Currently the Griffith Health Group Assessment Consultant
10 years previous experience as Learning Adviser at Griffith University, Logan campus
Prof Kethia Wilson:
Program Director of the FYE, Griffith Health
ALTC National Fellow for the FYE ( )

4. Session Overview 1. Mini-input Reflection & group discussion
Context for the intervention
Literature - Importance of assessment
Assessment review process
Assessment review template
Evaluation & Outcomes
Reflection & group discussion
3. Final summary of themes

5. Griffith University context
Large metropolitan university in Brisbane (1 of 7 in S-E Qld, & 1 of 4 in Brisbane)
Multi-campus - 5 campuses x 60 k corridor
Student enrolment of 40,000
70% of students are first-in-family at uni - FIF correlates with low SES & lower entry scores
6th highest low SES student intake in Australia
3rd highest Indigenous student intake in Australia
25% International student enrolment
10% students studying in distance mode
Keithia Wilson GU - May 2012

6. Why assessment matters!
There is very strong evidence in the literature to suggest –
Assessment is a central feature of curriculum
Assessment practices have a profound effect on student behavior (Entwistle, 1991)
Assessment is believed to have a more powerful influence on student behavior than teaching (Boud, 2012)
Assessment has also been identified by leading educators as one of the least sophisticated aspects of learning and teaching in higher education (James, 2010)

7. In Summary
“While students can with difficulty escape from the effects of poor teaching, they cannot (by definition if they want to graduate) escape the effects of poor assessment”.
(David Boud, 1995, p 35)

8. Effects of Poor Assessment
The effects of poor assessment have been well documented in the literature (Rust, 2002) –
Student disengagement from learning, and
The development of negative attitudes towards learning

9. Impact of poor assessment on Non-Traditional students
Anecdotal & observational data from Professional staff (e.g., Learning Advisors) indicates that poor assessment contributes to students, particularly non- traditional students, experience
a sense of personal inadequacy,
disempowerment,
perceived inability to negotiate the complexities of university culture, and
lower retention rates.

10. First Year Students’ Appraisals of Assessment
Wilson & Lizzio (2012) ALTC Grant
Commencing students evaluate their assessment tasks in terms of two factors:
The motivational value of the task, and
Its manageability, or the extent to which the task is scaffolded, with resources provided to enhance engagement and understanding.

11. Key Findings for FY Assessment
Students’ perceptions of the motivational value of assessment tasks are key in predicting student engagement,
Students’ perceptions of the effectiveness of the design of the assessment task are key in predicting their levels of motivation for the task,
Student evaluations of task manageability are key in predicting student’s sense of self-efficacy or confidence in undertaking the assessment task.

12. Dual Assessment Factors guiding the Review Process
Thus, the dual factors of
Effective /appropriate task design
Combined with processes for managing the assessment experience and scaffolding student understanding of the assessment task are core to a positive, empowering and potentially successful student experience with assessment
Reinforced by feedback from Professional staff, in particular, Learning Advisors

13. Common problems students experience with assessment
Difficult, ambiguous or complex tasks
Tasks that lack relevance to learning outcomes or ‘sense of purpose’
Misleading or generic marking criteria
Inconsistencies in task information
Insufficient resources to help scaffold tasks
Inappropriately timed tasks

14. Review Process Overview
Academic-professional staff partnership
Worked together to design and implement a process to review and enhance first year assessment, and work effectively with academic staff
Macro (FY Program) and micro (Individual tasks) level review and enhancement of first year assessment
Undertaken in two Schools in Griffith Health in 2010
Strategy:
Focus on student voice and understanding of student experience of assessment
Academic staff engagement
Design of template to evaluate written assessment tasks
Report, including recommendations and resource development
Change process combining FYE Team & Individual Convenor discussions for decision-making at FY Program & Course/Unit levels

15. The Review Process: Design of Templates
Macro level Template
Programmatic focus:
comparison of all assessment tasks re timing, type, variety, word length, weighting
Micro Level Template
Individual assessment task focus:
Assessment design
Scaffolding of tasks and targeted Resources
Final report: Overall summary and detailed evaluation of written assessment in all first semester, first year courses in each School
Emphasise the importance of this being an enabling as well as evaluative process – focus on enhancement, with clear suggestion for how tasks could be strengthened – esp. through resource enhancement

16. 2. Clarity: Is assignment task clear and unambiguous?
Assessment Design
1. Relationship to Course Learning Outcomes: Does the assignment task clearly relate to/develop the learning outcomes for the course?
2. Clarity: Is assignment task clear and unambiguous?
3. Criteria: Do marking criteria assist in understanding the assignment task?
4. Level of difficulty: Is assignment task suitable for students’ level of skill development?
5. Weighting: Is weighting appropriate for assignment task?
6. Size of task: Is word length appropriate for assignment task?
Assessment Resources
Are there sufficient resources provided to assist students with this task?
Suggestions for Scaffolding & Enhancement of student learning
Assessment redesign
Resource enhancement
Margaret Macleod, GU, 2011

17. The Review Process: Academic staff engagement
Ensuring academic support:
Creation of FY leadership team, comprising Dean L&T (Health), HOS/DHOS, Degree Program Convenor/s, First Year Advisors, & FY Course/ Unit Convenors
Creation of FY Enhancement/Management team, comprising FY Leadership Team + FY Course/Unit Convenors
Facilitating staff buy-in & engagement:
Consultation process with FY staff prior to review to seek commitment & agreement (or otherwise) to the audit process
3. Conducting the Assessment Review:
Accessing Course/unit outlines for analysis
Preparation of Final Report
Facilitating the feedback process:
Macro level: programmatic discussions with FY leadership teams to discuss report, share information & make FY Program decisions
Micro level: Learning Adviser met with individual course convenors to assist in redesign of assessment & creation of resources to scaffold student engagement & learning

18. Assessment Review Outcomes: Macro level audits
Program Level changes
Relocation of some more difficult courses & tasks to 2nd year (e.g., research methods)
Reduction of high stakes, highly weighted (e.g., 40 to 50%) first assessment tasks in favour of early, formative low stakes assessment tasks in most courses
Agreement in FYE Teams on total word lengths for semester 1 tasks (from 5-6,000 to 4,000 words);
Reduced standardised weightings for word length (e.g., 1,500 word range from 15 to 45%);
Reduction in number of group work tasks for assessment purposes & later submission dates (not before week 6)
Reduction in number of referencing styles (from 2/3 to 1)
Coordination of submission dates
Blackboard Audit resulting in consistent information storage

19. Assessment Review Outcomes: Micro level audits
Individual assessment task changes:
Total redesign of some assessment tasks
Partial redesign of many tasks re task clarity, level of difficulty, variety, relation to course aims, marking criteria
Development of task specific resources to scaffold student learning

20. Assessment Review Evaluation
Staff and student outcomes: evaluated using data from commencing students Survey – week 7/semester 1) and feedback from teaching staff (end sem 1)
Student feedback: Increases in 2011 from 2010 in
student engagement between 5 & 7%
student satisfaction between 6 & 8%
student capability up by 10%
perceptions of good teaching between 11 & 14%
These were the highest in the university
Academic staff feedback:
Substantial reductions in student enquiries, student anxiety, & failure rates in threshold courses/units
Positive perceptions of process – requests that review process be continued into 2nd and 3rd years.
survey: weeks 4-7, all commencing students , 40% responbse rate evaluates experinces of orientation and early learning environment, compares this to previous years. e Cond