Presentation on theme: "Margaret Macleod Keithia Wilson Griffith University"— Presentation transcript:
1 Margaret Macleod Keithia Wilson Griffith University Quality Improvement of First Year Assessment: A process and template for the review & enhancement of assessment design & managementMargaret MacleodKeithia WilsonGriffith University
2 Acknowledgment to Country In the Spirit of Reconciliation Following on from Sorry Day I would like to acknowledge & honour the Traditional Custodians of the land we are meeting on today, Turrbal and the Jagera People, and pay respect to their Elders past & present And acknowledge the contribution of our First Nation Peoples to Higher Education & learningKeithia Wilson GU - May 2012
3 Introductions Margaret Macleod: Currently the Griffith Health Group Assessment Consultant10 years previous experience as Learning Adviser at Griffith University, Logan campusProf Kethia Wilson:Program Director of the FYE, Griffith HealthALTC National Fellow for the FYE ( )
4 Session Overview 1. Mini-input Reflection & group discussion Context for the interventionLiterature - Importance of assessmentAssessment review processAssessment review templateEvaluation & OutcomesReflection & group discussion3. Final summary of themes
5 Griffith University context Large metropolitan university in Brisbane (1 of 7 in S-E Qld, & 1 of 4 in Brisbane)Multi-campus - 5 campuses x 60 k corridorStudent enrolment of 40,00070% of students are first-in-family at uni - FIF correlates with low SES & lower entry scores6th highest low SES student intake in Australia3rd highest Indigenous student intake in Australia25% International student enrolment10% students studying in distance modeKeithia Wilson GU - May 2012
6 Why assessment matters! There is very strong evidence in the literature to suggest –Assessment is a central feature of curriculumAssessment practices have a profound effect on student behavior (Entwistle, 1991)Assessment is believed to have a more powerful influence on student behavior than teaching (Boud, 2012)Assessment has also been identified by leading educators as one of the least sophisticated aspects of learning and teaching in higher education (James, 2010)
7 In Summary“While students can with difficulty escape from the effects of poor teaching, they cannot (by definition if they want to graduate) escape the effects of poor assessment”.(David Boud, 1995, p 35)
8 Effects of Poor Assessment The effects of poor assessment have been well documented in the literature (Rust, 2002) –Student disengagement from learning, andThe development of negative attitudes towards learning
9 Impact of poor assessment on Non-Traditional students Anecdotal & observational data from Professional staff (e.g., Learning Advisors) indicates that poor assessment contributes to students, particularly non- traditional students, experiencea sense of personal inadequacy,disempowerment,perceived inability to negotiate the complexities of university culture, andlower retention rates.
10 First Year Students’ Appraisals of Assessment Wilson & Lizzio (2012) ALTC GrantCommencing students evaluate their assessment tasks in terms of two factors:The motivational value of the task, andIts manageability, or the extent to which the task is scaffolded, with resources provided to enhance engagement and understanding.
11 Key Findings for FY Assessment Students’ perceptions of the motivational value of assessment tasks are key in predicting student engagement,Students’ perceptions of the effectiveness of the design of the assessment task are key in predicting their levels of motivation for the task,Student evaluations of task manageability are key in predicting student’s sense of self-efficacy or confidence in undertaking the assessment task.
12 Dual Assessment Factors guiding the Review Process Thus, the dual factors ofEffective /appropriate task designCombined with processes for managing the assessment experience and scaffolding student understanding of the assessment task are core to a positive, empowering and potentially successful student experience with assessmentReinforced by feedback from Professional staff, in particular, Learning Advisors
13 Common problems students experience with assessment Difficult, ambiguous or complex tasksTasks that lack relevance to learning outcomes or ‘sense of purpose’Misleading or generic marking criteriaInconsistencies in task informationInsufficient resources to help scaffold tasksInappropriately timed tasks
14 Review Process Overview Academic-professional staff partnershipWorked together to design and implement a process to review and enhance first year assessment, and work effectively with academic staffMacro (FY Program) and micro (Individual tasks) level review and enhancement of first year assessmentUndertaken in two Schools in Griffith Health in 2010Strategy:Focus on student voice and understanding of student experience of assessmentAcademic staff engagementDesign of template to evaluate written assessment tasksReport, including recommendations and resource developmentChange process combining FYE Team & Individual Convenor discussions for decision-making at FY Program & Course/Unit levels
15 The Review Process: Design of Templates Macro level TemplateProgrammatic focus:comparison of all assessment tasks re timing, type, variety, word length, weightingMicro Level TemplateIndividual assessment task focus:Assessment designScaffolding of tasks and targeted ResourcesFinal report: Overall summary and detailed evaluation of written assessment in all first semester, first year courses in each SchoolEmphasise the importance of this being an enabling as well as evaluative process – focus on enhancement, with clear suggestion for how tasks could be strengthened – esp. through resource enhancement
16 2. Clarity: Is assignment task clear and unambiguous? Assessment Design1. Relationship to Course Learning Outcomes: Does the assignment task clearly relate to/develop the learning outcomes for the course?2. Clarity: Is assignment task clear and unambiguous?3. Criteria: Do marking criteria assist in understanding the assignment task?4. Level of difficulty: Is assignment task suitable for students’ level of skill development?5. Weighting: Is weighting appropriate for assignment task?6. Size of task: Is word length appropriate for assignment task?Assessment ResourcesAre there sufficient resources provided to assist students with this task?Suggestions for Scaffolding & Enhancement of student learningAssessment redesignResource enhancementMargaret Macleod, GU, 2011
17 The Review Process: Academic staff engagement Ensuring academic support:Creation of FY leadership team, comprising Dean L&T (Health), HOS/DHOS, Degree Program Convenor/s, First Year Advisors, & FY Course/ Unit ConvenorsCreation of FY Enhancement/Management team, comprising FY Leadership Team + FY Course/Unit ConvenorsFacilitating staff buy-in & engagement:Consultation process with FY staff prior to review to seek commitment & agreement (or otherwise) to the audit process3. Conducting the Assessment Review:Accessing Course/unit outlines for analysisPreparation of Final ReportFacilitating the feedback process:Macro level: programmatic discussions with FY leadership teams to discuss report, share information & make FY Program decisionsMicro level: Learning Adviser met with individual course convenors to assist in redesign of assessment & creation of resources to scaffold student engagement & learning
18 Assessment Review Outcomes: Macro level audits Program Level changesRelocation of some more difficult courses & tasks to 2nd year (e.g., research methods)Reduction of high stakes, highly weighted (e.g., 40 to 50%) first assessment tasks in favour of early, formative low stakes assessment tasks in most coursesAgreement in FYE Teams on total word lengths for semester 1 tasks (from 5-6,000 to 4,000 words);Reduced standardised weightings for word length (e.g., 1,500 word range from 15 to 45%);Reduction in number of group work tasks for assessment purposes & later submission dates (not before week 6)Reduction in number of referencing styles (from 2/3 to 1)Coordination of submission datesBlackboard Audit resulting in consistent information storage
19 Assessment Review Outcomes: Micro level audits Individual assessment task changes:Total redesign of some assessment tasksPartial redesign of many tasks re task clarity, level of difficulty, variety, relation to course aims, marking criteriaDevelopment of task specific resources to scaffold student learning
20 Assessment Review Evaluation Staff and student outcomes: evaluated using data from commencing students Survey – week 7/semester 1) and feedback from teaching staff (end sem 1)Student feedback: Increases in 2011 from 2010 instudent engagement between 5 & 7%student satisfaction between 6 & 8%student capability up by 10%perceptions of good teaching between 11 & 14%These were the highest in the universityAcademic staff feedback:Substantial reductions in student enquiries, student anxiety, & failure rates in threshold courses/unitsPositive perceptions of process – requests that review process be continued into 2nd and 3rd years.survey: weeks 4-7, all commencing students , 40% responbse rate evaluates experinces of orientation and early learning environment, compares this to previous years. e Cond