Prof Keithia Wilson Program Lead for the FYE, Griffith Health GIHE Senior Fellow for the FYE 2010 ALTC National Fellow for the FYE 2007 Australian University Teacher of the Year Prof Keithia Wilson ALTC National Fellow

Acknowledgment to Country In the Spirit of Reconciliation Following on from Sorry Day I would like to acknowledge & honour the Traditional Custodians of this land that our Logan campus is built on, the Yugambeh People, and pay respect to their Elders past & present

Overview 1. Principles & practices for enabling commencing students success with FY Assessment Designing assessment Preparing students to undertake assessment Marking Assessment Post-Assessment feedback process to enable & support learning 2. Strategies for academic recovery with at-risk commencing students Prof Keithia Wilson ALTC National Fellow

Overview 1. Current Context for the FYE 2. Purposes of FY Assessment & Student Motivations 3. The FY Assessment Lifecycle from the perspectives of – Students and Staff 2. Principles & Models for enhancing effective Practice with FY Assessment Prof Keithia Wilson ALTC National Fellow

Information Sources The First Year Experience & Assessment literature generally Findings from a recent ALTC Grant on First Year Assessment Practice ( ) Findings from a current ALTC National Fellowship on the FYE working with FY teaching teams ( ) Focus on the student voice & understanding the student experience of assessment Prof Keithia Wilson ALTC National Fellow

Step 1 – Context How can we improve the assessment process and experience for commencing students? Understanding the current context for the FYE Prof Keithia Wilson ALTC National Fellow

Federal Gov’t reform agenda in Higher Education Aims to – Widen student participation in Higher Education – A FAIR GO! Increase the access of students from low SES & disadvantaged backgrounds to university (higher numbers - Target increase from 12% to 20% by 2020) Increase the success of students from low SES & disadvantaged backgrounds at university (higher retention) Moving from an elite model of HE (0-15% population participation) to a mass model of HE (16-50%) (Trow, 2004) 25% participation in Australia (50% USA) 1.3% of Indigenous Australians attend university (30% lower graduation rate) Prof Keithia Wilson ALTC National Fellow

Evolution of Approaches to FY Orientation & Engagement & the FYE First Generation Strategies = Co-Curricular - A focus on designing FYO&E supplemental activities & strategies which are outside of the classroom Second Generation Strategies = Curricular - A focus on enhancing FY curriculum design, pedagogy & assessment practices Third Generation Strategies = Whole-of-Institution - A focus on an Institution wide approach to 1 st & 2 nd generation strategies, with practice standardisation & QA mechanisms for continuous improvement = Whole-of-School/Program - A focus on the strategic combination of 1 st & 2 nd generation strategies for a particular disciplinary context (School or Program) Prof Keithia Wilson ALTC National Fellow

Understanding the Current Context for the FYE Meta Model 1 - FY Transition Practice Student Diversity Student Transition * Course Design * Course Delivery * Course Assessment Prof Keithia Wilson ALTC National Fellow

Student diversity So what is important to understand about student diversity? Defining student diversity Assumed knowledge Prof Keithia Wilson ALTC National Fellow

What is Student Diversity? Traditional Students (TS) medium-high SES second generation higher entry levels full time on-campus Elite Model Non-Traditional Students (NTS) low SES first-in-family lower entry levels full-time & working not on-campus much Indigenous NESB International, refugees disability home care responsibilities from rural & remote settings Mass Model Prof Keithia Wilson ALTC National Fellow

What is assumed knowledge & capabilities? Academic Skills Information Literacy Computer Literacy Reading Skills Written Communication Numeracy Skills Critical thinking & analysis Independent Learning (self-regulation) viz. time on task, self-study, time management, uni-work-social life balance, successful student behaviour Prof Keithia Wilson ALTC National Fellow

What is assumed knowledge & capabilities? Academic/Cultural/Social Capital – the “Hidden Curriculum” 1. Role Understanding student role expectations & appropriate & effective behaviour (rights & obligations) 2. Success Reading the academic context to accurately determine performance requirements re studying & assessment 3. Support Capacity for help-seeking without fear of negative labelling (dumb/stupid) 4. Personal Identity Sense of belonging & personal fit with university (overcome the “outsider within” phenomenon – “A stranger in a foreign land”) Prof Keithia Wilson ALTC National Fellow

Griffith University context & student profile Student enrolment of 40,000 Student enrolment of 40,000 70% of students are first-in-family at uni (NT) - FIF correlates with low SES & lower entry scores/Ops 70% of students are first-in-family at uni (NT) - FIF correlates with low SES & lower entry scores/Ops 6 th largest low SES student intake in Australia (16% overall, 30% Logan campus) 6 th largest low SES student intake in Australia (16% overall, 30% Logan campus) 3 rd largest Indigenous student intake in Australia (600) 3 rd largest Indigenous student intake in Australia (600) 30% International student enrolment (N’thn Europe, China, India, Indonesia, Middle East, Canada) 30% International student enrolment (N’thn Europe, China, India, Indonesia, Middle East, Canada) 10% students studying in distance mode 10% students studying in distance mode More of our students work in paid employment & longer hours in employment than the national average (NTS reality) More of our students work in paid employment & longer hours in employment than the national average (NTS reality) Prof Keithia Wilson ALTC National Fellow

Are NTS capable of being successful at university? The national research evidence shows - The success rate (or tendency to pass their year’s subjects) of low SES students is 97% of the pass rates of their medium & high SES peers & has been stable over the last 5 years (Bradley et al, 2008:30) However, they require higher levels of support to succeed e.g., financial assistance, academic support, mentoring & counselling services (Commonwealth of Australia, 2009) Prof Keithia Wilson ALTC National Fellow

Therefore....implications for assessment practice Taking account of assumed entry level knowledge in unit content & assessment design & making this explicit to students Building foundational academic skill development into unit content & assessment design Developing commencing students skills as independent, self-regulating learners across the first year Creating a unit & program level culture that respects & values diversity in all of its forms (age, gender, race etc.) Prof Keithia Wilson ALTC National Fellow

Student Transition So what is important to understand about Student Transition? An evidence-based model for conceptualising student transition Senses of success (Lizzio, 2006) Predictors of student success Prof Keithia Wilson ALTC National Fellow

Meta-model 2: The ‘Five-Senses’ of Student Success Sense of Student Identity Sense of Connectedness Sense of Capability Sense of Purpose Sense of Resourcefulness Prof Keithia Wilson ALTC National Fellow (Lizzio, 2006)

What predicts commencing students’ satisfaction with their degree program? Sense of Purpose Sense of Capability Good Teaching Sense of Connection Perceived Effectiveness of Orientation Strongly Enhances Enhances Time on Task Enhances Commencing Student Satisfaction Prof Keithia Wilson ALTC National Fellow

What predicts commencing students’ academic outcomes? Semester 1 GPA Academic Capital Low SES First in Family ESL Competing Demands Time in employment Time as carer Prior Academic Achievement Entry Scores (OP) Task Uni Attendance at Orientation Time on task Strongly Enhances Enhances Reduces Prof Keithia Wilson ALTC National Fellow

What predicts commencing students’ retention? Semester 1 GPA Academic Capital - Competing Demands - Prior Academic Achievement + Task Uni + Sense of Purpose Student Satisfaction + Student Retention into yr 2 Prof Keithia Wilson ALTC National Fellow

Therefore....Implications for practice Curriculum strategies for developing time on task & self-regulation skills  assists assessment success Strategies for enhancing sense of purpose in curriculum & assessment practices Building academic & social capital through unit & assessment design throughout the FY Effective Orientation Programs & early student engagement strategies to encourage attendance at Orientation Prof Keithia Wilson ALTC National Fellow

Step 2 – Assessment Principles & Practices So...what do students say about their experience of assessment? A quick overview of some salient research evidence! Prof Keithia Wilson ALTC National Fellow

Assessment characteristics which positively influence student learning & engagement 1. perceived validity of assessment tasks & approaches 2. perceived extent to which the learning environment is empowering 3. perceived fairness of the learning environment, especially with assessment tasks & practices Prof Keithia Wilson ALTC National Fellow

Students’ perceptions of the validity of assessment tasks & approaches (Sambell et al., 1997) 3 sets of priorities - 1. Educational values – authentic/meaningful tasks, perceived to have long term benefits, applying knowledge 2. Educational processes – reasonable demands, encourages independence by making expectations clear 3. Educational consequences – rewards effort and breadth & depth in learning Prof Keithia Wilson ALTC National Fellow

Students’ meta themes in assessment (Savin-Baden, 2004) 2 forms of student disempowerment: 1. Unrewarded learning – the relationship between quantity of work & its weighting 2. Disabling assessment mechanisms including: Processes – lack of information & inadequate feedback Forms – assessment methods that do not fit with espoused forms of learning Prof Keithia Wilson ALTC National Fellow

Students’ perceptions of fairness (Lizzio, Wilson & Hadaway, 2008) Strongly influenced by – Extent to which they feel personally respected by academic staff (convenors & sessional staff) in the learning & assessment process – relational culture Adequacy of the information & support systems provided for them to “do their job” in relation to assessment Prof Keithia Wilson ALTC National Fellow

The Reality of Assessment for Commencing Students Student feedback from multiple sources indicates: Strategic nature of Assessment for student engagement, success & retention For many students assessment IS the learning Assessment items which are too difficult, not meaningful, and not clearly explained are a key cause of student drop-out in the first semester Well designed & implemented assessment is key to student engagement, learning & retention Prof Keithia Wilson ALTC National Fellow

Assessment Practice in the First Year So what is important to our understanding? A model of purposes of FY assessment & student motivation with assessment A model for management of the FY Assessment Lifecycle informed by : Student perspective Staff perspective  Principles of Good Practice Prof Keithia Wilson ALTC National Fellow

Purposes of FY Assessment Diagnostic  Transition enabling  Motivating  Formative  Learning  Summative/evaluative  Terminal  Meta-reflective  Assessment of student readiness Assessment to aid transition to & engagement with uni Assessment as stimulation for learning Assessment for learning Assessment as learning Assessment of learning Assessment of capability/mastery Assessment of learning process Prof Keithia Wilson ALTC National Fellow

Student view of ‘motivating assessment’ (Wilson & Lizzio, 2011) Engaging Design Relevance (personal, academic, professional), intellectual challenge, teacher enthusiasm Enabling Management Task clarity, task scaffolding, formative conversations with & support from staff Teacher Authority Clear and firm expectations & standards, consequences explained, responsibility invoked Prof Keithia Wilson ALTC National Fellow

Meta-model 3: The first-year assessment lifecycle – student experience 1. Student’s prior experiences 2. Formal Framing & student appraisal 3. Progressive Enabling & Student Engagement 4. Submission Culture 5. Feedback Mechanisms 7. Feed-forward Mechanisms 6. Academic Outreach and Recovery Prof Keithia Wilson ALTC National Fellow

The first-year assessment lifecycle – staff experience 1. Student’s prior experiences 2. Formal Framing & student appraisal 3. Progressive Enabling & Student Engagement 4. Submission Culture 5. Feedback Mechanisms 7. Feed-forward Mechanisms 6. Academic Outreach and Recovery Design Phase Student Preparation Phase Assessment Phase Post- Assessment Phase Prof Keithia Wilson ALTC National Fellow

Principles of Effective Practice in First Year Assessment Good Practice Principles rely on: 1. A lifecycle approach to assessment (4 Phases) Design phase Student Preparation phase (pre-submission) Assessment phase (marking) Post-Assessment phase (feedback & feed-forward) 2. A system’s approach to assessment (3 Systems levels) Individual Unit/Course/Subject level First Year Program level – horizontal & vertical integration Degree Program level Prof Keithia Wilson ALTC National Fellow

1. Design Phase: Individual Unit/Subject Fit for purpose – Optimising student motivation & engagement by designing assessment tasks which are perceived to be relevant & meaningful to students & their learning (sense of purpose) Level of difficulty & complexity – Designing tasks with a conscious understanding of the assumed entry level knowledge & appropriate learning level of commencing students Progressive knowledge & skill development – Designing assessment tasks across a semester in such a way as to ensure effective, cumulative knowledge & skill development Variety – Ensuring a variety of different types of items both within a unit/subject, and across units /subjects in the same semester (FY program level planning) Prof Keithia Wilson ALTC National Fellow

1. Design Phase: Individual Unit/Subject Early success -Optimising an experience of ‘early success’ for students to build academic & personal efficacy & confidence Formative Assessment - Emphasising early formative assessment designed to develop skill & confidence viz. Early – weeks 4-6 is optimal Smaller pieces for either no marks, or fewer marks (10-20%) to encourage recovery from possible failure (building hope) Speedy, quality feedback (peer or staff, individually or collectively) with feedback by weeks 7-8 Self-assessment of all written items by self-evaluating on the identified criteria (builds meta-competence) Prof Keithia Wilson ALTC National Fellow

1. Design Phase: Individual Unit/ Subject Level Written presentation of assessment tasks which present the task clearly and without ambiguity–  Start with the aim of the task Present the sequence of steps involved in the task, from the simple to the complex, thus breaking down large tasks into manageable chunks Refer students to the marking criteria for the task Ensure that all available information is provided as much as possible in the one place Prof Keithia Wilson ALTC National Fellow

1. Design Phase – Essays – a “special case” There are a range of data sources to indicate that essays in the first year are a source of great angst for both students and staff, and may be more complex than we (staff) think : Professional Staff feedback Academic Staff feedback Student feedback Quality of FY essays, especially in the first semester Quality of student essays in the second & third years & beyond! Wilson & Lizzio – Findings from ALTC Grant Prof Keithia Wilson ALTC National Fellow

Assessment Tasks – exams & laboratory reports Engaging Assessment Design Assessment Management & Support Deep Approach to Learning Assessment Outcomes: Student Grades Assessment Task Efficacy Prof Keithia Wilson ALTC National Fellow

Assessment Tasks– Oral Presentation Engaging Assessment Design Assessment Management & Support Deep Approach to Learning Assessment Outcome - Student Grades Assessment Task Efficacy Prof Keithia Wilson ALTC National Fellow

Assessment Tasks - Essay Engaging Assessment Design Assessment Management & Support Deep Approach to Learning Assessment Outcome - Student Grades Assessment Task Efficacy Prof Keithia Wilson ALTC National Fellow

1. Design Phase – Essays – a “special case”  Semester 1 – essay tasks that are: Relatively short (e.g., words) Relatively simple (e.g., summary/description & simple interpretative analysis such as answers to how or why questions) Include limited research requirements (e.g., simple searches of readily obtainable information)  Semester 2 – essay tasks that are: Longer (1,200-2,500 words) Incorporate more complex analysis (e.g., critical analysis & evaluation tasks – compare & contrast) Include more complex research requirements Prof Keithia Wilson ALTC National Fellow

1. Effective FY Assessment Practice Principles – Design Phase: FY Program Level Workload distribution – Ensuring roughly equal workload between all units in a semester to assist development of time management skills Scheduling – coordination of submission dates to stagger the student workload Threshold/Difficult Units - Identify & front-load any threshold units in a semester to enable student success Variety of assessment types – Ensure variety across units in a semester Group Work – Coordinate any group-based assessment tasks across the FY Program & preferably reduce to 1 unit only if the assessment requires additional meetings outside of class Prof Keithia Wilson ALTC National Fellow

Task Design Provide me with ‘fit for purpose’ tasks and roles Meta-model 4: Progressive Enabling: How might we effectively manage the assessment process? Self-Management Help me by managing yourself Systems Awareness Help me by coordinating the experience Prof Keithia Wilson ALTC National Fellow

Reflective Task: Assessment Design Phase How does this information apply to your FY context? What are the opportunities for enhancing FY assessment design? What are the challenges with enhancing FY assessment design? Prof Keithia Wilson ALTC National Fellow

2. Student Preparation Phase: Individual Unit/Subject Level Motivating students – 3 Domains of Motivation Articulating explicitly the academic relevance of each assessment task (the knowledge and skills that will be useful to students later in their course/degree); Articulating explicitly the personal relevance of each assessment task (the knowledge and skills that will be useful for students in the future); Articulating explicitly the professional relevance of each assessment task (the knowledge and skills that will be useful to students later in their career); Prof Keithia Wilson ALTC National Fellow

2. Student Preparation Phase: Individual Unit/Subject Level Motivating students – 2 Types of Relevance Articulating future positive relevance (personally & professionally) of the knowledge and skills to be gained from an assessment task to the course/degree/future employment; Articulating future negative relevance of the knowledge and skills to be gained from an assessment task to the course/degree/future employment for students who may not engage sufficiently; Prof Keithia Wilson ALTC National Fellow

2. Student Preparation Phase: Individual Unit/Subject Level Motivating students – Challenge Articulating explicitly the intellectual challenge of each assessment task (the challenge offered to students to think and learn); Articulating to students the investment of work required to be successful with the task (encourages development of self-regulatory behaviour); Staff Stance Staff conveying to students their own personal enthusiasm for the task (staff engagement increase student engagement); Being clear and well organised (assists anxiety management & increases performance ability). Prof Keithia Wilson ALTC National Fellow

2. Student Preparation Phase: Individual Unit/Subject Level Management Strategies - Scaffolding learning -Designing a process for scaffolding assessment preparation which prepares students for each assessment task: Providing detailed goals, criteria, standards for each task to clarify what good performance is – viz. clear goals & standards Providing opportunities for students to actively engage with, & potentially modify those goals, criteria, standards Providing high & low quality examples of performance for each assessment task or a similar task Providing targeted resources such as practice items, quizzes, mini- essay writing, step-by-step processes organised sequentially etc. Providing multiple & regular opportunities for discussion of assessment tasks & requirements Ensuring consistent information & resources are provided from all teaching staff (convenors & tutors) on assessment tasks Prof Keithia Wilson ALTC National Fellow

2. Student Preparation Phase: First Year Program Level Clear Goals and Standards – Ensuring all course convenors have detailed criteria for all assessment tasks in their units/courses Responsive culture – Ensuring a responsive FY Program culture in all units/courses, including training of sessional staff  consistent messages about performance & success Consistency of referencing style – Providing a single referencing style only for FY students for their FY of study for all units/courses in their Program Consistency of information storage by unit convenors for web-site information for all units (same folders) Consistent terminology to describe same types of assessment tasks across a program e.g., critical reflection/critical analysis/essay/critique Prof Keithia Wilson ALTC National Fellow

Facilitating Motivation Help me to engage Building Capacity Help me to be task capable Task Design Provide me with ‘fit for purpose’ tasks and roles Progressive Enabling: How might we effectively manage the assessment process? Self- Management Help me by managing yourself Systems Awareness Help me by coordinating expectations Prof Keithia Wilson ALTC National Fellow

Facilitating Motivation Help me to engage Relating Functionally Help me to feel understood Managing Information Help me to understand the task Facilitating Process Help me to solve problems Managing Procedures Help me to navigate the rules of the game Building Capacity Help me to be task capable Task Design Provide me with ‘fit for purpose’ tasks and roles Progressive Enabling: How might we effectively manage the assessment process? Self- Management Help me by managing yourself Systems Awareness Help me by coordinating expectations Prof Keithia Wilson ALTC National Fellow

Reflective Task: Student Preparation Phase How does this information apply to your FY context? What are the opportunities for enhancing FY student preparation with assessment ? What are the challenges with enhancing FY student preparation with assessment? Prof Keithia Wilson ALTC National Fellow

3. Assessment Marking Phase: Individual Unit/Subject Level Transparency & Consistency – Ensuring consistency of marking standards between markers by: Marker preparation - having agreed, transparent, detailed standards for preparing markers to assess Marking trial – initially marking 5 or so & then meeting to compare standards of marking Prof Keithia Wilson ALTC National Fellow

3. Assessment Marking Phase: Individual Unit/Subject Level Marker Moderation - facilitating high quality moderation of final assessment marks (procedural justice) by staff teams viz. * staff team meeting face-to-face to discuss  education & capacity building for staff re marking standards * all staff reading all fails & HDs (depending on numbers) to clarify understanding of high & low end standards * all staff reading & re-assessing all assessment items on the margins of each grading category (high Ps & low Cs etc)  final re-adjustment of marks * keeping copies of high & low end examples to be used anonymously with future cohorts (with student permission) Prof Keithia Wilson ALTC National Fellow

3. Assessment Marking Phase: Individual Unit/Subject Level FY Assessment Debriefing & Learning for students providing speedy feedback (2 weeks optimal) Providing quality individual feedback which can feed- forward into other assessment tasks - * identifying achievements (encouragement) * identifying clearly & explicitly what students need to do to improve their performance in an encouraging way (developmental) * sufficient comments to justify the mark (fair & just) summarising cohort strengths & weaknesses Prof Keithia Wilson ALTC National Fellow

4. Post-Assessment Phase: Individual Unit/Subject Level Timely feedback – Ensuring students receive feedback within a short time frame (1-2 weeks is optimal) during semester Empowerment - Ensuring opportunities for individual and group discussion of marks and grades Academic recovery – identifying and intervening with at-risk students who have failed their first assessment item. Prof Keithia Wilson ALTC National Fellow

4. Post-Assessment Phase: First Year Program Level Empowerment - Creating a first year culture of ensuring opportunities for individual and group discussion of marks and grades Academic recovery – Program level strategy by identifying & front-loading the threshold (most difficult) unit/s with the aim of ensuring intervening with at-risk students who have failed their first assessment item to assist in their academic recovery. Prof Keithia Wilson ALTC National Fellow

Some Strategies for Academic Recovery Preventative Strategies –  Draft submissions for first assessment tasks with feedback provided  Re-submission of a failed assessment task for a possible passing grade only (1 only in each semester, or semester 1 only, or the first year) Prof Keithia Wilson ALTC National Fellow

Potential hierarchy of academic recovery interventions 1. Students responsible for initiating help-seeking without prompting; 2. communication to all students inviting contact to discuss first assessment and providing information regarding support services; 3. In-class activity discussing first assessment performance and feedback; 4. Targeted written communication to students who failed or nearly failed first assessment inviting contact to discuss first assessment and providing information regarding support services; Prof Keithia Wilson ALTC National Fellow

Potential hierarchy of academic recovery interventions 5. Self-reflective workbook distributed to students for independent completion and invitation for follow-up contact; 6. Phone call to students who failed or nearly failed first assessment inviting contact to discuss first assessment and providing information regarding support services; 7. Targeted invitation to students who failed or nearly failed first assessment inviting participation in structured face- to-face consultation and planning session with tutor. Prof Keithia Wilson ALTC National Fellow

First-Assessment First-Feedback Academic Recovery Intervention Key Idea: Efficacy building for students who ‘fail’ or ‘marginal pass’ first assessment in a core/threshold course Key Aspects: Students complete a self-directed workbook Individual structured session with tutor leading to an action plan Follow-up phone or contact 40% uptake Participation results in a 10% increase in submission rates & 20% increase in pass rates for 2 nd assessment item, & 40% increase in passing the course overall Prof Keithia Wilson ALTC National Fellow

First-Assessment First-Feedback Academic Recovery Intervention At-risk students’ self-reported evaluations of the process were uniformly positive – Academic related learning (5.7/7) Personal development (5.02/7) Insight into reasons for under-performance (5.6/7) Increased efficacy & optimism (5.6/7) Process rated as non-aversive (5.3/7) Tutors reported stronger relationships with students, higher attendance at tutes by those students, & greater student engagement Prof Keithia Wilson ALTC National Fellow

Reflective Task: Post-Assessment Phase How does this information apply to your FY context? What are the opportunities for enhancing FY post- assessment feedback processes? What are the challenges with enhancing FY post- assessment feedback processes? Prof Keithia Wilson ALTC National Fellow

Working with Diverse Students - implications for curriculum design & assessment Identifying our assumptions about the entry levels of Assumed Knowledge of our students (x course + x first semester & then first year overall) & designing course content, structure & assessment tasks to take account of FY students’ entry levels Scaffolding student understanding of assessment tasks by providing resources to make expectations explicit & assist task understanding Scaffolding student learning in each unit/course by building required attitudes, knowledge & skills progressively across the first semester & first year T&L in the FYE needs to be conceptualised as a social experience where students are provided with rich & varied opportunities for interaction & dialogue with peers & academic staff & we are building a relational school/program culture Prof Keithia Wilson ALTC National Fellow