1. Webinar Series This session is the first of e-Assessment Scotland 2013 20 Aug 2013: Challenges and Opportunities Associated With Implementing eMarking Workflows in a University Setting Courtenay Harris, Connie Price and Helen Flavell (Curtin University, Australia) Your hosts Professor Geoff Crisp, Dean Learning and Teaching, RMIT University geoffrey.crisp[@]rmit.edu.au Dr Mathew Hillier, Teaching and Educational Development Institute, University of Queensland mathew.hillier[@]uq.edu.au

2. Challenges and Opportunities Associated With Implementing eMarking Workflows in a University Setting Dr Courtenay Harris Dr Connie Price Dr Helen Flavell

3. Presenters Dr Courtenay Harris School of OTSW Faculty of Health Sciences Dr Connie Price Manager of Assessment Curtin Teaching and Learning Dr Helen Flavell Coordinator of SOTL Faculty of Health Sciences

4. PERTH WESTERN AUSTRALIA

5. Assessment in the 21 st Century Context  Threats to existence of universities  Accrediting student learning remains key function of HE  Infiltration of technology  Decreasing government funding  Increasing accountability  Increasing student numbers  Increasing student diversity  Increased focus on role and value of assessment  What aspects can be automated or done by AI  Need to develop accurate costing models for assessment  Including different assessment types  Greater proportion of workload around/not in the marking process  Every aspect of assessment is magnified  Need for focused feedback HE ContextAssessment Issue

6. Foundations for Professional Health Practice 100 (FPHP 100)  Common first year unit for all courses in the Faculty of Health Sciences  Super large unit  2900 students across both semesters  40 teaching staff  Inter-professional education  staff and students  Internal (F2F) and external (online) student enrolments  Teaching model and use of class based technology  Varied assessment types

7. Drivers for eMarking within FPHP 100 Electronic submission / return Practicalities due to 2900 students /40 tutors Feedback timeframes Access to student submissions Academic integrity Consistent and efficient assessment practice and feedback implementation of marking rubrics transparency for staff and students Change of balance of editing vs focused feedback Efficiency for staff, while maintaining rich feedback Students Student evaluation of the unit indicated improvement required

8. Electronic Management of Assignments

9. Submission Annotate Apply Rubric Calculate Grade Record Grade Return Feedback Grading Assessment Management Workflow

10. Submission Annotate Apply Rubric Calculate Grade Record Grade Return Feedback Grading Assessment Management Workflow

11. Submission Annotate Apply Rubric Calculate Grade Record Grade Return Feedback Grading Assessment Management Workflow

12. Submission Annotate Apply Rubric Calculate Grade Record Grade Return Feedback Grading Assessment Management Workflow

13. Workflow Set-up your iPad Open PDF Expert app Set-up your Computer Open up Blackboard FPHP Open Grade Centre with Smart view of tutorial group Open your commenting tool Pre-marking tasks Click on chevron to view student attempt Download student assignment file Open file in commenting tool (Word or Acrobat) Open master copy of marking rubric Marking tasks Check Originality Report Add comments to student’s file Complete and save marking rubric Post-marking tasks In Grade Centre enter mark Attach commented assignment file Attach marking rubric Repeat for each student

15. eMarking Working Party Priorities 1.Workflow Minimal steps Minimal tools Multiple screens Minimise workload 2.Enhancing Integrity Academic integrity (plagiarism check) Moderation Archiving of electronic artefacts 3.Assessment and Feedback Evaluation -> Mark Mark-up of artefact Provide feedback in a variety of ways

16. Electronic Assignment Workflow for Blackboard Learn 9.1 Launch MS Word or Adobe Acrobat Create Turnitin assignment Submit assignment Create Blackboard eRubric Grade Assignment Upload annotated submission View mark Download feedback View eRubric Store assignment artifacts Staff WorkflowStudent Workflow Download submissions from Grade Centre BlackboardDesktop Application View Turnitin Originality Reports Annotate submission View attempts via Grade Centre View Originality Report Receive Digital Receipt

17. Electronic Assignment Workflow for ReMarksPDF Launch ReMarksPDF App Create eSubmission Link Submit assignment as.pdf Create eRubric Grade Assignment Provide feedback View mark Download feedback View eRubric Store assignment artifacts Staff WorkflowStudent Workflow Download submissions from Grade Centre.zip file Upload.zip to Turnitin Blackboard ReMarksPDF Turnitin View Turnitin Originality Reports Connect to Bb database Upload annotated submissions to Grade Centre

18. Electronic Assignment Workflow for GradeMark Create eRubric Create eSubmission Link Submit assignment View Originality Report Complete eRubric Annotate document and provide feedback View mark View feedback View eRubric Store assignment artefacts Staff WorkflowStudent Workflow View Originality Report Receive Digital Receipt

19. “The 2013 Horizon Project Australia Advisory Board agreed with the global group that digital media literacy is not nearly pervasive enough in faculty training. There is a need for more training before being asked to teach, and for more professional development opportunities once in the profession. This key challenge is underscored by the widespread belief that most academics are not leveraging emerging technologies for their own work, whether that be in the classroom or in support of their own research. While a lack of adequate training opportunities is a part of the challenge, ultimately a change in the mind sets of disciplines and individual faculty will be required, along with cultural shifts within institutions, before emerging tools and technologies are routinely adopted and implemented as a matter of course.”

20. “Being agile as a teacher means staying true to your purpose, fixed in your intent to provide excellence in education but always adapting and adopting new methods of achieving that goal and where prompted to by changes in the world around you: the Internet demonstrates how agility is now a key aspect of knowledge work—make it part of your teaching.” (Allen M, 2012)

21. Developing academic staff capacity for agile teaching: a case study in implementing eMarking workflows for student assessment in a large first year Health Sciences unit Office of Learning and Teaching Seeding Grant Funded Project

22. Professional Development Approach Theories of change and change management within the context of academic leadership and current higher education change drivers Individual reflection on self as well as social and cultural beliefs in relation to innovation, teaching and learning and change

23. Professional Development Approach Exploration of intrapersonal skills such as self-talk to deconstruct habitual thinking and its effect on performance and acceptance of ICT innovations Develop an understanding of ICT system components as opposed to “how to use” a program to support a shift in thinking/approach from mastery t o adaptive responses.

24. Method Pre and post PD questionnaire combining measures relating to attitudes to teaching and technology acceptance. Semi-structured interviews with 16 participants following the first major assessment piece. Coded and analysed by a qualitative researcher using NVivo. Two focus groups (12 participants in total) to check themes that emerged from interviews.

25. Themes The PD influenced them in the following ways: Cognitive (majority of participants reported) - increased awareness by bringing new knowledge, adding to existing knowledge, refreshing knowledge. Knowledge on software applications, strategies, self awareness to create creative thinking / planning for immediate and future plans. Emotional (majority reported) – increased confidence, excitement, enthusiasm, motivation, empowerment Practical (majority reported)– taking action to improve teaching practice, manage both IT and teaching challenges, using IT and teaching strategies, improving skills

26. Characteristics of the participant are key to being agile in the adoption of ICT Characteristics demonstrated by participants included: Professional background / employment status / professional identity Previous work and teaching experience Family / personal responsibilities and experiences Self efficacy, willingness to persevere Willingness to engage with others, mindful of others in the unit / in regards to IT support Ability to employ intrapersonal strategies to cope with changing environments ‘Can-do’ attitude

27. Discussion Issues of self-selection Sessional staff Disciplinary background Impact of workload/personal life on uptake Change in behaviours requires ongoing commitment/time to reflect Structured nature of the unit

28. Sources Slide 3 images http://babynameobsessed.wordpress.com/2012/12/28/leilani-and-jax/the-cottesloe-beach- perth-australia/ http://babynameobsessed.wordpress.com/2012/12/28/leilani-and-jax/the-cottesloe-beach- perth-australia/ http://www.australia.com/explore/cities/perth/perth-surrounds.aspx http://www.police.wa.gov.au/Aboutus/Policejurisdictions/tabid/1352/Default.aspx Slide 12 image http://thatstotallytarot.com/wp-content/uploads/2011/01/work_stress.jpg http://thatstotallytarot.com/wp-content/uploads/2011/01/work_stress.jpg Slide 17 Johnson, L., Adams Becker, S., Cummins, M., Freeman, A., Ifenthaler, D., and Vardaxis, N. (2013). Technology Outlook for Australian Tertiary Education 2013-2018: An NMC Horizon Project Regional Analysis. Austin, Texas: The New Media Consortium. Slide 18 Allen, M.). Move quickly to adopt new approaches. Learning in Networks of Knowledge Retrieved June 22, 2012, from http://www.knowledgenetworklearning.net/concepts/pedagogy/teachingagility/ http://www.knowledgenetworklearning.net/concepts/pedagogy/teachingagility/