1. Programme assessment design: How might technology help? Dr Greg Benfield Oxford Centre for Staff and Learning Development

2. Landscape: social software ©BBC http://news.bbc.co.uk/1/hi/ technology/3830527.stm An underworld of digital communication among learners (LEX, Creanor et al 2006) Google and Wikipedia preferred information search & retrieval tools (LXP, Conole et al 2006) The concept of time is changing – both in terms of expectation of information and results on demand. There is evidence of a fragmentation of the learning timetable (LXP, Conole et al 2006)

3. Oxford Centre for Staff and Learning Development Immersed in technology technologically adept and had integrated ICT into their lives (JISC 2007: 10) student respondents [are] immersed in technology ownership and use and impatient with instructors who dont have adequate technical skills (Salaway et al 2007: 5) ©BBC http://news.bbc.co.uk/1/hi/ technology/3830527.stm

4. Oxford Centre for Staff and Learning Development Assessment and feedback measurable impact Formative use of CAA stands out as a rare application of e-learning leading to measurable impact on student performance (Sharpe et al 2006)

5. Oxford Centre for Staff and Learning Development Scenarios 1.VLE supporting self directed study in financial accounting 2.Automated assessments in life sciences 3.Personal response systems supporting discussion of problems in engineering Sharpe, R., Benfield, G., Roberts, G. & Francis, R. (2006) The undergraduate experience of blended e-learning: a review of UK literature and practice undertaken for the Higher Education Academy. At http://www.heacademy.ac.uk/4884.htmhttp://www.heacademy.ac.uk/4884.htm

6. Oxford Centre for Staff and Learning Development Background 600 undergraduate students Needed efficient management of self-directed study Student diversity Student expectations of flexibility Scenario 1: Newcastle Business School

7. Oxford Centre for Staff and Learning Development Background 600 undergraduate students Needed efficient management of self-directed study Student diversity Student expectations of flexibility An approach to the problem Online resources Weekly directed study tasks Answers on timed release Formative self- assessment quizzes Scenario 1: Newcastle Business School

8. Oxford Centre for Staff and Learning Development Scenario 1: Newcastle Business School conceptualisation construction dialogue

9. Oxford Centre for Staff and Learning Development Survey responses from 200 students Most agreed Blackboard helped them study Most used it weekly High usage of content and quizzes, less of discussions Students valued Access to learner materials Greater independence Time saving More convenient What the students said

10. Oxford Centre for Staff and Learning Development Background 1 st year introductory lecture based course Lecture notes on web since 2002 High failure rate Poor turnaround time for assignments Inconsistency in quality of feedback and marking Scenario 2: Life Sciences at Dundee

11. Oxford Centre for Staff and Learning Development Background 1 st year introductory lecture based course Lecture notes on web since 2002 High failure rate Poor turnaround time for assignments Inconsistency in quality of feedback and marking An approach to the problem Computer-aided assessment (CAA) Scenario 2: Life Sciences at Dundee

12. Oxford Centre for Staff and Learning Development No negative feedback regarding the use of CAA 99% of those who responded said the online tutorials improved their understanding of the topics covered 93% felt that the marks awarded were appropriate The ability to re-do tests and assignments again once you have already done it is good. It enables you to continue learning the more you try it. The overall module pass rate rose from 73% to 93%. Evaluation

13. Oxford Centre for Staff and Learning Development Background Weak conceptual understanding Insufficient interaction and discussion in class Low student motivation An approach to the problem Personal response system in class Question and answer with discussion/defence of answers. Scenario 3: Engineering at Strathclyde

14. Oxford Centre for Staff and Learning Development Evaluation 74% of students agreed that in comparison with traditional lecture classes, the teaching methods used here were more effective in helping them to acquire an understanding of fundamental engineering concepts. 95% agreed that I am more actively involved in PRS than traditional classes In focus groups students attributed their understanding to being active in class having time to think, reflect and discuss discussion with other students in peer groups. Scenario 3: Engineering at Strathclyde

15. Oxford Centre for Staff and Learning Development Extended, technology- enhanced assignments http://www.brookes.ac.uk/schools/education/arts/diaries/home.htmlhttp://www.brookes.ac.uk/schools/education/arts/diaries/home.html! http://blogs.warwick.ac.uk/groups/en-all http://core.mwbrookes.org.uk/ E.g. student journals using blogs

16. Oxford Centre for Staff and Learning Development …. or e-portfolios http://elgg.net/

17. Oxford Centre for Staff and Learning Development Wikis Podcasts Student research journals (Geoverse) Collaborative student artifacts

18. Oxford Centre for Staff and Learning Development communication/collaboration Students are part of a wider, networked, community of peers …[in which they] share resources, ask for help and peer assess (LXP, Conole et al 2006) Yet … In the surveys of courses making use of multiple features of the VLE, discussions frequently appear as the part which is least used and valued by students … and as being something that is difficult to engage undergraduate students with. (Sharpe et al 2006)

19. Oxford Centre for Staff and Learning Development Virtual Teamwork Virtual Teamwork (Business School) Multi-professional learning: Partnerships in Practice Multi-professional learning: Partnerships in Practice (Health and Social Care) VLEs beyond the filing cabinet: communication/collaboration

20. Oxford Centre for Staff and Learning Development Radical redesign of a 1 st year Chemistry course University of Dundee project (Morrris & Walker 2006) involved: complete redesign from traditional paper-based assessment to online assessment formative + summative CAA regular, frequent, formative online quizzes with multiple attempts allowed and feedback on incorrect answers Impact: pass rate rose from 73% to 93% between successive cohorts with introduction of CAA Comparison of entry qualifications of cohorts showed CAA cohort slightly less qualified than previous cohort Morris, L. and Walker, D. (2006). CAA sparks chemical reaction: integrating CAA into a learning and teaching strategy. Evaluation of the use of the virtual learning environment in higher education across Scotland, QAA Scotland.

21. Oxford Centre for Staff and Learning Development References Biggs, J. (2003). Teaching for Quality Learning at University Second Edition. Maidenhead, Open University Press. Browne, T. and Jenkins, M. (2003). 'VLE surveys: a longitudinal perspective between March 2001 and March 2003 for Higher Education in the United Kingdom.' UCISA. online http://www.ucisa.ac.uk/groups/tlig/vle/vle2003.pdf, accessed 12 November 2003 Catley, P (2005). One Lecturer's Experience of Blending E-learning with Traditional Teaching or How to Improve Retention and Progression by Engaging Students. Brookes eJournal of Learning and Teaching, 1(2) online at http://www.brookes.ac.uk/publications/bejlt/volume1issue2/academic/catley05_1.html! http://www.brookes.ac.uk/publications/bejlt/volume1issue2/academic/catley05_1.html Collis, B. & Moonen, J. (2005). An On-Going Journey: Technology as a Learning Workbench. [Online] at http://bettycollisjefmoonen.nl/Book-Learning-Workbench-V2.pdfhttp://bettycollisjefmoonen.nl/Book-Learning-Workbench-V2.pdf Conole, G., De Laat, M., Dillon, T. and Darby, J. (2006, November 2006). "JISC LXP: Student Experiences of Technologies Draft Final Report." November 2006.[Online] Retrieved 20 Nov, 2006, from http://www.jisc.ac.uk/media/documents/lxp_project_final_report_nov_06.pdf. Creanor, L., Trinder, K., Gowan, D. and Howells, C. (2006, August 2006). "LEX: The Learner Experience of e-Learning Final Project Report August 2006." [Online] Retrieved 2 November, 2006, from http://www.jisc.ac.uk/uploaded_documents/LEX%20Final%20Report_August06.pdfhttp://www.jisc.ac.uk/uploaded_documents/LEX%20Final%20Report_August06.pdf DfES (2005) 'Harnessing Technology: Transforming learning and children's services'. Online at http://www.dfes.gov.uk/publications/e-strategy/ http://www.dfes.gov.uk/publications/e-strategy/ HEFCE (2005). HEFCE strategy for e-learning, online at http://www.hefce.ac.uk/pubs/hefce/2005/05_12/ http://www.hefce.ac.uk/pubs/hefce/2005/05_12/ JISC (2003). 'Virtual and Managed Learning Environments.' Joint Information Systems Committee. online at http://www.jisc.ac.uk/index.cfm?name=issue_vle_mle, accessed 25 August 2005.http://www.jisc.ac.uk/index.cfm?name=issue_vle_mle JISC (2004). Effective Practice with e-Learning: A good practice guide in designing for e-Learning. Bristol, JISC. Online at http://www.jisc.ac.uk/uploaded_documents/jisc%20effective%20practice3.pdf http://www.jisc.ac.uk/uploaded_documents/jisc%20effective%20practice3.pdf

22. Oxford Centre for Staff and Learning Development References contd JISC (2005). Innovative Practice with e-Learning. Bristol, JISC. Online at http://www.jisc.ac.uk/uploaded_documents/publication_txt.pdf http://www.jisc.ac.uk/uploaded_documents/publication_txt.pdf JISC. (2007). "Student Expectations Study: Key findings from online research and discussion evenings held in June 2007 for the Joint Information Systems Committee." [Online] Retrieved 10 September, 2007, from http://www.jisc.ac.uk/media/documents/publications/studentexpectations.pdf http://www.jisc.ac.uk/media/documents/publications/studentexpectations.pdf Laurillard, D. (1993). Rethinking University Teaching-A framework for the effective use of educational technology. New York, Routledge. Laurillard, D. (2002). Rethinking University Teaching-a conversational framework for the effective use of educational technology. London, RoutledgeFarmer. Mayes, T and de Freitas, S. (2004) Review of e-learning theories, frameworks and models. JISC. Online at http://www.jisc.ac.uk/uploaded_documents/Stage%202%20Learning%20Models%20(Version%2 01).pdf http://www.jisc.ac.uk/uploaded_documents/Stage%202%20Learning%20Models%20(Version%2 01).pdf Sharpe, R, Benfield, G, Roberts, G and Francis, R (2006). "The undergraduate experience of blended e-learning: a review of UK literature and practice undertaken for the Higher Education Academy." Retrieved 3 October, 2006, from http://www.heacademy.ac.uk/research/Sharpe_Benfield_Roberts_Francis.pdf http://www.heacademy.ac.uk/research/Sharpe_Benfield_Roberts_Francis.pdf Salaway, G., Caruso, J. B. and Nelson, M. R. (2007). "The ECAR Study of Undergraduate Students and Information Technology, 2007." [Online] Retrieved 9 October, 2007, from http://www.educause.edu/ir/library/pdf/ers0706/rs/ERS0706w.pdf http://www.educause.edu/ir/library/pdf/ers0706/rs/ERS0706w.pdf

23. Oxford Centre for Staff and Learning Development Virtual teamwork 200+ 1 st year Business students Formed randomly into virtual teams of 6 members each Students collaborate online for 4 weeks to create a PowerPoint presentation on a specific teamwork theme (e.g. motivation theory) Issues: Practicality, logistics Learning Engagement Assessment

24. Oxford Centre for Staff and Learning Development Online tutoring (podcasts) Online CPD short course Weekly podcasts allow tutors to introduce and explain key ideas and comment on previous weeks learning points Easy and quick to produce Wiki is perfect for collaborative writing

25. Oxford Centre for Staff and Learning Development Online tutoring (Wiki 1) Groups produce a collaborative presentation whose content is first discussed/debated/organised using WebCT Discussions

26. Oxford Centre for Staff and Learning Development Online tutoring (Wiki 2) The full history of page revisions is preserved in a Wiki

27. Oxford Centre for Staff and Learning Development Partnerships in Practice

28. Oxford Centre for Staff and Learning Development Assessment for learning Scenario First term, first year compulsory law module A new subject for most (75%) students High failure rate (25%), poor general results (28% 3rd class, 7% Ist) Solution: Weekly optional WebCT quizzes (50% take-up) Outcome: Quiz takers: 4% fail, 14% 3rd class, 24% Ist Non-quiz takers: same pattern as before Overall:14% fail (approx half previous figure) 21% 3rd class 14% 1st (double previous figure) (see Catley 2005)