1. eaaeaa The e-Assessment Association www.e-assessmentassociation.com What can e-assessment do for Teaching & Learning? Cliff Beevers and Bill Foster e-Assess Association On behalf of eAA expert panel

2. eaaeaa The e-Assessment Association www.e-assessmentassociation.com Introduction caa.ecs.soton.ac.uk/Papers/ Technology to serve Education – Build on research More complex question & test structure – Fast/accurate feedback Efficient use of data

3. eaaeaa The e-Assessment Association www.e-assessmentassociation.com Definition E-assessment occurs when there is an automated marking/response, to student input on-screen in a test, informing on the process of answering a question and providing feedback to learners and their teachers through well-crafted advice and reports.

4. eaaeaa The e-Assessment Association www.e-assessmentassociation.com Outline ARG principles – Successful e-assessment Other research – Greenwood et al (1998) Int Conf Mathematics / Science Educ and Tech, Vol. 2000, Issue. 1, 2000, 179 – Sandine et al (2005) “The National Assessment of Educational Progress in Online Assessment in Mathematics and Writing” – Ashton et al (2005) BJET, 36(5), 771-787

5. eaaeaa The e-Assessment Association www.e-assessmentassociation.com ARG’s formative principles Effective planning of T&L Focus on how students learn Centre of classroom practice Promote professional skills Sensitive/constructive

6. eaaeaa The e-Assessment Association www.e-assessmentassociation.com ARG’s formative principles Foster motivation Promote learning goals Help improve learning Develop self-assessment Recognise range of achievement

7. eaaeaa The e-Assessment Association www.e-assessmentassociation.com ARG’s summative principles Valid: measure appropriate achievements Reliable: accurate and consistent Impact: desirable for T&L Practical: resources provide value

8. eaaeaa The e-Assessment Association www.e-assessmentassociation.com Examples of formative success ALTA at Primary/Secondary SCHOLAR at Secondary/Tertiary Medicine at Dundee Maths at Newcastle/Birmingham Physics at Surrey Open University courses Professional eg Accountancy

9. eaaeaa The e-Assessment Association www.e-assessmentassociation.com Newcastle Experience: Formative Headlines Fully integrated e-learning and formative assessment system in the School of Maths & Stats. since 2006. Used for all M & S modules in first two years as well as some large service courses. Easy to use, good feedback from students. Challenging questions at the right level (symbolic input - little or no MCQ). More opportunities for practising skills Use: ~4000 students from all three faculties. Around 180,000 sessions so far.

10. eaaeaa The e-Assessment Association www.e-assessmentassociation.com Newcastle: Why? More formative assignments needed to monitor progress and with timely feedback. More opportunities for student practice in basic skills needed. Diversify assignment methods. Marking issues of standard paper based assignments for first year courses: time, money and uniform standards in marking. Decided on CBA – biased towards formative assessment based on experience at Brunel, Heriot-Watt and Southampton Universities.

11. eaaeaa The e-Assessment Association www.e-assessmentassociation.com Newcastle:What Happened? Decided on i-assess, derived from CALM. Developed initially at Heriot-Watt. School policy developed** Report: Using computer based assessment in first year mathematics and statistics degree courses at Newcastle University: Report http://mathstore.ac.uk/headocs/Foster_B.pdf Student engagement high (90% completion of in-course assignments). Standards have been maintained. Evidence that 1st year retention improved. Formative e-Assessment Award Winner (eAssessment Scotland 2010).

12. eaaeaa The e-Assessment Association www.e-assessmentassociation.com Newcastle: Summative Assessments High volume, high stakes exams using LMSs (Moodle, Blackboard etc). Secure environments with secure databases. Created SCORM objects from i-assess assessments. Multi-use across many platforms.

13. eaaeaa The e-Assessment Association www.e-assessmentassociation.com Newcastle: How? i-assess i-assess on Univ Server SCORM Moodle SCORM Blackboard Clusters External World via Web External World via RAS University via LAN i-assess on separate server SCORM objects On DVDs Distance learning

14. eaaeaa The e-Assessment Association www.e-assessmentassociation.com Further Example of summative success CALM/Mathwise formative from 1985 at HW 1995 research to check on Reliability & Validity Impact: encouraged use of formative e-resources Practical: 500 undergraduates tested over week – randomisation – mathematical expressions – steps – provided immediate re-test.

15. eaaeaa The e-Assessment Association www.e-assessmentassociation.com QTI Benefits: – no lock in to supplier – question sharing potential Barriers: – reduced range & quality of question types – MCQ + variants too restrictive – lessons not passed on – not appropriate in STEM subjects – lack of adaptive modes

16. eaaeaa The e-Assessment Association www.e-assessmentassociation.com Barriers Time to create good questions High initial costs Poor collaboration history NIH Digital divide

17. eaaeaa The e-Assessment Association www.e-assessmentassociation.com PASS-IT in Scotland Collaborative project: HW/LTS/BBC/SFEU/SQA Explored formative/summative e-testing in post-16 education Pilots delivered: 6 levels in 1 subject – Variety of levels in 5 other subjects Support candidates with additional needs Summary at www.pass- it.org.uk/project_materials.asp

18. eaaeaa The e-Assessment Association www.e-assessmentassociation.com Conclusions What works in Education? Make that the standard. E-assessment can: Enhance teaching, improve learning and measure part of it. Formative: SCHOLAR (Scotland) from 1999 Research: www.pass-it.org.uk, 2003-5 Summative: e-NABs from 2006.