The potential Impact of Computer Aided Assessment Technology in Higher Education Ernest Azwindini Tshibalo University of South Africa, Pretoria, South Africa NADEOSA 10th Anniversary Conference 23-24 August 2006

Overview Introduction Computer Aided Assessment (CAA) Rationale for using CAA Potential benefits and limitations of CAA CAA and its potential impact on student learning Strategies for developing effective CAA technology in Higher Education Conclusion

Introduction An overhaul of traditional assessment procedures in higher education is long overdue, as present methods are ill-suited to the digital age in which using information is more important than remembering it (Mason 1998). Distance learning physically separates students from educators – interventions necessary Increased number of students in Higher Education Increased time spent by staff on assessment E-learning courses & assessment How can technology assist in this area?

Computer Aided Assessment - CAA … the use of computers to deliver, mark and analyse assignments or examinations, to keep a record of assessment and to provide feedback. (Greenberg 1998; Lingered 2000; McKenna & Bull 2000) “…any instance in which some aspects of computer technology is deployed as part of assessment process.” (Atkinson & Davies 2000:2)

Computer technology and its role in assessment Online completion of interactive exercises and tests The use of revision software Electronically mailed coursework Marking and feedback Web pages to set tasks for students and provide tutor support Software to detect plagiarism etc.

Rationale for using CAA in HE Increasing academic workload in HE Too many students Not enough teachers Students need many assessment opportunities The increase in student numbers: At Unisa, in some first year modules can now reach as many as 30 000 students eg, more than 20 000 in Economics 1.

Rationale continues… Paper-based versus computer-based assessment Traditional assessment methods – paper based Certain skills cannot be assessed e.g. listening CAA diversifies assessment tasks CAA provides flexible assessment ODL institutions need to change assessment practice to meet these needs.

Rationale continues CAA and students with learning disabilities The nature of computers as information processing tools, and their possibility of creating instructional tools to meet individual student needs, make computers a potential powerful technology for assessment (Kumar, 1994,4). Lack of the computer technology – prevents disabled students to access HE Lack of CAA ‘closes’ the ‘openness’ of ODL

Potential benefits and limitations It is generally agreed that assessment plays a central role in student learning irrespective of the mode. Computer based technology challenges students to learn new skills and ways of learning. Devlin (2002,2)argues that online assessment teaches skills that is not possible to gain from a normal essay

Some potential benefits of CAA technology Use of multimedia, for example, photographs, diagrams, maps, audio, video, and animations can be used as part of the question and or answer. Automatic, immediate, and detailed, feedback to all students. Objective and consistent standard during marking, however in traditional hand marking – subjectivity.. Information is more accessible to disabled students E.g.: Digital Audio Integration System – listen to text, Optical Character Recognition – scan text

Some limitations of CAA technology African students – technologically less advantaged CAA may be unfair to students who are unfamiliar with computer operations. CAA is unable to assess skills such as constructive argument, writing, presentation and interpersonal Testing High order cognitive skills such as synthesis and analysis – Bloom’s revised taxonomy can solve Student authentication, cheating and plagiarism – can be solved by invigilation, plagiarism detection software, and biometrics (finger print measurement)

CAA and its impact on student learning CAA is very good for formative assessment in distance education Formative assessment is designed to assist the learning process by providing feedback to learners. Feedback motivates students and inform them how well they are performing.

CAA and its impact continues.. Example 1. Online Marking and Results system (OMAR) OMAR is a web-based software that permits academics to customise online templates for performing their marking and offer feedback to students Its introduction at the University of Melbourne led to improved student learning (McKenzie, 2004,1-8) ODL institutions can benefit

CAA and its impact continues.. Example 2. Birkbeck College, University of London Baggott and Rayne (2001) used frequent computer-based assessments (CBA) to test strategies for improving learning support for mature, part-time students in a first year Biology Curriculum. Extensive feedback was given, areas of need were identified and assistant given. Results: Mean CBA score = 65% compared to 50% of none participants

CAA and its impact continues.. Conclusions made from the use of CBA CBA provided student support –as evidenced by benefits gained from formative tests CBA promoted learning as evidenced by the fact that the median score was maintained in open and closed book summative assessment Extensive feedback – student success

CAA impact on student learning continues.. Example 3. Research studies in Norway Lysne and Jostein (2000) used WebOrama computer tool to involve distance learning students in their learning WebOrama is a system where video clips, sounds and texts are integrated Its advantages: engagement, work & guidance. Results: ODL students compared to internal students

Impact continues Scores of the final exams revealed the following: Distance students scored equally or even better than internal students Conclusion: Increased emphasis on active students, and problem based working methods made for better scores Computer based technology has the potential to actively engage distance students in their learning and improve their scores

Strategies for developing effective CAA in HE Devlin (2002) introduced strategies for developing effective CAA. His strategies are summarised into three checklist: An access and usage checklist A quality of teaching and learning checklist A technical and administrative checklist

Strategies continues.. 1. Access and usage checklist Make sure all students have access to computers and that computer skills are developed Technologically disadvantaged students should receive support, African students? Unisa students for example, should have access to the main and regional campuses

Strategies continues 2. Quality of teaching and learning Assessment method should relate to the learning outcomes of the course Identify where CAA might be beneficial High order cognitive skills should be considered It must be clear what is being assessed and why Rapid feedback from both students and academics Appropriate technology for students with learning disabilities essential

Strategies continues.. 3.Technical and administrative checklist Computer used by both students and academics should be compatible Computer system used should deal with problems related to password, access and usage Adequate technical support staff during the development and use of CAA Emergency backup facilities should be put in place

Strategies continues.. Start up guidelines: Start with clear learning outcomes Start small Start were success is most likely

Conclusion Many issues should be explored before embarking on the use of CAA In this paper the following were explored: Definition, rationale, benefits and limitations, impact on student learning and development strategies.

Conclusion continues.. In conclusion. CAA cannot be an answer to all challenges related to assessment in HE It can be considered where it best benefits teaching and learning. Where the technology is available, institutions are encourages to optimise its use for the benefit of all

It should be used where it best benefits teaching and learning. CAA should not be considered as a complete replacement of all other methods of assessment… It should be used where it best benefits teaching and learning.

Contact details Azwindini Ernest Tshibalo Education Consultant Institute for Curriculum & Learning Development University of South Africa, Pretoria Tel: +27-12-429-6569 tshibae@unisa.ac.za