1. The potential Impact of Computer Aided Assessment Technology in Higher Education
Ernest Azwindini Tshibalo
University of South Africa,
Pretoria, South Africa
NADEOSA 10th Anniversary Conference
23-24 August 2006

2. Overview Introduction Computer Aided Assessment (CAA)
Rationale for using CAA
Potential benefits and limitations of CAA
CAA and its potential impact on student learning
Strategies for developing effective CAA technology in Higher Education
Conclusion

3. Introduction
An overhaul of traditional assessment procedures in higher education is long overdue, as present methods are ill-suited to the digital age in which using information is more important than remembering it (Mason 1998).
Distance learning physically separates students from educators – interventions necessary
Increased number of students in Higher Education
Increased time spent by staff on assessment
E-learning courses & assessment
How can technology assist in this area?

4. Computer Aided Assessment - CAA
… the use of computers to deliver, mark and analyse assignments or examinations, to keep a record of assessment and to provide feedback.
(Greenberg 1998; Lingered 2000;
McKenna & Bull 2000)
“…any instance in which some aspects of computer technology is deployed as part of assessment process.”
(Atkinson & Davies 2000:2)

5. Computer technology and its role in assessment
Online completion of interactive exercises and tests
The use of revision software
Electronically mailed coursework
Marking and feedback
Web pages to set tasks for students and provide tutor support
Software to detect plagiarism etc.

6. Rationale for using CAA in HE
Increasing academic workload in HE
Too many students
Not enough teachers
Students need many assessment opportunities
The increase in student numbers:
At Unisa, in some first year modules can now reach as many as students
eg, more than in Economics 1.

7. Rationale continues… Paper-based versus computer-based assessment
Traditional assessment methods – paper based
Certain skills cannot be assessed e.g. listening
CAA diversifies assessment tasks
CAA provides flexible assessment
ODL institutions need to change assessment practice to meet these needs.

8. Rationale continues CAA and students with learning disabilities
The nature of computers as information processing tools, and their possibility of creating instructional tools to meet individual student needs, make computers a potential powerful technology for assessment (Kumar, 1994,4).
Lack of the computer technology – prevents disabled students to access HE
Lack of CAA ‘closes’ the ‘openness’ of ODL

9. Potential benefits and limitations
It is generally agreed that assessment plays a central role in student learning irrespective of the mode.
Computer based technology challenges students to learn new skills and ways of learning.
Devlin (2002,2)argues that online assessment teaches skills that is not possible to gain from a normal essay

10. Some potential benefits of CAA technology
Use of multimedia, for example, photographs, diagrams, maps, audio, video, and animations can be used as part of the question and or answer.
Automatic, immediate, and detailed, feedback to all students.
Objective and consistent standard during marking, however in traditional hand marking – subjectivity..
Information is more accessible to disabled students
E.g.: Digital Audio Integration System – listen to text,
Optical Character Recognition – scan text

11. Some limitations of CAA technology
African students – technologically less advantaged
CAA may be unfair to students who are unfamiliar with computer operations.
CAA is unable to assess skills such as constructive argument, writing, presentation and interpersonal
Testing High order cognitive skills such as synthesis and analysis – Bloom’s revised taxonomy can solve
Student authentication, cheating and plagiarism – can be solved by invigilation, plagiarism detection software, and biometrics (finger print measurement)

12. CAA and its impact on student learning
CAA is very good for formative assessment in distance education
Formative assessment is designed to assist the learning process by providing feedback to learners.
Feedback motivates students and inform them how well they are performing.

13. CAA and its impact continues..
Example 1. Online Marking and Results system (OMAR)
OMAR is a web-based software that permits academics to customise online templates for performing their marking and offer feedback to students
Its introduction at the University of Melbourne led to improved student learning (McKenzie, 2004,1-8)
ODL institutions can benefit

14. CAA and its impact continues..
Example 2. Birkbeck College, University of London
Baggott and Rayne (2001) used frequent computer-based assessments (CBA) to test strategies for improving learning support for mature, part-time students in a first year Biology Curriculum.
Extensive feedback was given, areas of need were identified and assistant given.
Results: Mean CBA score = 65% compared to 50% of none participants

15. CAA and its impact continues..
Conclusions made from the use of CBA
CBA provided student support –as evidenced by benefits gained from formative tests
CBA promoted learning as evidenced by the fact that the median score was maintained in open and closed book summative assessment
Extensive feedback – student success

16. CAA impact on student learning continues..
Example 3. Research studies in Norway
Lysne and Jostein (2000) used WebOrama computer tool to involve distance learning students in their learning
WebOrama is a system where video clips, sounds and texts are integrated
Its advantages: engagement, work & guidance.
Results: ODL students compared to internal students

17. Impact continues Scores of the final exams revealed the following:
Distance students scored equally or even better than internal students
Conclusion: Increased emphasis on active students, and problem based working methods made for better scores
Computer based technology has the potential to actively engage distance students in their learning and improve their scores

18. Strategies for developing effective CAA in HE
Devlin (2002) introduced strategies for developing
effective CAA.
His strategies are summarised into three checklist:
An access and usage checklist
A quality of teaching and learning checklist
A technical and administrative checklist

19. Strategies continues.. 1. Access and usage checklist
Make sure all students have access to computers and that computer skills are developed
Technologically disadvantaged students should receive support, African students?
Unisa students for example, should have access to the main and regional campuses

20. Strategies continues 2. Quality of teaching and learning
Assessment method should relate to the learning outcomes of the course
Identify where CAA might be beneficial
High order cognitive skills should be considered
It must be clear what is being assessed and why
Rapid feedback from both students and academics
Appropriate technology for students with learning disabilities essential

21. Strategies continues.. 3.Technical and administrative checklist
Computer used by both students and academics should be compatible
Computer system used should deal with problems related to password, access and usage
Adequate technical support staff during the development and use of CAA
Emergency backup facilities should be put in place

22. Strategies continues.. Start up guidelines:
Start with clear learning outcomes
Start small
Start were success is most likely

23. Conclusion
Many issues should be explored before embarking on the use of CAA
In this paper the following were explored:
Definition, rationale, benefits and limitations, impact on student learning and development strategies.

24. Conclusion continues.. In conclusion.
CAA cannot be an answer to all challenges related to assessment in HE
It can be considered where it best benefits teaching and learning.
Where the technology is available, institutions are encourages to optimise its use for the benefit of all

25. It should be used where it best benefits teaching and learning.
CAA should not be considered as a complete replacement of all other methods of assessment…
It should be used where it best benefits teaching and learning.

26. Contact details Azwindini Ernest Tshibalo Education Consultant
Institute for Curriculum & Learning Development
University of South Africa, Pretoria
Tel: