An Introduction to Computer- assisted Assessment Joanna Bull and Ian Hesketh CAA Centre Teaching and Learning Directorate

Overview What is CAA? Why use it? Objective tests Integration: assessment methods & strategies Hands-on session

Computer-assisted Assessment I Computer-assisted assessment (CAA) is a common term for the use of computers in the assessment of student learning.

Computer-assisted assessment II Deliver, mark and analyse assignments or examinations (computer or web-based) Collate and analyse data gathered from optical mark readers (OMR) Record, analyse and report on achievement Collate, analyse and transfer assessment information through networks

Assessment Formative primarily to provide feedback to students and promote student learning Summative contributes to final grade for course/module Diagnostic to determine ability and understanding Self-assessment to allow students to evaluate their learning

Why use CAA ? What are the benefits to using CAA?

Why use CAA ? Increasing the frequency of assessment Increasing feedback Widening the range of assessment methods Provide objective and consistent marking Decrease marking Administrative efficiency

Increasing the frequency of assessment Motivating students to learn Encouraging students to practice skills Broadening the knowledge assessed

Increasing feedback Students to motivate students needs to be timely, accurate and constructive Staff about student performance about test and question performance

Widen range of assessment methods Different assessment methods for different skills, abilities and knowledge Balance of assessment methods Utilisation of multimedia to create innovative assessments

Provide objective and consistent assessment Assessment can be unreliable, subjective and inconsistent Automated marking enables consistent, objective assessment for large groups Rigorous evaluation of questions ensures overall consistency and standards.

Decrease marking Potential time savings: dependent on level of implementation must be offset against investment in design may represent a re-distribution of workload may demand a cultural shift

Administrative efficiency Automatic transfer of data to MIS, or SRS Examination papers and handling Speed of results

Objective Tests Examples of question types Guidelines for writing items Assessing skills and abilities Feedback and scoring

Objective Tests Require students to provide a response to a question with a pre-determined answer: select a solution from a set of choices identify an object or position supply brief numeric or text responses

Question Types Multiple choice Multiple response Assertion-reason Graphical hotspot Text/numerical Matching/ranking Sore finger questions Sequencing Assembly Graphing

Designing questions I Questions single, definitive statement avoid unnecessary or irrelevant material use clear language use negatives sparingly put as much of the question in the stem do not use double negatives

Designing questions II Distractors ensure there is only one correct answer use plausible distractors avoid clues to the correct answer avoid all of the above and none of the above base distractors on common student errors avoid ambiguity

Blooms Taxonomy knowledge, comprehension, application, analysis, synthesis and evaluation Example questions

Feedback Objective tests allow provision of : timely, specific formative feedback to students fast and detailed feedback to staff Enables a consistent level of feedback to large student groups Can focus and direct learning through anticipated problem areas Enables curriculum design modification during a course

Scoring One score per correct answer Issue of guessing Negative marking

Discrimination and facility Discrimination power of an individual question to differentiate between weaker and better students Facility measures the difficulty of a question by dividing the number of correct responses by the total number of responses.

Integration: assessment methods & strategies Appropriateness and balance Quality of feedback and student learning Extending the skills and knowledge assessed

Test structure II Analyse material content difficulty assessment type question style Identify areas of low coverage and uneven spread

CAA Website CAA Centre

CAA in business studies Introduction to financial accounting Intermediate/advanced financial accounting Principles of finance Production and operations management Professional certificate in management

Advantages of CAA- business studies respondents Speed Remote access Anonymity Neutrality Quick feedback Wide course coverage

Disadvantages of CAA-business studies respondents Inflation of students' marks Difficult to set appropriate questions Steep learning curve/time investment Fails to develop students' argumentation skills

CAA Website CAA Centre