Computerised Assessment in Mathematics : The Human Side CDC Steele, PJW Bolton, WR Lionheart. School of Mathematics University of Manchester

Computerised Assessment Computer asks question Student responds to question Computer processes response of student Provides (one or more of) –A mark –Feedback –Further Questions

Computerised Assessment in Mathematics Why is this different ? Assessment software can include computer algebra system. Can ask more sophisticated questions.

Imagine the question Which of the following answers would be accepted ? All are correct. Text match approach not useful. Need something more sophisticated.

The STACK system Developed at university of Birmingham by Dr C Sangwin et al. Works in conjunction with VLE Moodle

The STACK system Some property of students answer compared with property of ‘stored’ answer. Will identify correct and incorrect aspects.

Advantages of STACK Analyses a mathematical expression Can give feedback specifically generated by form of answer. Can vary coefficients in a sensible and related manner.

STACK and students Students new to the system Entering form of answer x*y issues functions, brackets and cases Many questions at first but then general understanding.

STACK and students Students practice and assessment tests System geared to repeated practice so practice tests arranged alongside assessment versions. 1) Practice the mathematics 2) Get used to the entry of similar functions.

STACK and students Students practice and assessment tests Formerly, practice attempts did not count. Many students would look briefly at practice questions, assume familiarity and head straight to assessment questions, subsequent problems with both subject matter and entry method on assessment questions.

STACK and students Students practice and assessment tests More recently, best attempt at practice questions had equal weighting with assessment questions. Now students paying much more attention to practice questions.

STACK and students Worked solution. Formerly time when students could ask for worked solution at same time as submitting answers. Had effect of providing worked solution but no marks available for question.

STACK and students Worked solution. Now worked solutions not available at this stage. Can be asked for later.

STACK and students Second and subsequent attempts. Can allow second and subsequent attempts at a penalty. Great difficulty in getting this information over to students.

Occasional Lapses Question (about Fourier Coefficients) asking for an answer in the form Clearly, another correct input is Normally system would accept this but on occasion did not. Necessary to give specific mention on decision tree to this form of input.

STACK and students Any minor errors. Magnified by students. Discussions, rumour etc.

A recent problem One student found a way of inducing error from the system. Displayed answer and subsequently accepted students inputing this answer ! At the same time, system running slowly, very error-prone etc.

Crossroads on Computerised Assessment Route 1) Ask ambitious questions using software such as STACK, accepting mathematical input, giving feedback based on student answer, having sensible relation between coefficients, possibly risking fragile system. Route 2) Ask more basic questions e.g. multiple-choice or numerical questions, not being able to give detailed feedback etc.