2. CAA MUST be more than multiple-choice tests for it to be academically credible? Phil Davies School of Computing University of Glamorgan

3. What is CAA? Multiple Choice tests Formative? Learn?? Summative Objective Testing / Higher level skills? / Gender Issues Employable Skills? Who gains …… student or staff Academically acceptable? BATTLE

4. Cynical Example of MCQ What colour is the sky? A Don’t know.. Give me a clue –Blue / Green / Brown –Blue B Blue C Sometimes it is blue, red or grey, depending upon weather, time, season,etc. Who gets the job? Process?? Expected Answer

5. How to make CAA acceptable Subjective and Objective Testing Integration of methods End product more important than individual process results OBJECTIVE = MCQ SUBJECTIVE = ESSAY (oh no more marking!!!)

6. OLAL

7. ESSAYS Time consuming to mark Subjective (Lecturer).. Opinionated Should we assess grammar –Foreign Students –Pre-Requisite –Learning Outcomes FEEDBACK … WHAT FEEDBACK???

8. Computerised Assessment (by Peers) or (with Plagiarism) C.A.P. SYSTEM Computerised Assessment of essays Peer Assessed Embedded Web Referencing Marked Properly!!! Copious Comments (Destructive (honest?))

12. Success / Failure Major benefits –style / feedback / objectivity through subjectivity Problems –Wide range of marks / conflicting comments / why bother marking Marks for MARKING (how?)

13. Integration Produce essay Do multiple choice A Perform peer assessment Get feedback Perform multiple choice B

16. Compare Report and MCQ Average reports = 62% St dev = 8 No student less than 40% overall Average MCQ1 = 70% St dev = 14 Average MCQ2 = 72% Improvement Report to MCQ1 class average +8% achieved report <50% … improve +24% Group MCQ2 = +6% Weaker students benefit from MCQ & CAP

17. Student Comments about peer marking Learn from marking the work of others Peer work acted as marker to own ability “they said they couldn’t read my report, I could” “you realise you’re not as thick as you thought you were” 30% took more care because it was to be marked “properly”

18. Student Opinion Plagiarism OK.. Sentence or two I’m foreign so my English is not so good why waste time rewording when I could be learning 75% put in more effort with report because they were to be further tested via MCQ “writing a report is just rephrasing web sources, the tests checked understanding”

19. Conclusions EXPLAINATIVE not Formative Peer assessment must be shown to be beneficial to the students Quality of Assessment …..Marking must be judged (higher level skill?) MCQ.. Part of the process.. Not Process Acceptance … Integrative Process

20. CAA MUST be more than multiple-choice tests for it to be academically credible? Trying to make it acceptable by “types of questions” / “formatting of questions” / so called “deeper learning” => Complexity Integration of Subjective and Objective Testing may provide credibility