1. Confidence-Based Assessment of Two-Alternative Format Tests
AHMED A. BELAL (1) & DIALA F.AMMAR (2)
(1)Computer Engineering and Informatics- Beirut Arab
University, Lebanon
(2)Psychology department, Lebanese American University,
Lebanon

2. Computer Aided Assessment
Multiple Choice Exams :
No Partial Credit
Penalty for Wrong Answers

3. Designing good distracters
Multiple Select Exams
Penalty for Wrong Answers
Multiple T/F Exams
Question may have no correct answers
No partial credit
Individual T/F questions
Weak resistance to guessing

4. Correct =+ X
No Answer = 0
Wrong Answer= -Y
Poor discrimination
Confidence-based Assessment

5. Uncertainty improves Discrimination
The Element Of
Uncertainty improves Discrimination
Three Questions
+10
Correct 1 2 3
Grade 10 20 30
Correct 1 2 3
Grade
-15 15 30
Scaled 45
+10 -5
Correct 1 2 3
Grade
-30
-10 10 30
Scaled 20 40 60
+10
-10

6. Adding an element of uncertainty
Correct
+10
Wrong -5
No answer
Correct 1 2 3
Wrong
No answer
9 Different Grades
Grade
-15 15 30
-10 5 20 10 -5
Scaled 45 35 25
Correct
+10
Wrong
-10
No answer
Grade
-30
-10 10 30
-20 20
Scaled 40 60 50
7 Different Grades

7. Incorporating uncertainty in the test
Confidence Level
Correct Answer
Wrong Answer 1 +3 -1 2 +4 -2 3 +5 -5
Leclereq [1,2]
Hard to correctly estimate one’s confidence level
Humans are more oriented to judge their uncertainty in things relative to each other

8. Relative Uncertainty
Student ranks questions relative to each other.
Value of correct answer decreases as level of confidence decreases.
Wrong answers change the confidence level.
Use a step size and a reduction function to assign values for a correct answer.

9. Relative Uncertainty : An Example
Answer Vector
000
001
010
011
100
101
110
111
R=5 5 10 15 20 30
R=10
-10/0
R=15
-20/0
-5/0
S=1

10. Relative Uncertainty (Cont)
Confidence Level C1 C2 C3 C4 C5
Reduction Value 2 10 8 6 4 2
Reduction Value 5 5 -5
-10
10 Questions : answer vector
Step size = 1
R =2
Score = = 36
R = 5
Score = = 15
Step size = 2
R =2
Score = = 46
R = 5
score = = 40

11. Answering Form

12. Absolute
(10-2p)
(10-p)
(10-5p)
R=1;s=2
R=2;S=2
Answer Vector 16 24 32 10 37 34 38 44 47 35 50 25 40 15 56 52 30 62 64 72 76 80 90 66 73 45 42 46 18 36 20 39 51 60 86 78 84 65 58 67 55 70 49 22 5 41 27 1 26 29 28 4 54 68 61
Table 1

13. Strategy
Average
Standard Deviation
Absolute
10-2P
10-P
51.75
10-5P
R=1 S=2
R=2 S=2
Table 2

18. Thank you