1. 2003IADR Gothenburg Nikos Mattheos Nikos Mattheos Centre for Educational Research and Technology in Oral Health, University of Malmö, Sweden The Interactive Examination Assessing students’ self-assessment ability Assessing students’ self-assessment ability

2. 2003IADR Gothenburg Nikos Mattheos Parallel learning goals: Biomedical knowledge Clinical skills Clinical skills Critical thinking Critical thinking Life-long learning Problem Solving Problem Solving

3. 2003IADR Gothenburg Nikos Mattheos

4. 2003IADR Gothenburg Nikos Mattheos The Interactive Examination: evaluate knowledge, skills and self assessment ability evaluate knowledge, skills and self assessment ability Evaluate process - outcomeEvaluate process - outcome Compare to standards - criteria Compare to standards - criteria Identify needs Identify needs Define new learning objectives Define new learning objectives

5. 2003IADR Gothenburg Nikos Mattheos 1.Self assessment (through Internet: 11 ordinal scales 1-6, text) The Interactive Examination:

6. 2003IADR Gothenburg Nikos Mattheos 2. Essay question (40 min) 3. Oral examination – discussion (70 min) 4. Evaluation (10 min) 1.Self assessment (through Internet: 11 ordinal scales 1-6, text) The Interactive Examination:

7. 2003IADR Gothenburg Nikos Mattheos 5. Compare own essay with expert’s (1 week after) 6. Individual feedback (e-mail) (1 month after) 2. Essay question (40 min) 3. Oral examination – discussion (70 min) 4. Evaluation (10 min) 1.Self assessment (through Internet: 11 ordinal scales 1-6, text) The Interactive Examination:

8. 2003IADR Gothenburg Nikos Mattheos Material and Method: One cohort of 2 nd year students in Periodontology (2001 n=54) - Essay and oral performance grades (1-6) - Self assessment vs instructors judgment - Self assessment and performance / gender - Qualitative Analysis of comparison documents: (differences, arguments, learning needs) - Attitudes towards examination (ordinal scales 1-9, free text)

9. 2003IADR Gothenburg Nikos Mattheos Results: 0,5% 3% 15% 40% 35% 6% 0,5% 40 % of the self assessment scores were in agreement with those of the clinical Instructors. No gender differences 18 5 31 43 % of the students had significant deviation from instructors ( 34% over – 9% under) (2 tailed Wilcoxon’s signed rank test) General: Individual:

10. 2003IADR Gothenburg Nikos Mattheos Results: Nr. 3 (p=0.0002) “Overall diagnosis and treatment of periodontitis” Nr. 4 (p=0.02) “Competence in clinically differentiating healthy from pathological gingival” Nr. 5 (p=0.0006) “Measuring of pocket depth” Nr. 8 (p=0.003) “Competence in evaluating changes in the radiographic image of the periodontium”.

11. 2003IADR Gothenburg Nikos Mattheos Results: 1. How effective do you think is Interactive Examination for learning? not effective 1 2 3 4 5 6 7 8 9 very effective median 8 (n=52) 2. How do you think the Interactive examination would affect your learning style from now? no difference 1 2 3 4 5 6 7 8 9 increase my motivation median 7 (n=52)

12. 2003IADR Gothenburg Nikos Mattheos Results: Students underestimating their competence scored better in the oral part of the exam Female students performed significantly better in both oral and written parts of the exam Students appreciated the element of discussion, but not the written essay part.

13. 2003IADR Gothenburg Nikos Mattheos Results: Results: What did the students compare 1. Form and structure: Length, text diagramme or flow chart, Use of images, language, style etc 2. Content : Additions, emissions, terms etc 3. Attitude: Prioritising, elaborating, depth of detail etc

14. 2003IADR Gothenburg Nikos Mattheos differencesarguments needs

15. 2003IADR Gothenburg Nikos Mattheos Conclusions: 2nd year students in general tended to overestimate their competence Students overestimated their competence in diagnostic rather than treatment skills The interaction element in examination was highly appreaciated Student’s comparison document reflected students’ understanding, prioritising and self-assessment skills through the sequence: differences  arguments  needs

16. 2003IADR Gothenburg Nikos Mattheos http://periodont.mah.se/nikos/research