1. Anders Jönsson (LUT) & Nikos Mattheos (OD) Malmö University Dynamic assessment and Interactive Examination

2. How can professional-directed education foster reflecting and self-assessing practitioners? Jönsson, A. & Mattheos, N. Malmö University The students must be given the opportunity to practice these skills, as well as be assessed on them.

3. - a structured assessment methodology - aims to evaluate students content specific skills in parallel to their metacognitive Jönsson, A. & Mattheos, N. Malmö University The Interactive Examination

4. 1. Describe the methodology, 2. Present some results from last years cohorts, 3. Highlight some interesting differences between the two education centres. Jönsson, A. & Mattheos, N. Malmö University This presentation will:

5. teacherdoctor Autumn 2004: 34 3rd semester dental students 174 1st semester student teachers

6. The Interactive Examination 6 steps Self-assessmentPersonal taskComparison task EvaluationAssessmentFeedback Jönsson, A. & Mattheos, N. Malmö University Method 6 steps

7. 1. Self-assessment Likert-like questions from 1 (poor) to 6 (excellent) The Interactive Examination Self-assessmentPersonal taskComparison task EvaluationAssessmentFeedback - 11 competencies -match to instructors judgement -13 competencies - match to examination results Method step 1

8. The Interactive Examination Self-assessmentPersonal taskComparison task EvaluationAssessmentFeedback 2. Personal task An authentic problem from professional life -A patient walks is your clinic and... Internet based case presentations Actual Classroom scenarios A series of short Internet video films Method step 2

9. 3. Comparison task a. identify differences between own and expert answer, b. reflect on the reasons for these differences, c. define own needs for further learning. The Interactive Examination Self-assessmentPersonal taskComparison task EvaluationAssessmentFeedback Method step 3

10. The Interactive Examination Self-assessmentPersonal taskComparison task EvaluationAssessmentFeedback 4. Evaluation Asking the students to evaluate the whole experience, through a standardised form Standardised form – 10 fields 8 identical – 2comparable Method step 4

11. 5. Assessment of students - personal task, - comparison task. The Interactive Examination Self-assessmentPersonal taskComparison task EvaluationAssessmentFeedback Method step 5

12. 6. Personalized feedback Comments on: - students self-assessment -the personal task –the comparison document. The Interactive Examination Self-assessmentPersonal taskComparison task EvaluationAssessmentFeedback Method step 6

13. Results Evaluation Results from the Likert-like questions that were identical in both centres: Median score OD Median score TE Learning experience 86 Clear expectations 65 Chance to show what you know 76 Help to prepare for work 66 Difficulty 66 Results Evaluation

14. Results Self- assessment Students as a group: Self- assessment OD (n = 34) TE (n = 166) In agreement38%16% Higher38%72% Lower24%12% Jönsson, A. & Mattheos, N. Malmö University Results Self- assessment

15. Results Self- assessment Individual students: Self- assessment OD (n = 34) TE (n = 166) Significantly higher (p<0.05) 35%78% Significantly lower (p<0.05) 17%- Jönsson, A. & Mattheos, N. Malmö University Results Self- assessment

16. Results Self- assessment Regression analysis: Self- assessment OD Self- assessment TE Gender no relation Group no relation Success on exam no relation eta = 0.50 (p<0.001) Clinical instructor no relation- Jönsson, A. & Mattheos, N. Malmö University Results Self- assessment

17. Jönsson, A. & Mattheos, N. Malmö University Qualitative analysis: comparison document -differences -arguments -learning needs, which and how? Results qualitative analysis

18. Jönsson, A. & Mattheos, N. Malmö University differences in: - form - content - attitudes - interpretation Qualitative analysis: comparison document Results qualitative analysis

19. - differences..! - mainly content - few attitude - few form - hardly any interpretation Learning objectives: mainly content driven need to know more... - similarities..! - mainly attitude - some content - some interpretation - few form Learning objectives: need more in-service training Results qualitative analysis

20. Jönsson, A. & Mattheos, N. Malmö University Results Comparison task Reflecting... - differences in the nature of the assessed task? -differences in the institutional learning culture? -differences in students perception of the end product?

21. Future research... -longitudinal observation of self-assessment skills? self-assessment skills? -Evaluation of interventions? -investigate students perception of the end product? Jönsson, A. & Mattheos, N. Malmö University

22. Jönsson, A. & Mattheos, N. Malmö University

23. Contact: Anders Jönsson anders.jonsson@lut.mah.se Nikos Mattheos nikolaos.mattheos@od.mah.se Dynamic assessment and Interactive Examination