1. Assessment of Professionals M. Schürch

2. How do you assess performance? How do you currently assess the performance of your residents? What standards do you apply? What evidence can you provide? Is the evidence robust enough? Could the resident challenge your assessment?

3. Outline Principles of assessment Methods of assessment What standards do apply Assessment in the new curriculum Conclusion

4. Why assess? Provide direction and motivation for future learning: assessment drives learning … Screening out those who are incompetent Meet public expectations Choose applicants for advanced training

5. What to assess? Knowledge Skills –technical –non-technical Attitude

6. How to assess? Here is the problem Depends on why and what!

7. Formative vs summative assessment Formative assessment –guides future learning –provides reassurance –promotes reflection –shapes values Summative assessment –overall jugement of competence –fitness to practice –qualification for advancement can: reinforce intrinsic motivation inspire to set higher standards

8. Formative vs summative assessment Formative assessment –guides future learning –provides reassurance –promotes reflection –shapes values Summative assessment –overall jugement of competence –fitness to practice –qualification for advancement can: reinforce intrinsic motivation inspire to set higher standards

9. Methods of assessment Simulation –standardized patients –computer based simulation computer programms, high fidelty simulators Work-based assessment –case based discussion –mini-clinical evaluation exercise (mCEX) –direct observation of … (DOPS, DOCE) –multisource (360°) assessments

10. Hierarchy of professionalism Competent Knowledge Technical Skills Non-Technical Skills Pro- fessional Behaviour MCQ VIVA OSCE VIVA MCQ: multiple choice questions OSCE: objective structured clinical examination VIVA: oral exam

11. Hierarchy of professionalism Competent Knowledge Technical Skills Non-Technical Skills Pro- fessional Behaviour mini- CEX CBD DOCE MSF (RET?) mini-CEX: clinical evaluation exercise CBD: case based discussion DOCE: direct observation of clinical encounter MSF: multi source feedback

12. Criteria to choose methods Reliability accurate, reproducible Validity measures what it claims to Impact on future learning and practice Acceptability to learners and faculty fair, feasible Costs trainee, institution, society

13. What standards do apply? Limen referenced –Written exam, MCQ –Standardized patients, simulation Relying on assessors judgement of competency against expectation at a particular level of training –Workplace based assessment Patient difficulty Faculty stringency Equivalence of assessments?

14. Challanges of work based methods No absolute standards –most trainees will be satisfactory Alternative assessments for those who are unsatisfactory Importance of selection of assessors Equivalence –not useful for ranking Trainees  Multiple assessments by multiple assessors

15. Who should assess? „ … one of the capabilities of a competent individual is the skill of recognising competence in others.“ Ronald Burnett: The Limits of Competence 1991

16. Assessment in the new Curriculum www.sgar-ssar.ch

17. defines competencies and standards

19. Assessments in the curriculum Formative Summative

20. Formative assessments

21. DOCE

25. RET (Resident Evaluation Tool) the Resident Evaluation Tool RET (Anästhesie Basel, Abbott) can be used to give Multisource-Feedback RET combines items of mini-CEX and ANTS (non technical Skills)

27. Example of a rating, that needs an action plan

28. Example of an excellent rating

29. Conclusion Assessment is essential for learning Good assessment is a challenge

30. „ … one of the capabilities of a competent individual is the skill of recognising competence in others.“ Ronald Burnett, 1991