2. Workplace assessment
Dr. Kieran Walsh,
Editor,
BMJ Learning. 2

3. Workplace assessment – pluses
Lots of clinical material available
Skilled clinicians/teachers available
Strong educational impact 3

4. Workplace assessment – potential difficulties
Broad range of problems/patients/procedures required
Integrated skills
Potential to practice
Specific assessment experience/expertise is often rare
Small number of trainees in each discipline in an average sized hospital 4

5. Workplace assessment – theory
Knows
Knows how
Shows how
Does 5

6. Workplace assessment – judgement based on….
Single encounter
Actually observed performance
Case specific
Routine performance
Observation over a period on different occasions on different problems
Reduces problems with case specificity 6

7. Workplace assessment – nature of judgement
Did it happen?
Checklist approach
Objective
Structured
Consistent
But not very discriminatory 7

8. Workplace assessment – nature of judgement
Was the trainee any good?
Global rating
More valid 8

9. Workplace assessment – nature of judgement
Was the trainee good enough for a
particular purpose? 9

10. Workplace assessment – influence on learning
Formative assessment and feedback
Direct observation (in the past many trainees never observed in a patient encounter) 10

11. Workplace assessment – faculty development
Workshops needed
Training in behavioural observation
Performance dimension training
Frame of reference training 11

12. Workplace assessment – reliability
Number of encounters observed
Number of assessors
Aspects of performance being assessed 12

13. Workplace assessment – is it a fair test?
Different patients
Different assessors
Faculty development
Common problem list
Number of assessors for each trainee 13

14. Workplace assessment – stakes
Assessments influenced by what is at stake
Colleagues reluctant to make negative assessment when stakes are high
BUT
External examiners
Faculty development
Anonymity in peer assessment
Should trainees choose their assessor? 14

15. Workplace assessment – costs
Planning
Time of trainees and assessors
Real patients
Training of both groups
Administration
Material
Collating and processing results
Training
Cost effective?? If used for correct purpose 15

16. Workplace assessment – CbD
Case based discussion
Trainee selects case studies, assessor chooses one of these
Assessor should ask why this selection
Assesses quality of … diagnosis, assessment, management, record-keeping
15 minutes (5 minutes for feedback) 16

17. Workplace assessment – MiniPAT
Peer assessment tool
8 assessors
Including senior colleagues, nurses, AHPs
Including self assessment
Routine performance
Feedback reviewed with trainee and supervisor
Agreed action plan 17

18. Workplace assessment – MiniCEX
Faculty member observes a trainee interacting with a patient in a clinical setting
15 minutes (+5 minutes for feedback)
Interviewing skills, physical exam, professionalism, communication skills
Formative feedback 18

19. Workplace assessment – MiniCEX
Not for high stakes exams
Not to rank trainees
Not to compare training programmes
4-6 times in the year
Different assessor each time
Encounter chosen by the trainee, confirmed by the assessor 19

20. Workplace assessment – DOPS
Direct observation of procedural skills
Real patients
15 minutes (+ 5 minutes feedback)
Trainees select from an approved list of procedures
Indications, communication, technique, analgesia, asepsis 20

21. Workplace assessment – DOPS
6 times in the year
Valid
Global rating scales 21

22. Workplace assessment – portfolios
Different types of information
Summary of educational experiences
Reviewed by an assessor
Reflection
Action plan
Purpose must be clear (e.g. for summative use)
Need be specific about what portfolio should contain 22

23. Workplace assessment – screening
If you do really well on your first 2 MiniCEX’s, how will you get on in your next 4? 23

24. Workplace assessment – checklists vs global ratings?
Scores are broadly correlated
Checklists – objective, reliable, lower level of expertise
Global ratings – subjective, more valid
Only experts can use global ratings
Anyone can be trained to use a checklist 24

25. Workplace assessment – references and further reading
Norcini JJ. Workplace assessment. In Understanding Medical Education:
Evidence, Theory and Practice. ASME 2010
Miller A, Archer J. Impact of workplace based assessment on doctors' education
and performance: a systematic review. BMJ Sep 24;341
Van Tartwijk J, Driessen EW. Portfolios for assessment and learning: AMEE
Guide no. 45. Med Teach Sep;31(9):
Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE
Guide No. 31. Med Teach Nov;29(9):
Wilkinson TJ. Assessment of clinical performance: gathering evidence. Intern
Med J Sep;37(9):631-6. 25