1. FACULTY OF PHARMACEUTICAL MEDICINE
ASSESSMENT WORKSHOP
Sharon McCullough
TRAINEES’ MEETING
Monday 18th May 2009 1

2. Why assess? To find out whether learning has taken place
To diagnose learners needs
To provide public acknowledgement and certification of learning
To carry out selection
To evaluate learning programmes
To motivate and encourage learners
Tummons J. Assessing learning in the lifelong learning sector; 2nd edition 2007.

3. Assessment methods Must be valid Must be reliable
Face validity: what was intended
Content validity: the whole thing
Construct validity: an appropriate method
Predictive validity: future performance
Authenticity: drawing on everyday practice
Must be reliable
Gives consistent results: learners and assessors
Tummons J (2007). Assessing learning in the lifelong learning sector; 2nd edition. Learning Matters.

4. What are we assessing in PMST/HMT?
Performance and competency
Via real life or simulated activities
At appropriate level
Doing it alone / unaided
Doing it as part of a team
Demonstrating understanding of principles

5. PMETB mandatory requirements
Assessment methods chosen on basis of
validity, reliability, feasibility, cost-effectiveness, opportunities for feedback, impact on learning
Rationale for choice of method
will be documented and evidence based
Workplace assessments
must be subject to reliability and validity measures
PMETB (2008). Standards for curricula and assessment systems.

6. Tools for workplace assessment
Clinical medicine
Mini CEX
DOPS
MSF (360o feedback)
CbD
ACAT
Audit assessment
Patient survey
Teaching observation
Pharmaceutical medicine
MSF (360o feedback)
PMAT
PBD
Teaching observation

7. Your task Review one of the three proposed assessment tools
In groups of 3-4
Using tools, supplementary information, scenarios and questions provided
Give feedback
On flip chart paper
One sheet per tool

8. Thank you!

9. The Pilot Study of Assessment Tools
FPM / Educ. Dept. RCP
Local
Study
Coordinators = SSAs
Site 1
Site 2
Site 3
Site 4
Assessors
= Educational
Supervisors
Assessor A2.1
Assessor A2.2
NTN S2.1
NTN S2.2
NTNS2.3
NTNS2.4
NTNs
= Stock of proformas 9

10. Feedback Assessment Workshop
Trainees’ meeting
18th May 2009

11. General comments: Trainees
Heroic performance in crowded conditions!
Similarities between ES/SSA workshops
Specific comments about detail of tools
Important differences
Trainees have a better perspective of the whole of the area to be assessed
ES/SSAs focused on the tool and how to use it

12. ES/SSAs PMAT Trainees Advantages Advantages Disadvantages
Quick
Standardised
Comprehensive
Disadvantages
May not suit all activities
Too many outcomes
Rigid
Advantages
Structured and comprehensive
Gives an overall score
Captures some soft skills
Disadvantages
Not so good for technical knowledge
What does ‘Unable to comment mean?

13. ES/SSAs PBD Trainees Advantages Advantages Disadvantages Disadvantages
Only if allowed to substitute for evidence
This is what we do anyway
Disadvantages
Impossible to validate
Not objective
Duplicates current reflection
What is a project?
Advantages
Global grade
Natural follow on from what we use now
Encourages reflection and helps focus on what to work on
Disadvantages
Few
Lack of structure
Who chooses the project?

14. Teaching observation ES/SSAs Trainees
Advantages
Good checklist
Disadvantages
No overall score
Shouldn’t be the only measure
Duplicates other feedback forms
Advantages
Structured
Transparent
User friendly
Soft skills
Disadvantages
Assessor must be present
Assessor training?
Rigid

15. Thank you!
To everyone who worked so hard in the session