1. Wayne State University
Tool Time: Creating a template for critical appraisal of your clerkship
Jira Coumarbatch MD
Juliann Binienda PhD
Wayne State University
School of Medicine

2. Introduction Dr. Coumarbatch Dr. Binienda
Asst. Clinical Professor WSUDFMPHS
Clerkship Director (2.5 yrs)
Dr. Binienda
Asst. Professor WSUDFMPHS
Beh Scientist/Predoc Educator

3. Outline Goals/Objectives Background (Why we did a clerkship review)
Developing the tool (how we did it)
Demonstrate critical review
Outcomes
Questions, comments and suggestions

4. Session Goal/Objectives
By the end of the session, the participants will be able to conduct an appraisal applicable to their own particular clerkship.
Objectives:
Given a rationale for the important elements of a critical
appraisal, participants will be able to list aspects necessary for a
comprehensive clerkship review.
Given a template for clerkship appraisal, the participants will be able to know methodology for all aspects of a thorough review.
Given input from others in attendance, participants will be able to compare and contrast clerkship review methods.
Goal – how to conduct a clerkship review
Objectives – discuss elements, discuss template, feedback

5. Background WSUSOM Clerkship 300 students per class
Largest single campus SOM
Urban setting
Clerkship
4 weeks
over 100 sites (urban, suburban, rural)
all volunteer faculty
25 students per month/one student per site
Except for WSU FACULTY, ABOUT 8

6. Background (Why?) Rationale for Clerkship Review
– New clerkship director
Department Chair’s request
Improve effectiveness
LCME
STFM PDDI project
Alliance for Clinical Education (ACE) recommendations (Fincher et al, 2005)
I became CD 2 yrs ago, no training, no experience
2. Chair wanted a review of clerkship goals/objectives.
3. What need to be better from students, preceptors, evaluations, guidelines, nat recommendation perspective
4. What works, how well
5. Wsu STARTED PREPARING For LCME. were we within their requirements for a clerkship?
6. I signed up as a PDDI fellow in Desigened to train new predoc directors and clerkship dierctors. Excellent, STFM, mentors
7. multidisciplinary group formed in 1992 to enhance clinical instruction of medical students. The Clerkship Directors guidebook by the Aliance for Clinical Education recommends a yearly review. Excellent reference for all aspects of a clerkship. Section dedicated to reviewing clerkships.

7. Background (How?) Type of evaluation
Internal to clerkship
Internal to institution
External to institution
Type of Information to be collected
Paper vs. electronic data collection
Time
Personnel
Plan ahead
We decided tthat we wanted to do an evaluation internal to the clerkship.
2. program evaluation internal to the clerkship (i.e., data collected on multiple aspects of the clerkship, including didactic and clinical experiences)
3. program evaluation internal to the organization (i.e., data across clerkships within an institution)
4. program evaluation external to the organization, which can be useful for benchmarking purposes. 
We needed to collect data. We decided to use a combination of data collection methods: paper, electronic, BB, Campus
. Time is a big factor. Make sure you set aside scheduled time. I did 2-4 hrs per wk9.
10. Man power is very important. You cant do it alone. I needed Dr. Binienda (review data) and our coordinator (entered data)

8. Background Continuous Quality Improvement (CQI) and PDSA Models (ACE)
Plan – decide
Do – collect
Study - evaluate
Act - implement
We incorporated a process for CQI based on the Plan, Do, Study, Act model (ACE)
The first 2 steps invole the actual clerkship activities. The review begins at the Study phase and then it becomes cyclical from year to year.
2. Plan a change or a test aimed at improving quality.
3. Do or carry out the change/test (preferably on a small scale)
4. Study the results or the effects of the change by collecting and analyzing relevant data.
5. Then we Acted on what has been learned by adopting or abandoning a change/test or deciding to run through the cycle again.
6.  

9. Background (What?) Recommended elements for clerkship review
   - student evaluations
- student logs
- clinical evaluations
- exam grades
- internal, local, national data
1. ACE recommends a thorough/objective review of all aspects of your clerkship and has some recommendations, but also you want to customize
Student ratings of specific elements and global scores
Review students’ logs of clinical encounters
Assess students’ performance/competence 
Employ a comprehensive approach
Collect and review data internal to the clerkship
Review cross-clerkship data within your institution
Review cross-clerkship data between institutions (national data)
Systematically organize and review data that promotes continuous quality improvement

10. Important Considerations
Create a matrix of important categories
Important to individualize your tool
Must be critical
Depth and breadth
Remain committed
First template is the hardest
1. Its good to use a tool to help organize, guide, simplify the review

11. Tool Time Review the tool Available on CD
For more information contact:
Jira Coumarbatch MD
Clerkship goals and objectives
What were we going to use to evaluate the goals and objectives?
WSUSOM goals /obj important
LCME important (SAA)
STFM/FFM (SAA)
Preceptor evals of students performance
Students eval of experience
NBME scores
CQI