1. Improving Resident Education in Quality Improvement and Patient Safety
Ramon Cancino, MD
STFM 04/30/2011

2. Objectives
Discuss research in residency education in the field of quality improvement and patient safety (QI/PS)
Discuss tools available to install an educational program to teach residents about QI/PS
Present findings of an IRB-approved study using an online curriculum to teach family medicine residents the principals of QI/PS
Discuss methods of how to further evolve this field and establish best practices

3. Background Quality Improvement (QI)
Formal approach to performance analysis and systematic efforts to improve it

4. History
In 1999, the Institute of Medicine (IOM) found as many as 98,000 people die each year from medical errors that occur in hospitals.
Medical error: the 5th leading cause of death in the United States.
The IOM, in 1999, recommended that performance standards and expectations for heath care organizations should focus greater attention on patient safety (Kohn LT 1999).
The IOM created a new awareness that the status quo was no longer acceptable.
Crossing the Quality Chasm: A New Health System for the 21st Century (2001) outlined fundamental changes that must be made in order to improve care.

5. Health Care Must Be:
Safe
Effective
Patient-centered
Timely
Efficient
Equitable

6. Background

7. Recent studies into QI/PS curricula have all shown benefit.
Motivation
Residents involved in QI training strongly believe they gained important skills.
(Voss et al. 2008)
Recent studies into QI/PS curricula have all shown benefit.
(Canal, Torbeck et al. 2007), (Criley, Keiner et al. 2008), (Oyler, Vinci et al. 2008)
A web-based interactive educational program led to significant improvement in cardiac examination competence and was better than clinical training alone.
(Criley, Keiner et al. 2008)
systems thinking and human factors analysis
root cause analysis
process mapping
minimal hospitalist involvement

8. Question
Using an established, widely-used, online curriculum which focuses specifically on QI/PS, this study:
Investigated when QI/PS education is beneficial to the family physician in-training.
Assessed family medicine resident opinions, views, and experiences in the area of QI/PS. 

9. Our Aim:
Determine whether first year residents have less knowledge of QI/PS than second and third year residents.
Determine if training increased resident knowledge of QI/PS
Determine whether upper level residents retained knowledge of QI/PS better than first year residents after six months.

10. Materials
Recruitment Letter
Pre- and Post-test Questionnaires
Institute for Healthcare Improvement Open School website
Family Medicine Residents

11. Materials

12. Methods
Online lessons were chosen based on their relevance to both early and late residency training.
Fundamentals of Improvement
The Model for Improvement: Your Engine for Change
Introduction to Patient Safety
Fundamentals of Patient Safety

13. Methods QI 101: Fundamentals of Improvement
Lesson 1: Errors Can Happen Anywhere—and to Anyone
Lesson 2: Health Care Today
Lesson 3: The Institute of Medicine's Aims for Improvement
Lesson 4: How to Get from Here to There: Changing Systems
QI 102: The Model for Improvement: Your Engine for Change
Lesson 1: An Overview of the Model for Improvement
Lesson 2: Setting an Aim
Lesson 3: Measuring
Lesson 4: Developing Changes
Lesson 5: Testing Changes
PS 100: Introduction to Patient Safety
Lesson 1: Understanding Medical Error and Patient Safety
Lesson 2: Understanding Unsafe Acts
Lesson 3: A Call to Action — What YOU Can Do
PS 101: Fundamentals of Patient Safety
Lesson 1: To Err Is Human
Lesson 2: Responding to Error
Lesson 3: Identifying and Reporting Errors
Lesson 4: Error versus Harm
Methods

14. Methods Pre-Test Questionnaires
Recruitment letter via electronic mail, which asked for consent to enter this voluntary study.
Pre-Test Questionnaires
prior training in QI/PS
opinion about the area of study
opinion in regards to its importance in their current/future careers.
Instructions to register with the Institute for Healthcare Improvement Open School in order to access an online curriculum
Online curriculum and testing
Post-Test Questionnaires

15. Methods Data was collected by the principal investigators.
Data was organized using Microsoft Excel.
Analysis was done by Mayo Clinic Biostatistics division.

16. Results Total = 76 residents Pre-Test Questionnaire: 76
Online Curriculum: 34
Post-Test Questionnaire: 33

17. Results 6 Institutions Various Community settings
Various Institution Type

18. Pre-Test Results

19. Pre-Test Results

20. Pre –Test Results Residents who have had formal training in QI/PS:

21. Pre-Test Results

22. Pre-Test Results

23. Pre-Test Results
Important Issue?
p = 0.007

24. Pre-Test Results
86 % believe a formal curriculum in QI/PS belongs in medical education
88% believe a formal curriculum in QI/PS belongs in residency education

25. Average Testing Results (%)
95.8
QI 101: Fundamentals of Improvement
92.3
QI 102: The Model for Improvement: Your Engine for Change
93.0
PS 100: Introduction to Patient Safety
96.3
PS 101: Fundamentals of Patient Safety

26. PGY1 PGY2 PGY3 94% 96% 95% 100% Average Scores
Fundamentals of Improvement
94%
96%
Model for Improvement
Introduction to Patient Safety
Fundamentals of Patient Safety
95%
100%

27. Post-Test Results
How strongly do you believe that issues of QI/PS are important to patient care?

28. Post-Test Results

29. Post –Test Results
Level of Difficulty per Training Level
p = 0.02

30. Post –Test Results
p = 0.008

31. Post –Test Results
If you were the chair of the QI/PS committee of the department of family practice, which online courses would you require incoming residents to take?

32. Post –Test Results
85% of residents feel family medicine clinic and hospital staff may benefit from taking an online curriculum in QI/PS

33. Discussion
Residents enter training feeling ill- equipped to handle issues of QI/PS.
At the time of questioning, many residents were not yet thoroughly educated on the topic.
Residents feel QI/PS education belong in both medical and residency training.

34. Discussion An online curriculum in QI/PS was generally well-received.
Residents gained an appreciation for the importance of this issue.
Residents feel all staff members would benefit from this sort of education.

35. Limitations Number of participants Retention of participants
Not double-blind study
Testing administration

36. Future Study
6 months after the start of the residency year, a post-test, which contained questions drawn from the online curriculum, will be administered.
This portion of the study will also be optional

37. Future Study Quality Improvement activities amongst graduates
Patient safety-related training
Use of technology in resident education

38. In Summary:
Upon entering residency, trainees feel ill- equipped in knowledge of QI/PS
Online training and testing modules were well-received
Increase in perceived importance of QI/PS
All hospital/clinic staff may benefit from this sort of training
Increase in this fund of knowledge is best practice for our patients.

39. Thank You Drs. Pantin, Bachman, Willis. Participating sites:
Dr. Brian Arndt at Wisconsin University Dr. Grace Chen Yu at Stanford University Dr. Steve Wolfe at Forbes Family Medicine Residency Program
Dr. Janet Albers at Southern Illinois University
Dr. Robert Bonacci at Mayo Clinic Rochester
Dr. Walter Taylor at Mayo Clinic Jacksonville
Ramon S. Cancino, MD
Mayo Clinic Florida
4500 San Pablo
Jacksonville, FL 32224

40. Resources
Canal, D. F., L. Torbeck, et al. (2007). "Practice-based learning and improvement: a curriculum in continuous quality improvement for surgery residents." Archives of Surgery 142(5): ; discussion
Criley, J. M., J. Keiner, et al. (2008). "Innovative web-based multimedia curriculum improves cardiac examination competency of residents." Journal of Hospital Medicine 3(2):
Kohn LT, C. J., Donaldson MS, eds (1999). To Err Is Human: Building a Safer Health System. J. M. C. Linda T. Kohn, and Molla S. and Donaldson. Washington, DC.
Oyler, J., L. Vinci, et al. (2008). "Teaching internal medicine residents quality improvement techniques using the ABIM's practice improvement modules." Journal of General Internal Medicine 23(7):
Richardson, W. C. B., Donald M.; et al (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. I. o. M. Committee on Quality of Health Care in America. Washington, DC.
Voss, J. D., N. B. May, et al. (2008). "Changing conversations: teaching safety and quality in residency training." Academic Medicine 83(11):