1 Assessing Outcomes After Theyre Gone – Measuring Preparedness and Quality in Practice Presented by: Mary Pat Wohlford-Wessels, Ph.D. Vice President for Institutional Research and Effectiveness, KCUMB Diane Hills, Ph.D. Associate Dean, Academic Affairs, DMU

2 Background For years, accrediting agencies have stressed the importance of measuring and using information regarding student performance to drive curricular change. The ultimate outcome of a medical school rests with its graduates ability to secure a residency and perform competitively against allopathic and osteopathic peers.

3 Objectives This presentation will: focus on how DMU-COM moved beyond assessing student performance and began soliciting feedback about the curriculum from graduates and residency directors. describe the methods used to solicit feedback and state how the results were utilized in curriculum change

4 Two Components The process of evaluating graduate preparedness included two components: one focused on residency directors perceptions of the quality of DMU-COM graduates and the other focused on graduates perceptions of preparedness

5 Residency Directors Perceptions of DMU- COM graduates An instrument was designed after reviewing instruments utilized by three allopathic medical schools A database of residency locations for graduates from the class of 2004 & 2005 was obtained (surveys sent in 2005 & 2006). Letters and surveys were sent to residency directors who were asked to assess specific graduates.

6 Instrument The instrument included questions that could be answered using a 5 point Likert scale in addition to open ended questions. Residency directors were asked to respond to questions related to: graduate performance graduate professional characteristics & the value of the Deans Letter

7 Residency Directors Results There was little difference in results 2004 vs Lowest mean scores Ability to apply basic science knowledge Ability to understand personal strengths and weaknesses Use of evidence based medicine Highest mean scores Ability to communicate effectively Ability to provide sensitive, empathic care to patients of varying cultural, racial and socioeconomic backgrounds Ability to maintain professional, respectful, and honest relationships with patients, their families and colleagues

8 Residency Directors Results When asked if they would renew the residents contract – 99% stated yes Helpfulness of the Deans letter in making a decision about the applicant: 90% stated the letter was helpful 87% felt the letter accurately portrayed the applicant 11% felt the applicants abilities were overstated and 1% felt the letter underestimated the applicants true abilities.

9 Graduates Perceptions of Preparedness An instrument was designed after reviewing instruments from three allopathic medical schools. Lists of graduates along with mailing addresses were obtained – 2004 & Letters and surveys were sent to graduates

10 Instrument The instrument included questions that could be answered using a 5 point Likert scale The survey asked graduates to indicate how well the DMU-COM curriculum prepared them for residency

11 Results In general, graduates from the class of 2004 felt more prepared for residency Lowest mean scores Skills and practices in performing routine procedures for patients Changing social, cultural, legal, political and economic trends that can effect health care delivery Knowledge and application of the behavioral sciences to medicine Preparation in dealing with stress

12 Results Highest mean scores Knowledge of basic science concepts Effective skills to take a patient history, perform a physical examination, diagnose common diseases, and make appropriate clinical decisions Knowledge of professional behavior in doctor - patient interactions Knowledge, attitudes and skills necessary to provide continuing, comprehensive, and preventive longitudinal care to individuals and families

13 Conclusion Soliciting feedback from Residency Directors and Graduates is vital to ensuring an effective outcomes program (and relevant curriculum) Perceived curricular strengths and weaknesses need to be checked against graduates experiences in practice

14 Research Colleagues Thanks to: Dr. Kendall Reed Dr. Tom Mueller Dr. Dana Shaffer Dr. Dave Plundo

15 Questions?