FAIMER Program Evaluation 3: How is it used? Evaluation as Research Stewart Mennin Bill Burdick

2 FAIMER 2007 HOW PROGRAM EVALUATION INFORMS & IMPROVES CURRICULUM  Faculty & students monitor & improve curriculum  Short-loop feedback ---continuous improvement  Confirms the extent to which curriculum is working  Education developed

3 FAIMER 2007 HOW PROGRAM EVALUATION INFORMS & IMPROVES CURRICULUM  Reveals stresses & strains on the faculty & students  Adjustments made in curriculum  Capture unintended outcomes  Compare student performance with 2 ° data (national, other)

4 FAIMER 2007 HOW PROGRAM EVALUATION INFORMS & IMPROVES CURRICULUM  Collect & analyze long-term measures  Specific issues studied upon request  Produce reports  Research & scholarship in medical education

5 FAIMER 2007 What Is Studied? About Students  Characteristics, demographics 2°  Noncognitive dimensions  Professional identity, specialty choice, location  Cognitive behavior & performance data  The work place (during & after graduation)

6 FAIMER 2007 What Is Studied? About Teachers  Attitudes & experiences with curriculum change and implementation  Educational effort  Perception of Students

7 FAIMER 2007 What Is Studied? About Curriculum  Perceptions, experiences, & attitudes  Integration of basic & clinical sciences  Performance on assessments  Community-based learning  Research projects  Other

8 FAIMER 2007 What is Studied in Practice Health Professionals as Team Member Health Professionals as Managers in Patient Care Clinical Expertise Patient Management Patient Communication Patient Education Quality Assurance Health Professionals as Managers of Themselves Insight Into Own Limits Self Care Lifelong Learning Continuing Education Personal Development Health Professionals as Managers of Their Environment Peer Communication Resource Utilization Systems Managtement Clinical Governance Respect for Colleagues Health of Populations Teaching Students Quality Assurance Hays et al, 2002

9 FAIMER 2007 Evaluation Methods  Quasi - experimental  Comparison groups?  Triangulation  Qualitative before quantitative  3-year minimum

10 FAIMER 2007 Data Comparisons Standards & groups to which data are compared?  Previous classes  Subgroups within the student population  Curricula & students at other schools  National or international data when available  Criterion based standards developed by staff

11 FAIMER 2007 Other Key Considerations  Support & resources  Grants 6-10% of budget  Safe data, storage  Talent, skills, expertise  Publication  Internal  External

FAIMER Thank you very much