© Medical College of Georgia 2007 Recognizing, Evaluating, and Rewarding Educators and Educational Scholarship Health Professions Educational Research Symposium January 14, 2007 Fort Lauderdale, FL Ruth-Marie E. Fincher, MD Vice Dean for Academic Affairs Medical College of Georgia School of Medicine

“Education Is Our Core Mission” Research Service Patient Care Education From Michael Wilkes, 2003

How It Seems Research Service Patient Care Education Modified by R Fincher, 2004

Making Us Feel Like...

Agenda for Presentation Background –Teaching & educational scholarship Framework –Assessing teaching & learning GEA scholarship project –5 categories of educational activities –Assessment of each category Quantity Quality Engagement with educational community –Scholarly approach –Scholarship

Changing Institutional Culture May Be More Difficult than Moving a Graveyard The Scholarship Culture is Changing Swanson G. In Tosteson et al. New Pathways to Medical Education. Cambridge MA, Harvard University Press, 1994 Slide adapted from Malcolm Cox

Teaching and Its Scholarship Are Often... Overshadowed by –Research –Clinical care Undervalued in reward system –Promotion –Tenure –Salary

Institutions Reward What They Value Teaching is highlighted in most mission statements Teaching and related scholarship must be rewarded if they are truly important to an institution

Recent Evolution of Medical School Environment Caused a “Crisis of Mission” ResearchTeachingClinical Care FundingSource (Is it?) Cost (Is it?) Source KnowledgeDiscoveryTransmissionApplication Value“Opportunity”“Obligation”Critical for survival Concept from Cuban: How Scholars Trumped Teachers, 2001

Think outside the box!

Medical Schools Only Unique Mission = Teaching....“widespread agreement that those members of the faculty who are most committed to... the education of medical students must be supported and rewarded.... Faculty appointments, promotion and tenure policies must reflect the changing roles and responsibilities of medical school faculty.” Whitcomb M: Acad Med 78: , 2003.

Evolution of Concept of Teaching and Scholarship Pre-Boyer –Teaching = academic citizenship Everyone does it Little to do with scholarship or promotion Boyer –Reframed discussion by Challenging concept that “everyone teaches” Examining teaching as a form of scholarly work Boyer: Scholarship Reconsidered, 1990.

Continued Evolution: Glassick* Common criteria form foundation for all forms of scholarship Clear goals Adequate preparation Appropriate methods Significant results Effective presentation Reflective critique * Scholarship Assessed, 1997.

AAMC’s Group on Educational Affairs: Promotes Education Across Continuum of Medical Education Undergraduate Continuing Graduate RIME

GEA Scholarship Project Phase Phase Phase Phase ? GOAL: Value faculty who support and advance medical education by connecting education, scholarship and academic advancement

GEA Scholarship Project Phase Phase Phase Provided examples of activities and evidence Simpson & Fincher (1999) Defined educational scholarship Outlined infrastructure needed to support educational scholarship Fincher et al. (2000) Phase ?

Characteristics of Scholarship Demonstrates the “3 P’s”: –Product that can be reproduced and built upon –Open to peer review –Publicly disseminated Scholarship moves field forward Assessment parallels research

Phase GEA Scholarship Project Phase Phase Identified types of educational activities & evidence Simpson et al. (2004) Characterized infrastructure – Academies Irby et al. (2004) Phase ?

Education Activity Categories Teaching Curriculum Advising/Mentoring Educational Administration/Leadership Assessment of Learner Performance Simpson D, et al. Acad Med 2004;79:

Infrastructure to Support Education Faculty development centers –Career development –Teaching/educational research skills Societies or Academies –Recognize accomplishments based on selection criteria

Phase GEA Scholarship Project Phase Phase GEA Consensus Conf on Educational Scholarship Defined areas of agreement & concerns in 3 areas Category inclusions Evidence & presentation displays for each category Areas needing further investigation Phase ?

Categories of Education Activity Teaching Curriculum Advising/Mentoring Educational Administration/Leadership Assessment of Learner Performance Simpson D, et al. Acad Med 2004;79:

Q 2 + Engagement with Community Contribution to education mission Quantity Quality Engagement with education community Scholarly approach to education activity Educational scholarship Draws FromContributes To

Scholarly and Scholarship Existing literature Best practices Resources in field Colleagues locally  internationally Dissemination –Peer reviewed forums Local  international audiences Impacts the field Scholarly work is influenced by: Scholarship contributes to the field:

Documentation: Teaching Category Brief Description –Role, activity description Evidence of Quantity –Who, what, when, where, how much, how many Evidence of Quality –Summary of student/peer evaluations –Short excerpts from supporting letters –Invitations to teach outside department or school –Repeat invitations to teach same group or in course

Teaching Definition and Inclusions Is the design and implementation of activities to promote learning Includes –Course design –Development of instructional materials –Interactions with students –Formative and summative assessment Act of teaching is not scholarship

Teaching: Quantity & Quality RoleYr(s)QuantityNumber Learners Quality Family Medicine Clerkship Seminar Leader 1997-~19 hr/ year 8-12/ rotation; 8 rotations/ year ’05-06: “Effective seminar leader” Mean = 5.41 (7-point scale)* Give comparative ratings for each year; compare with *Comparative ratings should be given over time; Compare with peer group if possible *Give comparative ratings for each year; provide comparative ratings with peers if possible

Teaching: Quantity and Quality

Engagement with Community Scholarly approach: Learns from –Educational literature –Best practices Scholarship: Contributes to field –Product: Interactive cases –Peer review: MedEdPORTAL –Public: Web

Scholarly Approach Teaching - learning connection –Consult education literature –Apply intervention to enhance learning –Observe outcomes –Analyze results –Obtain peer evaluation –Use results to improve teaching Product reflected in student learning Richlin L: Scholarship Revisited: Perspectives on the Scholarship of Teaching. Jossey-Bass, 2001

Evidence of Scholarship Product: Instructional material –Interactive, diagnostic decision making cases: Cough, chest pain Describe role Peer review and public dissemination –Website: xx hits; xx schools adapted or adopted at least 1 case; representative feedback comments –Accepted by national peer-reviewed venue; e.g., MedEdPORTAL

Curriculum Development Definition and Inclusions Longitudinal designed educational activities –Any training level –Various venues and delivery formats Includes –Goals and objectives –Learning experiences to achieve goals & objectives –Organization and sequencing to ensure effective learning –Evaluation of effectiveness

Curriculum Development Activity and Role Title of course –Evidence-based Medicine (Yr. 1 students) Role: Course director –Organized all lectures and self-directed learning exercises for 5 credit hour course Developed objectives and content Recruited and trained facilitators Worked with clinicians and statisticians to develop illustrative cases

Engagement with Community* Clear Goals –Create a new EBM course for all first-year students that students perceive as clinically relevant. The predecessor course consistently received “very poor student evaluations” and EBM content was “lacking in the curriculum.” Adequate Preparation –PhD in Public Health –Review of “best practices: McMaster’s curriculum, NBME test content –Review of literature; application to new course *Applying Glassick’s Criteria

Engagement: Glassick  Appropriate Methods  Multi-method approach including:  Interactive lecture series  Clinical vignettes  Abstract followed by article critique  Increased collaboration  Small groups co-led by basic scientist/MD  Assessment methods: Test questions assessed application of knowledge

Engagement  Significant Results (Outcomes)  Didactic series evaluation  Improved 1 SD from prior offering  AAMC Graduation Questionnaire  EBM rating improved from inadequate exposure to appropriate/excessive  USMLE sub-test score  Improved from below to above average  OSCE performance on ambulatory practice module  Improved

Engagement with Community: Scholarship (Contributing to) Effective Presentation (Dissemination) –Results presented to curriculum committee –Internal review; comparison over last 2 years –Content replicated in clerkships & residencies Reflective Critique (Next steps for continuing improvement) –Convert course to on-line format to minimize # faculty needed –Submit to AAMC MedEdPORTAL (Peer review)

Advising and Mentoring Definition and Inclusions Advising and mentoring –Educator provides guidance or counsel to facilitate accomplishment of a learner’s or colleague’s goals Advising –Limited time; advisor serves as guide to enable advisee to achieve her or his goals Mentoring –Sustained relationship; mentor and protégée obtain reciprocal benefits

Advising & Mentoring NameCW: Medical Student  Medicine Resident Duration/ Process 9/01 - present: 4 mtg/yr; Edit (paper, CV, application to be LCME student liaison) OutcomeMD Received; Paper published in Acad Med; student liaison to LCME Effective- ness “I wanted to thank you for … making me the young man I am today…not sure you realize impact you’ve had in my life…” Engage- ment Scholarly: Learned from literature, professional development activities, etc. Scholarship: Mentor manual adopted by others; workshop at national meeting

Educational Leadership Definition and Inclusions Definition –Exceptional leadership transforms educational programs and advances field Inclusion criteria –Pursues excellence –Evaluates and engages in self-reflection –Builds on work of others –Disseminates results – advances the field –Garners and maximizes resources

Leadership Documentation Quantity –Describes nature of activity, duration, rationale for change, goals, and leader’s role Quality –Describes actions, accomplishments, evaluation and resources garnered/utilized Engagement with educational community –Builds on literature and best practices –Disseminates and reflectively critiques

Leadership Project Basic Science/Clinical Integration Need –Courses lacked clinical relevance, poorly taught Goal –Increase integration across 4 years, clinical relevance, quality Methods and preparation –Revised courses –Conducted faculty development during –Garnered resources: Support staff and faculty time Results (Evaluation) Presentation (Peer review and dissemination –6 peer reviewed activities Reflective Critique –Next year plan to… Rating Overall = Poor 5 = Outstanding

Learner Assessment Definition and Inclusions Activities associated with measuring learners’ knowledge, skills, and attitudes Assessment-related activity(ies) include: –Development Identifying and creating assessment processes and tools –Implementation Collecting data using processes and tools –Analysis Comparing data to correct answer key and/or performance standards –Synthesis and presentation Interpreting and reporting results

Learner Assessment Documentation Quantity –Type of assessment –Learner population E.g., who and how many –Size/scope E.g., number of items, frequency of use –Intended uses of results –Faculty member’s role Quality and engagement –Evidence of adherence to Glassick’s criteria

Learner Assessment Example of Presentation

Results: Caveats & Remaining ?s Each institution must determine what counts for promotion –Contributions to institution’s mission (Q 2) Q 2 within and across categories Sustained contributions? –Contributions to the field (Engagement) Draw from and contribute to field Sustained contributions? –How to value individual vs group contributions Core elements education infrastructure

GEA Scholarship Project Next Steps Phase Phase Phase Dissemination & Infrastructure: Individual & Institutional to Support Educators AAMC-GEA Annual & Regional Meetings + Publish Institutional Team Workshops Individual Faculty Workshops Phase ?