Practice Based Learning and Improvement Stephen J. Kimatian MD Assistant Professor of Anesthesiology and Pediatrics The Penn State, Milton S. Hershey Medical Center

PRACTICE-BASED LEARNING AND IMPROVEMENT Residents must be able to investigate and evaluate their patient care practices, appraise and assimilate scientific evidence, and improve their patient care practices.

Residents are expected to: analyze practice experience and perform practice-based improvement activities using a systematic methodology locate, appraise, and assimilate evidence from scientific studies related to their patients’ health problems obtain and use information about their own population of patients and the larger population from which their patients are drawn

Residents are expected to: apply knowledge of study designs and statistical methods to the appraisal of clinical studies and other information on diagnostic and therapeutic effectiveness use information technology to manage information, access on-line medical information; and support their own education facilitate the learning of students and other health care professionals

The Mandate Develop the skills of critical self examination and self improvement Become life long learners

The Question How do you evaluate yourself if you do not know what the standard for competence is? Must have an established standard. How do you establish the standard for competence? ACGME competency as applied to practice? Practice as applied to ACGME competency?

Perspective What is a consultant? Knowledge Skill Judgment How do you evaluate? Open Ended Easy to use but low on detail Highly Specific Complex and time intensive

Evaluation Tools Easy to understand Intuitive to use Generate useful information For self evaluation For evaluation of competence For focused remediation Minimize Bias

The Evaluation Project Goal – to develop an evaluation tool that maintains a global perspective while allowing detailed analysis of specific areas of strength or weakness. A tool that is simple to use Standardized language for the discussion of competency.

The Evaluation Project Phase 1 – Define “Consultant” using discrete statements. Phase 2 – Characterize defining statements. Phase 3 - Create a Database (rubrics) that defines Competency Phase 4 - Manipulate the data base to create evaluation tools.

The Project - Phase 1 Define “Consultant” in discrete statements. Intravenous Access Display knowledge of indications for IV access Develop plan for IV placement considering medical and surgical factors effecting gauge, position, and timing of IV placement. Communicate with patient throughout procedure showing respect and compassion Display knowledge of aseptic technique and universal precautions Display proper aseptic technique and universal precautions

The Project – Phase 2 Characterize defining statements Subspecialty (general, peds, cardiac ….) Phase of Care (preop, op, postop,clinical,…..) Core Competency (pt care, pbli, comm…..) Educational Domain (knowledge / skill / judgment) Training Level (PGY1, PGY2, PGY3…. consultant)

The Project – Phase 2 (cont) Intravenous Access Display knowledge of indications for IV access General / Preop / Pt Care / Knowledge / PGY 1 Develop plan for IV placement considering medical and surgical factors effecting gauge, position, and timing of IV placement General / Preop / Pt Care / Judgment / PGY 1 Communicate with patient throughout procedure showing respect and compassion General / Preop / Pt Care / Skill / PGY 1 Display knowledge of aseptic technique and universal precautions General / Preop / Pt Care / Knowledge / PGY 1 Display proper aseptic technique and universal precautions General / Preop / Pt Care / Skill / PGY 1

The Project – Phase 3 Create a Database (rubrics) that define “Competency” IV AccessCognitive / KnowledgeSkills / BehaviorsAttitude / Judgment Patient Care: Residents must be able to provide patient care that is compassionate, appropriate, and effective for the treatment of health problems and the promotion of health. PGY1/2 -Display Knowledge of indications for IV access - Display knowledge of aseptic technique and universal precautions PGY1/2 - Communicate with patient showing respect and compassion -Display proper aseptic technique and universal precautions PGY1/2 - Develop plan for IV placement considering medical and surgical factors effecting gauge, position, and timing of IV placement CA2 - Display knowledge of advance IV access modalities CA2 -Demonstrate skills needed to establish IV access in difficult patient - Effectively Place IV in uncomplicated patient with little or no assistance CA2 CA3

The Project – Phase 4 Develop Web based evaluation form Rotating selection of 10 – 15 questions Substandard rating result in automatic expansion of question Resident performance tracked System tracks which competencies have been evaluated Comparison of resident self evaluation vs. faculty evaluation Deficiencies characterized to create remediation plan focused on specific problems and with identifiable points for reevaluation Faculty Clinical Teaching and Evaluation tracked Puts evaluations in perspective with “norms” Identify Biases / “Halo Effect” Part of 360 degree evaluation

Project Time Line Phase 1 / 2 Underway for General Anesthesia Working model by June 03 Subspecialties added by Fall 03 Phase 3 / 4 First meeting with program team 15 April Database parameters by June 03 Phase 3 Beta testing by Sept 03 Phase 4 Beta testing by Fall 03

Questions ?