Presentation is loading. Please wait.

Presentation is loading. Please wait.

June, 2002AFPRD Workshop Web-Enabled Evaluation Techniques: Applying a Competency-Based Curriculum Charles O. Frazier, M.D. Jon J. Kaminer, M.D.

Similar presentations


Presentation on theme: "June, 2002AFPRD Workshop Web-Enabled Evaluation Techniques: Applying a Competency-Based Curriculum Charles O. Frazier, M.D. Jon J. Kaminer, M.D."— Presentation transcript:

1 June, 2002AFPRD Workshop Web-Enabled Evaluation Techniques: Applying a Competency-Based Curriculum Charles O. Frazier, M.D. Jon J. Kaminer, M.D.

2 June, 2002AFPRD Workshop Web Cameras

3 June, 2002AFPRD Workshop Patient Encounters

4 June, 2002AFPRD Workshop Command and Control Center

5 June, 2002AFPRD Workshop Feedback

6 June, 2002AFPRD Workshop Feedback

7 June, 2002AFPRD Workshop

8 June, 2002AFPRD Workshop Objectives 1.Why effective evaluation is critical 2.How the ACGME guidelines can fit in an effective evaluation scheme 3.How a web-enabled evaluation tool developed at Riverside can be used to apply a competency-based curriculum

9 June, 2002AFPRD Workshop Why is an evaluation system needed? –Its a requirement –Further defines expectations –Promotes goal directed behavior –Provides comfortable atmosphere for constructive criticism and positive reinforcement

10 June, 2002AFPRD Workshop What is effective evaluation? –Specific/focused –Timely –Consistent –Target behavior/performance reasonable –Conducted in collegial manner –Ends with defined goal/expectation of future behavior/performance

11 June, 2002AFPRD Workshop What are the main evaluation modalities? feedbackfeedback summative evaluationsummative evaluation

12 June, 2002AFPRD Workshop Feedback Immediate commentImmediate comment

13 June, 2002AFPRD Workshop Summative Evaluation Longitudinal measure of performanceLongitudinal measure of performance

14 June, 2002AFPRD Workshop Modality Comparison

15 June, 2002AFPRD Workshop How does the ACGME outcomes project fit in to this evaluation framework?

16 June, 2002AFPRD Workshop Competencies

17 June, 2002AFPRD Workshop List of ACGME competencies Patient CarePatient Care Medical KnowledgeMedical Knowledge Practice-Based Learning and ImprovementPractice-Based Learning and Improvement Interpersonal and Communication SkillsInterpersonal and Communication Skills ProfessionalismProfessionalism Systems-Based PracticeSystems-Based Practice

18 June, 2002AFPRD Workshop

19 June, 2002AFPRD Workshop Toolbox Record reviewRecord review Chart Stimulated recallChart Stimulated recall ChecklistChecklist Global ratingGlobal rating SPSP OSCEOSCE Simulations and ModelsSimulations and Models

20 June, 2002AFPRD Workshop Toolbox 360º Global Rating360º Global Rating PortfoliosPortfolios Exam MCQExam MCQ Exam oralExam oral Procedure case or logsProcedure case or logs Patient surveyPatient survey

21 June, 2002AFPRD Workshop Benefits –defines the ACGMEs expectations of us –Defines our expectations for residents to graduate –allows for consistency –changes focus of evaluation to resident performance rather than residency structure

22 June, 2002AFPRD Workshop Limitations –Incredible amount of work and paper work –How will we use all this information to provide meaningful feedback to the resident –Will this take away from resident teaching time

23 June, 2002AFPRD Workshop Riverside Family Practice Residency

24 June, 2002AFPRD Workshop Our former evaluation framework FeedbackPrecepting

25 June, 2002AFPRD Workshop Our former evaluation framework Summative Evaluation –Trimester Faculty meetings –Trimester directors meetings Rotational evalsRotational evals Full faculty impressionFull faculty impression

26 June, 2002AFPRD Workshop Pros –Plenty of face-time and feedback during precepting –Gestalt of faculty about resident shared and discussed –Not much paperwork for faculty

27 June, 2002AFPRD Workshop Cons –No specifics –Summative evaluation disconnected from feedback –Inefficient –No good individualized action plan –Overloaded residency director

28 June, 2002AFPRD Workshop Structural Changes Made –Advisor system –Quarterly advisor meetings with Educational Prescription –Semiannual director meetings Advising - March 2001

29 June, 2002AFPRD Workshop Competency-based System Paper evaluation forms developed in light of competenciesPaper evaluation forms developed in light of competencies Expectations defined for residents and facultyExpectations defined for residents and faculty Our residency toolbox was expanded and redefined based on ACGME guidelinesOur residency toolbox was expanded and redefined based on ACGME guidelines

30 Tool Matrix Page425

31 June, 2002AFPRD Workshop Resident Rotations Precepting DOP Patients Nurses Peers Medical Students Care Managers Research Project Procedure Report RecordCompletionReport

32 June, 2002AFPRD Workshop Reams of useless paper

33 Rotations Precepting DOP Patients Nurses Peers Medical Students Care Managers Research Project Procedure Report RecordCompletionReport

34 Rotations Precepting DOP Patients Nurses Peers Medical Students Care Managers Research Project Procedure Report RecordCompletionReport OnlineComp-EvalServer FacultyResident

35 June, 2002AFPRD Workshop August/September - Nurse Evaluation / Faculty Advisor Evaluation November/December - Nurse Evaluation / Faculty Advisor Evaluation December - Semi-Annual Program Director Evaluation February/March - Nurse Evaluation / Faculty Advisor Evaluation May/June - Nurse Evaluation / Faculty Advisor Evaluation June - Semi-Annual Program Director Evaluation

36 June, 2002AFPRD Workshop Web-Enabled System A stand-alone evaluation systemA stand-alone evaluation system Universally accessibleUniversally accessible Highly customizableHighly customizable Data automatically collected in databaseData automatically collected in database –stored, studied, and shared

37 June, 2002AFPRD Workshop Technical description - Intranet Compaq Proliant ServerCompaq Proliant Server Microsoft Windows NT 4.0Microsoft Windows NT 4.0 Microsoft Internet Information Services 5.0Microsoft Internet Information Services 5.0 Microsoft FrontPage 2000Microsoft FrontPage 2000 –Active Server Pages Microsoft Internet ExplorerMicrosoft Internet Explorer

38 June, 2002AFPRD Workshop Active Server Pages Script (programming) is executed on the server, and resulting HTML is sent back to and displayed on browserScript (programming) is executed on the server, and resulting HTML is sent back to and displayed on browser –Connects to database –Underlying script secure

39 June, 2002AFPRD Workshop WebBrowser Database

40 June, 2002AFPRD Workshop Technical Description - Intranet SecuritySecurity –NT Server Authentication CostCost –Server $3,000 –Software $1500 –Labor (estimate) 450 hours

41 June, 2002AFPRD Workshop Technical Description - Internet VCU IIS Public Information ServerVCU IIS Public Information Server Security with ASP database passwordsSecurity with ASP database passwords No direct cost to residencyNo direct cost to residency Domain name registrationDomain name registration –www.pubinfo.vcu.edu/webeval

42 June, 2002AFPRD Workshop Demonstration

43 Questions?

44 June, 2002AFPRD Workshop The future –Ability to study competency-based education –Evaluate Program Curriculum –Ability to describe expected Natural History or Growth Curve for resident –Generate documentation to support putting a resident on academic probation

45 June, 2002AFPRD Workshop Questions?


Download ppt "June, 2002AFPRD Workshop Web-Enabled Evaluation Techniques: Applying a Competency-Based Curriculum Charles O. Frazier, M.D. Jon J. Kaminer, M.D."

Similar presentations


Ads by Google