Presentation is loading. Please wait.

Presentation is loading. Please wait.

Assessment of Trainee Competencies: Preliminary Psychometric Study David J. Martin, Ph.D. Harbor-UCLA Medical Center Geffen School of Medicine at UCLA.

Similar presentations


Presentation on theme: "Assessment of Trainee Competencies: Preliminary Psychometric Study David J. Martin, Ph.D. Harbor-UCLA Medical Center Geffen School of Medicine at UCLA."— Presentation transcript:

1 Assessment of Trainee Competencies: Preliminary Psychometric Study David J. Martin, Ph.D. Harbor-UCLA Medical Center Geffen School of Medicine at UCLA

2 Models of Competency Cube Model of Competency Development (Rodolfa, Bent, Eisman, Nelson, Rehm & Ritchie, 2005) –Functional Competency –Foundational Competency –Stages of Professional Development

3 Models of Competency Cube Model (Rodolfa et al., 2005): –Functional Competency Assessment-Diagnosis-Case Conceptualization Intervention Consultation Research Evaluation Supervision-Teaching Management-Administration

4 Models of Competency Cube Model (Rodolfa et al., 2005): –Foundational Competency Reflective Practice-Self Assessment Scientific Knowledge-Methods Relationships Ethical-Legal Standardspolicy Individual-Cultural Diversity Interdisciplinary Systems

5 Models of Competency Cube Model (Rodolfa et al): –Stages of Professional Development Doctoral Education Doctoral Internship/Residency Post-doctoral Supervision Residency/Fellowship Continuing Competency

6 Models of Competency

7 Practicum Competencies Outline –Association of Directors of Psychology Training Clinics (ADPTC) Practicum Competencies Workgroup (February 27, 2004) –Council of Chairs of Training Councils (CCTC) Practicum Competencies Workgroup (March 25, 2004) –Final CCTC comments, November, 2004 –Summarized by Hatcher & Lassiter (2006)

8 Models of Competency Practicum Competencies Outline –Levels of Competence Novice Intermediate Advanced Proficient Expert

9 Models of Competency Practicum Competencies Outline –Areas of Competence Relationship/Interpersonal Skills –With Patients/Clients/Families –With Colleagues –With Supervisors –With Support Staff –With Teams at Clinic –With Community Professionals –For the Practicum Site Itself

10 Models of Competency Practicum Competencies Outline –Areas of Competence Skills in Application of Research Intervention Skills Psychological Assessment Skills Consultation Skills/Interdisciplinary Collaborations Diversity (Individual & Cultural Differences)

11 Models of Competency Practicum Competencies Outline –Areas of Competence Ethics Professional Development –Practical Skills to Maintain Effective Clinical Practice –Professional Development Competencies Metaknowledge/Metacompetencies Development of Leadership Skills Supervisory Skills

12 Harbor-UCLA Assessment Psychological assessment skills ExternFellow Ability to select and implement multiple methods and means of evaluation in ways that are responsive to and respectful of diverse individuals, couples, families, and groups IP Ability to utilize systematic approaches to gathering data to inform clinical decision making IP Knowledge of psychometric issues and bases of assessment methods AP Knowledge of issues related to integration of different data sources AP

13 Harbor-UCLA Assessment Competenc y Level UnsatisfactoryNeeds Improvement SatisfactoryStrongOutstandingN/A 1.Ability to select and implement multiple methods and means of evaluation in ways that are responsive to and respectful of diverse individual, couples, families and groups. P Ability to utilize systematic approaches to gathering data to inform clinical decision making. P Knowledge of psychometric issues and bases of assessment methods. P Knowledge of issues related to integration of different data sources. P123456

14 Warning ! Attempts at complete review of the following slides may result in serious information overload!

15 Internal Consistency FellowsExterns ItemsN α N α Relationship/Interpersonal Skills With Patients/Clients/Families With Colleagues With Supervisors With Support StaffN/A With Teams at Clinic With Community ProfessionalsN/A For the Practicum Site Itself

16 Internal Consistency FellowsExterns ItemsN α N α Skills in Application of Research Intervention Skills Psychological Assessment Skills Consultation Skills/Interdisciplinary Collaborations Diversity (Individual and Cultural Differences) Ethics Professional Development Practical Skills to Maintain Effective Clinical Practice Professional Development Competencies Metaknowledge/Metacompetencies Development of Leadership Skills Supervisory Skills

17 Descriptive Statistics Fellows NMinMaxMeanS.D. Relationship/Interpersonal Skills With Patients/Clients/Families With Colleagues With Supervisors With Support Staff With Teams at Clinic With Community Professionals For the Practicum Site Itself

18 Descriptive Statistics Fellows NMinMaxMeanS.D. Skills in Application of Research Intervention Skills Psychological Assessment Skills Consultation Skills/Interdisciplinary Collaborations Diversity (Individual and Cultural Differences) Ethics Professional Development Practical Skills to Maintain Effective Clinical Practice Professional Development Competencies Metaknowledge/Metacompetencies Development of Leadership Skills Supervisory Skills

19 Descriptive Statistics Externs NMinMaxMeanS.D. Relationship/Interpersonal Skills With Patients/Clients/Families With Colleagues With Supervisors With Support Staff-- With Teams at Clinic With Community Professionals For the Practicum Site Itself

20 Descriptive Statistics Externs NMinMaxMeanS.D. Skills in Application of Research-- Intervention Skills Psychological Assessment Skills Consultation Skills/Interdisciplinary Collaborations Diversity (Individual and Cultural Differences) Ethics Professional Development Practical Skills to Maintain Effective Clinical Practice Professional Development Competencies Metaknowledge/Metacompetencies Development of Leadership Skills-- Supervisory Skills--

21 Inter-Scale Correlations Fellows Inter-Scale Correlations Ranged From.39 (Relationship Skills with Support Staff – Leadership Skills) -.93 (Assessment Skills – Consultation Skills) Externs Inter-Scale Correlations Ranged From.63 (Relationship Skills with Team – Ethics) -.95 (Intervention Skills – Consultation Skills)

22 Pre – Post Comparisons Fellows Evaluation Periodtp 12 NMS.D.NM Relationship/Interpersonal Skills With Patients/Clients/Families With Colleagues With Supervisors With Support Staff With Teams at Clinic With Community Professionals For the Practicum Site Itself

23 Pre – Post Comparisons Fellows Evaluation Periodtp 12 NMS.D.NM Skills in Application of Research Intervention Skills Psychological Assessment Skills Consultation Skills/Interdisciplinary Collaborations Diversity (Individual and Cultural Differences) Ethics Professional Development Practical Skills to Maintain Effective Clinical Practice Professional Development Competencies Metaknowledge/Metacompetencies Development of Leadership Skills Supervisory Skills

24 Pre – Post Comparisons Externs Evaluation Periodtp 12 NMS.D.NM Relationship/Interpersonal Skills With Patients/Clients/Families With Colleagues With Supervisors With Support Staff With Teams at Clinic With Community Professionals For the Practicum Site Itself

25 Pre – Post Comparisons Externs Evaluation Periodtp 12 NMS.D.NM Skills in Application of Research-- Intervention Skills Psychological Assessment Skills Consultation Skills/Interdisciplinary Collaborations Diversity (Individual and Cultural Differences) Ethics Professional Development Practical Skills to Maintain Effective Clinical Practice Professional Development Competencies Metaknowledge/Metacompetencies Development of Leadership Skills-- Supervisory Skills--

26 Discussion Preliminary data suggest that competencies can be measured reliably Inter-scale correlations raise important questions concerning competency constructs

27 Discussion Future Directions –Need for additional data to inform competency evaluation process –More controlled study of competency scales (e.g., mid-term and final ratings) –Factor-analytic approaches may address questions of inter-scale shared variance –Need for validation strategies


Download ppt "Assessment of Trainee Competencies: Preliminary Psychometric Study David J. Martin, Ph.D. Harbor-UCLA Medical Center Geffen School of Medicine at UCLA."

Similar presentations


Ads by Google