Presentation on theme: "Michele J. Karel, PhD VA Boston Healthcare System"— Presentation transcript:
1Development of a Rating Scale to Assess Geropsychology Practice Competencies Michele J. Karel, PhDVA Boston Healthcare SystemHarvard Medical SchoolFor the CoPGTP Task Force on Geropsychology Competency Assessment:Michele Karel, Jeannette Berman, Jeremy Doughan, Erin E. Emery, Victor Molinari, Sarah Stoner, Yvette N. Tazeau, Susan K. Whitbourne, Janet Yang, Richard ZweigAPPIC Conference, April 18, 2009
2ObjectivesIncrease awareness of efforts to define and evaluate competencies for geropsychology practiceConsider challenges for designing and implementing a competency rating scaleUtility in internship and postdoc settingsAddress ongoing challenges and directions for work in this area
3Importance of Professional Geropsychology Training and Evaluation Aging populationWorkforce shortages for geriatric health care, across disciplinesRelatively few psychologists have formal training in geropsychologyIncreasing numbers will be working with older adultsHow to evaluate competence, and define training needs, for psychological practice with older adultsHow to help training programs, and post-licensure psychologists, expand opportunities for geropsychology training
4Growth of Professional Geropsychology Growing number of grad, intern, and fellowship programs offering geropsychology trainingGrowth of research, journals, professional organizationsSeveral recent milestones:APA Office on Aging, established 1998APA Guidelines for Psychological Practice with Older Adults (2004)CRSPPP recognition of gero as proficiency area, 1998 and 2005Pikes Peak Model for Geropsychology Training, National Training ConferencePikes Peak Training Model* defined Attitude, Knowledge, Skill competencies for geropsychology practiceHow to evaluate professional geropsychology competence?*Knight, B.G., Karel, M.J, Hinrichsen, G.A., Qualls, S.H., & Duffy, M. (in press). Pikes Peak model for training in professional geropsychology. American Psychologist.
5Project InitiationCouncil of Professional Geropsychology Training Programs (CoPGTP)Formed after 2006 Pikes Peak conferenceFor training programs at all levelsCoPGTP undertook to explore methods to evaluate Pikes Peak competenciesCoPGTP established a Task ForceAgreed to start with goal of developing a competency rating scale
6Project Goals Develop an evaluation tool that would: Address the Pikes Peak geropsychology attitude, knowledge and skill competenciesBe useful for supervisor evaluation of traineesFormative and summative evaluationBe useful for self-evaluation by learnersBe useful across levels of trainingGraduate practicum, internship, fellowship, post-licensureBe useful to help define ongoing training goals
7Pikes Peak Geropsychology Practice Competencies Competency definitions informed by:APA Guidelines for Psychological Practice with Older Adults (2004)Cube Model for Competency Development (Rodolfa et al., 2005)Psychology foundational and functional competencies develop over course of training career2006 Pikes Peak Conference process:Working groups address 6 competency domains(separate working groups address 4 levels of training)Iterations of competencies lists during conference, and afterwards via listserv discussion
8Pikes Peak Competencies Are aspirationalAim to facilitate as many psychologists as possible to work effectively with older adults, not to pose barriers to joining the fieldCompetencies are aimed at level of practice of a newly licensed psychologist who has finished one year of geropsychology postdoctoral trainingCompetencies not geared to level of an “expert”, but to someone who practices with wide range of older adults, families, systems
9Pikes Peak Competencies: Attitudes Work within one’s scope of competenceRecognize how one’s beliefs about aging and older adults may affect practiceAppreciate diversity among older adults, and interactions between age/cohort and other aspects of individual diversitySeek continuing education, supervision, consultation
10Pikes Peak Competencies: Knowledge Base General knowledge about adult development, aging, and the older adult populationFoundations of professional geropsychology practiceAssessmentInterventionConsultation
11Pikes Peak Competencies: Skills Professional Geropsychology Functioningi.e., foundational skills applied to geropsychology practiceAssessmentInterventionConsultationNote: Skills for functional domains of research, supervision, and management are important for growth of geropsychology, but were not included in Pikes Peak core competencies for practice with older adults
12Pikes Peak Competencies: Settings of Care Geropsychologists should be able to deliver services effectively in range of (at least 2) settings, such as:Outpatient mental health servicesOutpatient primary care/medical settingsInpatient medical serviceInpatient psychiatric serviceLong-term care settings including nursing homes, assisted living facilities, home care, day programsRehabilitation settingsHospiceOthers
13CoPGTP Competency Evaluation Task Force Methodology ~Monthly conference calls, 9/07 to 7/08Reviews of relevant, informing materialsSmall working groups define “behavioral anchors” for each competency domainIterative reviews of all scale items by larger groupGroup discussion, selection, editing of:Rating scale/anchors and vignetteIntroduction to the evaluation toolTask Force and other CoPGTP members try out the scale to evaluate: a student, oneself, or both
14Informing Materials: Psychology Competencies Assessment Guiding principles and recommendations for the assessment of competence(Kaslow et al., 2007 and other publications of APA Task Force on Assessment of Competence)Assessment of Competency Benchmarks Work GroupA developmental model for defining and measuring competence in professional psychologyJoint effort of APA BEA and CCTCThe Practicum Competencies OutlineHatcher & Lassiter (2007), workgroups of ADPTC and CCTC
15Informing Materials: Geropsych Competency Assessment Example of evaluation tools used in various geropsychology practicum, internship, and postdoctoral programsAlso, examples of competencies and evaluation methods from:Medicine – ACGME Outcome Project, 2000Geriatrics, Palliative Care, Geriatric Social Work, Geriatric Psychiatry
16Delineation of Geropsychology Knowledge and Skill Competencies Each Pikes Peak competency domain (e.g., foundational geropsychology skills) is defined by 3-8 specific competencies (e.g., ethical practice, appreciation of diversity, self-reflection…)We aimed to further specify each of these competencies with behavioral anchorsHow would you know it if you saw it (in oneself or another)?e.g., what does it really mean to “Relate effectively and empathically with older adults clients, families, and other stakeholders in a range of professional roles and settings?”This approach leads to a very long evaluation tool!
17Example: Delineating one of the foundational skill competencies
18Example: Delineating one of the assessment skill competencies
19Evaluation of Attitudes Pikes Peak attitudes for competent geropsychology practice are embedded in the foundational skillsE.g., self-reflection re: biases/discomforts, awareness of limits of one’s competence, seeking consultation, seeking continued educationAttitudes are specified in tool’s introductionFor now, attitudes not evaluated separatelyPerhaps they should be? How?
20Rating Scale Developmental rating scale Novice, Intermediate, Advanced, Proficient, ExpertScale anchors conceptually complexAdapted definitions from Hatcher and Lassiter (2007)Included in anchor definitions how much supervision/consultation likely neededProvided vignette and approach to that case to illustrate each level of competenceInstructions to rate each Pikes Peak competency, not each behavioral specifier (but can if desired)
21Rating Scale Abbreviated scale anchors, printed on each page: N = Novice: Possesses entry-level skills; needs intensive supervisionI = Intermediate: Has a background of some exposure and experience; ongoing supervision is neededA = Advanced: Has solid experience, handles typical situations well; requires supervision for unusual or complex situationsP = Proficient: Functions autonomously, knows limits of ability; seeks supervision or consultation as neededE = Expert: Serves as resource consultant to others, is recognized as having expertise
22Rating ScaleExpected ratings vary considerably through levels of trainingPracticum: Novice through AdvancedInternship: Novice through ProficientFellowship and post-licensure: Intermediate through ExpertIdeally, usable to evaluate a trainee, AND to evaluate oneself
23Introduction to the Evaluation Tool Reviews:Purpose of toolPikes Peak competencies as aspirationalApplication across range of geropsychology practice and training settingsInstructions for useIncluding that evaluation should include the learner's perspective (self-assessment), observation of the learner's work, and regular supervision involving case discussionPsychologists and trainees conducting self-assessments can evaluate their training and supervision needs in each areaDefinition of rating scale anchors, and illustration of its use through a case vignette
24Piloting the Evaluation Tool Asked CoPGTP members, and ourselves, to complete the tool to evaluate a student or oneself13 responses6 CoPGTP members (other than us)4 Task Force members3 from Task Force members’ students/staffUsed tool to:4 to evaluate a student5 to evaluate oneself2 evaluated oneself and a student2 provided general feedback without specifying
25Asked for Feedback: Asked folks to answer these questions: How did you use the tool (to evaluate a student, or yourself?)What was your general impression of using the tool?Was the introduction to the tool clear…?Was the rating scale (Novice, Intermediate, Advanced, Proficient, Expert) understandable…?Was the length of the tool a problem? Did it adequately address breadth/depth of geropsychology competencies?What do you think the best use of this tool will be?Do you expect to use this tool? In what way?Any other feedback?
26FeedbackMost felt tool is very long, but worth it given attention to breadth and depthA few felt it was too long for practical purposes(Challenge – how to cut?)Most felt introduction was clear, but needed clarification not to rate each specifierMost liked the NIAPE rating scale
27Feedback Concerns raised about the scale included: Does it apply as well to knowledge as skills?Does it work as well for practicum level training where students may be novices in most areas?Is Expert rating too “loaded” (people not comfortable using that term?)Can scale capture development over time?If/how best to evaluate attitudesSuggestion to have summary pageStrengthsAreas for growthOngoing training goals
28Use of the Scale to DateDisseminated to CoPGTP training programs and APA 12-2 and 20, PLTC listservsPrograms (grad, intern, and postdoc) are starting to use the tool for:Initial training needs assessment with studentsMid-year/final supervisory evaluationsStudent self-evaluation (encouraging “metacompetence”)
29Use of the Scale to DateTool being used by some for program self-study and developmentDefining training program objectivesEvaluating, in discussion with students and supervisors, how well program is addressing each competency domainHelping to evaluate competence of new geropsychology supervisors and define their training goalsBeing used in community/CE education programs on geropsychology
30Uses for Intern/Postdoc Training Along with APA standards, can help define rotation/track training objectivesGero evaluation scale is rotation/track specific, and adjunctive to training program’s more general evaluation toolsExtremely helpful for initial training needs assessment because students come in with widely varying previous gero experienceClinical, academic, researchCan tailor initial training plan and review over course of the training year/rotationIt is additional work at evaluation time for supervisor and student – not clear how strong a barrier this may pose
31Study-in-the-WorksAims: Examine utility and validity of the rating scale, initially for self-evaluation purposesWhich competencies rated as most vs least well-developed?Do self-rated competencies relate to years of training or practice?To settings and major activities of practice?What feedback is provided after completing the scale?Planning on-line surveyOf psychologists who work with older adults and geropsychology graduate, intern, posdoc studentsEventually, plan longitudinal study of supervisory evaluations of developing geropsychologists
32Concerns/ChallengesWill clinicians see such a tool as helpful or somehow restrictive to practice?Will supervisors/learners use such a long evaluation tool (even if can pick and choose domains of relevance)?Can competencies be reduced/streamlined?Can we boil down further “core” competencies? Are some more essential than others? (??)May vary across settings/populations of older adultsWill tool be useful as self-evaluation, self-study aid?Can competency ratings be linked to training resources?
33Opportunities/Future Directions Helpful for initial training needs assessment with a student, to guide training planMay help to develop on-line version, then select only domains that are relevant to training program, rotationUseful as guide to developers of CE programs?Develop organized, sequential CE offerings?Ultimately, plan to link competencies to training resources and opportunitiesWorking to keep such list updated at CoPGTP websiteExpand toolbox of gero competence evaluation toolsLargest challenge - how to expand interest in, and opportunities for, training in professional geropsychology