1. Evaluation and Measurement of Inter-professional Geriatric Programs
Paul Gould, LCSW, PhD Binghamton University State University of New York Kristina Hash, LICSW, PhD West Virginia University

2. WHAT DO WE MEAN BY EVALUATION?

3. Program Evaluation
The process of systematically applying research methods to assess program:
design
implementation
improvement
outcomes
Rossi, P. H., & Freeman, H. E. (1993). Evaluation: A systematic approach (5th ed.). Newbury Park, CA: Sage Publications, Inc.

4. Program Evaluation
Needs Assessment: The process of systematically applying research methods to assess needs, gaps, or areas for improvement

5. EXAMPLES OF EVALUATION INTER-PROFESSIONAL GERIATRIC PROGRAMS

6. exaMPLE #1 West virginia university

7. Example 1: West Virginia University
Gerontology Practitioner Certificate
Needs Assessment and Program Development
K. Hash, J. Englehardt, &
Amazing Work Group
Funded by: WV Partnership for Elder Living
$500o grant from Benedum Foundation/WVLTCP to assess training needs in geriatrics of interdisiclinary professionals

8. Example 1: West Virginia University
Objectives:
1.Evaluate the aging education needs of social workers and related professionals (with associate degrees and higher) in the state
2.Develop a gerontology practitioner certificate and curriculum
3.Develop best practices for recruiting and retaining professionals in the field 
The School began by gathering the interdisciplinary, statewide workgroup in August.  A survey would be developed by this workgroup to distribute through various professional mailing lists and electronic listservs, including our own PCE listserv by September.  The survey would assess the aging education needs of direct care workers as well as inquire about the best ways to attract and retain quality workers in the field. 

9. Example 1: West Virginia University
Objectives:
1.Evaluate the aging education needs of social workers and related professionals (with associate degrees and higher) in the state
2.Develop a gerontology practitioner certificate and curriculum
3.Develop best practices for recruiting and retaining professionals in the field 
The School began by gathering the interdisciplinary, statewide workgroup.  A web-based survey developed by this workgroup to distribute through various professional mailing lists and electronic listservs, including our own PCE listserv by September.  The survey assessed the aging education needs of direct care workers as well as inquired about the best ways to attract and retain quality workers in the field. 

10. Example 1: West Virginia University
Online Survey, N= 391
Training Needs
77% interested in enrolling in a CEU-based certificate
Training needed: cognitive changes/dementia; end of life; legal issues; HC and aging policy
Great interest in online training
  The majority of respondents who identified a discipline in terms of education were trained as social workers (46%, n=164), followed by nurses and speech pathologists (each 16%, n=57). Professionals with educational backgrounds in education, counseling, sociology and psychology each represented about 3% of the respondents. Most had earned a bachelors (27%, n=97) or masters (56%, n=195) degree their identified discipline and over 85% also held a professional license. In terms of formal geriatric training, 43% (n=153) identified CE offerings as the source of this training. Thirty-three percent (n=118) indicated that they had no formal training in geriatrics. Largely from WV
The major curriculum areas of a previously offered certificate were listed and respondents asked to base their interest and need on this general curriculum
Cognitive changes and dementia and end of life issues were each selected by almost half of the respondents. Legal issues and health care and social policies and programs were also of great interest to over 30% of respondents. Respondents currently meet their CE needs primarily through employer-based trainings (33%, n=121) and attendance at the NASW-WV annual conference and the Summer Institute on Aging and other Division of Social Work-sponsored workshops (each selected by over 20% of the respondents).

11. Example 1: West Virginia University
“A curriculum is very much needed for our workers to understand the population that we are dealing with and the need is urgent.”
“It is very evident that WV NEEDS to offer this certificate.”

12. Example 1: West Virginia University
Recruitment and Retention
What led them to the field?
Job opportunity & Personal Experience
What will keep them in the field?
Career growth, Increased Salary & Benefits,
Leadership Opportunities
Love the Field and Population
When asked what led them to work in the field, many noted a job opportunity (57%, n=201) as well as personal and family experiences (29%, n=101). Open-ended responses to this question indicate that many respondents have always “enjoyed” working with this population. Increased opportunity for career growth as well as increased salary and benefits were all selected by over 40% of respondents. Increased opportunity for leadership in the field was selected by 28% (n=98) of those who worked in the field.

13. Example 1: West Virginia University
“No motivation is needed. I love the population, and I intend to continue working with them, regardless of pay-rate or continuing education”
“I love what I do and just want to keep doing it.”
Many noted their love for the field and population as reasons to stay in the field

14. Example 1: West Virginia University
Outcomes of Needs Assessment
Development of CEU-based certificate
Needs-identified curriculum
Partnerships with other organizations offering trainings
Development of online trainings
Recruitment and retention ideas shared
Certificate Launching in February 2012!

15. EXAMPLE #2 BINGHAMTON UNIVERSITY

16. Geriatric Consultation Clinic
Shawn Berkowitz, MD, CMD
Paul R. Gould, LCSW, PhD
Youjung Lee, PhD – Evaluator
Suk-Young Kang, PhD – Evaluator
Supported with funding from the John A. Hartford Foundation

17. Binghamton University HPPAE Geriatric Consultation Clinic
Interprofessional Learning Experience
Part of the rotational model for HPPAE
MSW students collaborate with Medical Students from Upstate Medical University
Conduct a 2-part Comprehensive Geriatric Assessment
Home Visit
Clinic Visit
Team Structure:
Clinical Social Worker
Geriatrician
Students from both disciplines
Family

18. Geriatric Consultation Clinic Student Learning Evaluation
Evaluation Methods
A mixed-method using a triangulation design-multilevel model with concurrent data collection and analyses was employed for the evaluation of the project.
Data analysis for the focus group was conducted in three steps: (1) open coding, (2) axial coding, and (3) selective coding (Strauss, 1987).
Tools
Post-Experience Focus Groups with interprofessional team members
Index of Interprofessional Collaboration (IIC) (Bronstein, 2002)

19. Geriatric Consultation Clinic Patient Outcomes Evaluation
Evaluation Methods
Quantitative comparative analysis:
Baseline at time of consult
8-12 weeks after consult
6 months after consult
Data Collected
Mini-Mental State Exam (MMSE)
Zarit Caregiver Burden Scale
Understanding of Medication Usage & Side Effects
FS-8 Health Survey

20. Binghamton University Community Enhancement Initiative in Aging Preparedness
Purpose
Binghamton University’s HPPAE in place for 5 years
Project placed HPPAE Fellows in non-MSW agencies
Assess impact of program upon participating organizations and overall community
Methods
Individual interviews with agency administration
Focus group with task supervisors

21. Example #3 University of Louisville

22. Overview: Geriatric Evaluation & Self-Management (GEMS) Project
Tested effectiveness of:
Interdisciplinary geriatric home-based assessment
Self-management support services
Target population:
Community dwelling older adults with 1+ chronic illnesses
Providers:
Interdisciplinary teams of social workers and physical therapists (students and professionals) collaborating with PCPs

23. EXAMPLE #4 BOSTON UNIVERSITY

24. Center for Aging and Disability Education and Research
CADER provides comprehensive, validated, skill-based online training programs to ensure that workers have the knowledge and skills needed to meet the challenges in a rapidly changing health and social services landscape. Demonstrating that learners have mastered the competencies needed for effective practice is a key objective of our program. Competence requires professional judgment and critical thinking to translate knowledge, skills, and values into effective practice behaviors.

25. Evaluation Tools
Pre and Post Competency Self-Assessment of Individual Courses
Participants complete pre- & post-test using the approximately 20-item instrument that measures the respondent’s perceptions of their skill level on a scale of 0 to 4 (0=not skilled at all; 4=expert skill)
Course Evaluation
This evaluation includes questions about the effectiveness of the online course, including whether course objectives were clearly described and met, the quality and applicability of the course, whether the course expanded learner knowledge and understanding of the topic area, and whether the training program will help them to apply practice skills in the topic area.

26. Evaluation Tools Program
Included in the evaluation are questions related to the training program from both the perspective of the individual learner and in relationship to their organization:
Individual learner: these include questions related to the expansion of knowledge, development of new skills, and the ability to do a better job as a result of the training program.
Supervisor expectations, as well as organizational support are also included.
In order to assess the degree to which new learning is maintained, evaluation is repeated at 3 months following completion of the program

27. EXAMPLE #5 FLORIDA ATLANTIC UNIVERSITY

28. IPE & SAGE Programs
The School of Social Work, College of Nursing, and College of Medicine at Florida Atlantic University developed an interpro­fessional curriculum to bring social work, nursing, and medical students together for collaborative practice-based learning. This interprofessional curriculum included two compon­ents: (a) the Interprofessional Education (IPE) program, and (b) the Senior Aging and Geriatrics Educator (SAGE) Mentoring program.

29. IPE & SAGE Programs Pretest/posttest evaluation (IPE Session 1)
(b) Teamwork IQ Quiz
(c) reflective journaling (after the IPE & SAGE Mentoring programs)
(d) follow-up assignments (after the IPE & SAGE Mentoring programs)
(e) surveys before and after the IPE and SAGE Mentoring programs to measure the effectiveness of the programs and reinforce learning from these experiences.

30. HPPAE Model is Competency Based
Gero Social Work Competencies
Developed by social workers and aging experts nationally
Field tested at demonstration sites
Aligned with the 2008 CSWE Education and Accreditation Standards (EPAS)
Competencies are measurable practice behaviors—what all graduates are able to do—that are comprised of social work knowledge, skills, and values learned in foundation class and field curriculum.

31. Gero Social Work Competencies
Five Domains of practice
Values, Ethics & Theoretical Perspectives
will be used as example to connect to EPAS
Assessment
Intervention
Aging Services, Programs & Policies
Leadership in Practice (new domain recently added)
Each domain has ten competencies (50 total)
Gero SW Competencies considered Practice Behaviors under new EPAS

32. Benefits of Geriatric Social Work Competencies (Practice Behaviors)
Addresses practice at BOTH micro and macro levels
Direct practice
Policy
Research
Advocacy
Recognizes social workers demonstrating leadership in aging need to be proficient in both micro and macro levels
Geriatric social workers providing direct service must be proficient in:
identifying service gaps
assessing community needs
developing programs to help older adults and caregivers cope with aging
Macro practitioners must be:
knowledgeable about the challenges older adults and families experience
able to identify older adult and family strengths and assets

33. BRAINSTORMING AND RESOURCE- SHARING

34. Participant Discussion
Describe the current or potential Inter-professional Geriatric Programs at your institution
How are you currently or what plans do you have to evaluate the program(s)?
What successes and barriers have you faced in this evaluation or its planning?

35. Contact Information
Paul Gould, LCSW, PhD Kris Hash, LICSW, PhD