1. Meeting the Intent of the Accreditation Standard: Making ResiTrak® Work for Your Residency Program Beth Bryles Phillips, PharmD, BCPS Jamie Kalus, PharmD, BCPS (AQ-Cardiology) Justin Hare, PharmD Akilah F. Strawder, PharmD, CDE, BCACP

2. Objectives 1.Discuss frequently cited items on residency accreditation survey reports and the intent of the ASHP residency accreditation standard. 2.Identify items within the ASHP residency accreditation standard that programs find challenging to implement in ResiTrak®. 3.Describe ways that Resitrak® configuration can be optimized to help residency programs meet challenging aspects of the ASHP residency accreditation standard.

3. I am a... PGY1 Residency Program Director (RPD) PGY1 Preceptor PGY2 RPD PGY2 Preceptor B C D A

4. XYZ Medical Center Residency Program Preceptors feel overwhelmed – Evaluations take forever to complete – Complain of “burn-out” RPD feels overwhelmed – ResiTrak® set-up taking hours to complete

5. How Can ResiTrak® Help Meet the Intent of the Standard? P ROGRAM D ESIGN Summative evaluations [4.1e(1)(a), 4.1e(1)(b), 4.1e(2)(b), 4.1e(3), 4.2d] – Scheduling evaluations – Overall Evaluation Status report – Send reminders – Track late evaluations

6. How Can ResiTrak® Help Meet the Intent of the Standard? P ROGRAM D ESIGN Formative Evaluations [4.1e(2)(a)] – Schedule snapshots – Custom evaluations Customization of Program [4.2b(1), 4.2e(2)] – ASHP Standard Entering Interests – ASHP Goal-based Entering Interests – Customized plan templates

7. Program Scenario Sally RPD is an enthusiastic, energetic practitioner who was just named RPD for the PGY1 Pharmacy Residency at XYZ Medical Center. She is taking over from the previous RPD who started a new position. Her first “project” as the RPD is to re-design the program based on the Accreditation Survey report which contained 27 partial compliance findings as a result of the site visit last month.

8. Accreditation Survey Finding The program is not adequately customized to individual resident’s needs. Further, changes to the overall program plan have not been made based on an assessment of the resident’s incoming knowledge, skills, attitudes and abilities and updated on a quarterly basis. [Items 4.2b(1) and 4.2e(2)]

9. Entering Interests and Goal-Based Self-Assessment The new residency class will be starting in one month. The RPD has met with the Residency Advisory Committee and has several ideas. She now needs to build the changes into ResiTrak®. What tools can be used to design the customized plan for each resident?

10. Initial Customized Plan The first 3 weeks of the residency have passed. The residents are completing their orientation rotation. Based on the resident’s self-evaluation and the preceptor/RPD assessment of entering skills and abilities, how can the customized plan be developed?

11. Copying Residents’ Schedules The RPD is spending a lot of time mapping out the schedule for the new residents. Are there any tools that can help with resident scheduling?

12. Accreditation Survey Finding Preceptor summative evaluations are not completed at the end of block experiences and at least quarterly for all longitudinal experiences in a timely manner and some are not completed at all. [Items 4.1e(1)(a), 4.1e(1)(b), 4.2d(1)] How can the RPD ensure that summative evaluations are scheduled appropriately in ResiTrak®?

13. Snapshots and Custom Evaluations Preceptors do not provide residents with sufficient opportunities to complete documented, criteria-based, formative self- evaluations. [Item 4.1e(2)(a)] What tools in ResiTrak® are available to document formative evaluation?

14. How Can ResiTrak® Help Meet the Intent of the Standard? P ROGRAM A DMINISTRATION Evaluations [4.1e(1)(c), 4.2d(1), 4.2d(3), 5.10] – Overdue Tasks report – Overall Evaluation Status report

15. How Can ResiTrak® Help Meet the Intent of the Standard? P ROGRAM A DMINISTRATION Customized Plan Updates and Tracking [4.2b(1), 4.2e(1), 4.2e(2)] – Resident Progress to Date report – Goals Achieved for Residency – Needs Improvement (NI) report – New Customized Plan Templates

16. Accreditation Survey Finding Preceptors provide verbal feedback to residents on a regular basis; however, some preceptors do not provide written, criteria-based summative feedback to assist residents in improving their performance. [Items 4.2(d)(1) and 5.10]

17. “Written, criteria based summative feedback” The new RPD emphasized the importance of providing quality and timely written feedback to residents through a series of preceptor development activities. How can the RPD use Resitrak® to monitor whether these activities have led to an improvement in the quality and timeliness of feedback?

18. Updating the Customized Plan It is now the end of the 1 st quarter and time for the new RPD to update her residents’ customized plan. What reports are available in Resitrak® that could help her to identify areas of strength or weakness that could prompt changes to the residents’ customized plan?

19. Updating the Customized Plan Now that she has reviewed reports for her residents, how can the RPD use Resitrak® to update the Customized Plans?

20. How Can ResiTrak® Help Meet the Intent of the Standard? P ROGRAM Q UALITY A SSURANCE Preceptor and Learning Experiences [Item 4.3a, 4.3c] – Average Score for all Preceptors and Learning Experiences – Preceptor and Learning Experience Summary reports – N/A report – Send Back for Edit report

21. How Can ResiTrak® Help Meet the Intent of the Standard? P ROGRAM Q UALITY A SSURANCE Ensure all educational goals assigned to learning experiences [Items 4.1b, 4.1c(1), 4.1c(4)] – Taught/Taught and Evaluated (T/TE) Table Program Purpose [Item 4.4] – Graduate Tracking Report

22. Accreditation Survey Finding A plan for improving the quality of preceptors’ instruction, based on an assessment of residents’ written evaluations of preceptors’ performance and other sources, is not developed adequately. [Items 4.3a and 4.3c]

23. Evaluating Preceptors and Learning Experiences The Residency year has ended, and prior to the next class starting, the RPD would like to meet with each preceptor to develop an individual preceptor development plan and implement changes to learning experiences based on an assessment of residents’ written evaluations. How can the RPD use ResiTrak® to optimize this process?

24. Evaluating Preceptors and Learning Experiences Even with repeated reminders, some preceptors still quickly fill out the summative evaluations and submit them to be cosigned. Many of these were even sent back to them for revisions along with a request for additional comments. The RPD would like to fairly identify these preceptors and provide them with additional training.

25. Accreditation Survey Finding The design of the program structure does not ensure that all of the program’s required goals and objectives for the residency are included and evaluated during one or more of the program’s required learning experiences. [Items 4.1b, 4.1c(1), and 4.1c(4)]

26. Assigning Educational Goals During a summer preceptors’ meeting, preceptors complained that their evaluations seemed very lengthy and thought that some of the residents were evaluated on the same goals excessively. They would like their evaluations to be shorter. How can the RPD ensure that each resident is evaluated on ALL required goals (and program selected elective goals)?

27. Accreditation Survey Finding The RPD, through employment and other career information of residency graduates, does not evaluate whether the residency produces the type of practitioner described in the program’s purpose statement. [Item 4.4]

28. Graduate Tracking In response to the recent survey report, the RPD reviews the program’s purpose statement. After reading it, she would like to develop a method for tracking residents, post-graduation, to ensure the program produces the type of graduate described in the purpose. How can she utilize ResiTrak® to assist her with this task?

29. What ResiTrak® Does NOT Do... Run reports Define the overall program plan Provide effective feedback Complete evaluations Assign educational goals and objectives to learning experiences Evaluate preceptors

30. In Conclusion Become familiar with the new updates that have been incorporated in ResiTrak® since June 2012 Utilize ResiTrak® functionality and available reports to help meet challenging aspects of the ASHP Accreditation Standard Recognize that ResiTrak® can be used to help design, maintain, and optimize your residency program

31. Meeting the Intent of the Accreditation Standard: Making ResiTrak® Work for Your Residency Program Beth Bryles Phillips, PharmD, BCPS Jamie Kalus, PharmD, BCPS (AQ-Cardiology) Justin Hare, PharmD Akilah F. Strawder, PharmD, CDE, BCACP