1. Promoting Quality Prevention Counseling Project: What have we learned? Spring 2005 Texas Tour Dallas, Fort Worth, Houston, Midland, Austin

2. Agenda Welcome and Introduction Welcome and Introduction Background Background Overview of project Overview of project Implementation experiences by sites Implementation experiences by sites General evaluation findings General evaluation findings Next Steps Next Steps Q & A Q & A

3. Background Revised HIV Counseling, Testing and Referral (CTR) Guidelines, November 2001 Revised HIV Counseling, Testing and Referral (CTR) Guidelines, November 2001

4. Background Contd CDCs Project RESPECT: Evidence-based intervention showing significant reduction of STDs with protocol-based HIV prevention counseling CDCs Project RESPECT: Evidence-based intervention showing significant reduction of STDs with protocol-based HIV prevention counseling RESPECT-2: Refined HIV prevention counseling protocol & further developed counseling quality assurance methods RESPECT-2: Refined HIV prevention counseling protocol & further developed counseling quality assurance methods

5. RESPECT Methodology 5758 heterosexual, HIV-negative patients older than 14 years who came in for STD examination 5758 heterosexual, HIV-negative patients older than 14 years who came in for STD examination Five public STD clinics (Baltimore, Denver, Long Beach, Newark and San Francisco) Five public STD clinics (Baltimore, Denver, Long Beach, Newark and San Francisco)

6. Project RESPECT Results *: HIV Prevention Counseling Effective Kamb, M.L., et al (1998) Efficacy of risk-reduction counseling to prevent human immunodeficiency virus and sexually transmitted diseases, JAMA, 280 (13):1161-1167 (*p<0.05)

7. How could we translate this intervention into a real-world setting? Risk Reduction Specialist support Risk Reduction Specialist support Supervisor support Supervisor support Practical tools Practical tools

8. Goals of the Project Develop and evaluate tools to support protocol-driven prevention counseling based on the RESPECT model Develop and evaluate tools to support protocol-driven prevention counseling based on the RESPECT model Develop and evaluate QA procedures Develop and evaluate QA procedures Better understand the barriers and facilitators of good prevention counseling Better understand the barriers and facilitators of good prevention counseling

9. Definitions Evidence-based interventions Evidence-based interventions Interventions that have demonstrated desired outcomes through rigorous research Interventions that have demonstrated desired outcomes through rigorous research Core elements Core elements Components of the intervention that are believed to be essential to achieve the desired behavior change Components of the intervention that are believed to be essential to achieve the desired behavior change Protocol Protocol A structured approach to achieve core elements A structured approach to achieve core elements

10. Definitions Client-centered prevention counseling Client-centered prevention counseling One-on-one interactions with risk-reduction as its primary goal One-on-one interactions with risk-reduction as its primary goal Risk Reduction Specialist Risk Reduction Specialist A trained specialist responsible for maintaining the focus on a clients specific risk reduction needs A trained specialist responsible for maintaining the focus on a clients specific risk reduction needs Tools Tools Job aides to ensure fidelity to core elements of protocol Job aides to ensure fidelity to core elements of protocol

11. cc Tarrant County Health Dept. Ft. Worth, TX Resource Center Of Dallas Dallas, TX Valley AIDS Council, McAllen, TX Brownsville, TX City of Laredo Health Dept. Laredo, TX Site Locations

12. What was introduced during the project? Counseling protocol Counseling protocol Training on the protocol Training on the protocol Counseling tools Counseling tools Spiral book with goals and sample questions Spiral book with goals and sample questions Laminate wheel Laminate wheel Documentation form with space for RR plan and referrals Documentation form with space for RR plan and referrals Personal review form Personal review form

15. Session Documentation Form

16. What QA activities were part of the project? QA protocol QA protocol Emphasizing standardized preceptorship, observation, routine meetings, documentation review, and feedback on observations and documentation Emphasizing standardized preceptorship, observation, routine meetings, documentation review, and feedback on observations and documentation QA tools QA tools Supervisor observation tool Supervisor observation tool RRS self assessment RRS self assessment Chart abstraction and summary tools Chart abstraction and summary tools

17. Quality Assurance Tools

18. Comparison of Core Elements GOALPCPERESPECTProtocol 1:1 CounselingYes 2-Session ModelYes Follow ProtocolNoYes Focus: Client Risk BehaviorYes Increase self-perception RiskYes Negotiate Realistic RR StepYes Provide ReferralsYes Support Test DecisionYes Interpret test resultsYes Partner Elicitation/ ReferralYesNoYes Standardized QA ProcedureNoYes QA Tools SpecifiedNo Yes Session ToolsNoYes

19. Implementation Training developed for protocol, tools and QA Training developed for protocol, tools and QA Supervisors and Risk Reduction Specialists trained in October 2003 Supervisors and Risk Reduction Specialists trained in October 2003 On-site and off-site TA provided for start- up On-site and off-site TA provided for start- up Staggered and tailored implementation of protocol, tools, and QA Staggered and tailored implementation of protocol, tools, and QA Additional ongoing TA after start-up Additional ongoing TA after start-up

20. What Do We Want to Learn? Can you implement protocol-based prevention counseling with existing resources? Can you implement protocol-based prevention counseling with existing resources? Did the protocols and tools help them implement the intervention with fidelity? Did the protocols and tools help them implement the intervention with fidelity? What were some of the facilitators and barriers of the implementation of the protocols? What were some of the facilitators and barriers of the implementation of the protocols?

21. Evaluation Design Two data collection phases: Pre- and Post- intervention Two data collection phases: Pre- and Post- intervention Evaluation data triangulation: 9 complementary quantitative and qualitative instruments Evaluation data triangulation: 9 complementary quantitative and qualitative instruments

22. Quantitative Instruments Supervisor time logs (pre and post) Supervisor time logs (pre and post) Client Questionnaires (pre and post) Client Questionnaires (pre and post) Counseling chart reviews (pre and post) Counseling chart reviews (pre and post) Observations of counseling by evaluators (pre and post) Observations of counseling by evaluators (pre and post)

23. Qualitative Instruments Risk Reduction Specialist (pre and post) Risk Reduction Specialist (pre and post) Supervisors (pre and post) Supervisors (pre and post) Site Program Managers (post only) Site Program Managers (post only) WAP (post only) WAP (post only) TDH (post only) TDH (post only)

24. Results Lessons Learned

25. Did the protocols and tools help implement the intervention with fidelity?

26. Observations: Initial Session Goals *P<.05

27. Observations: Follow-up Session Goals *P<.05

28. Client survey: Initial session Goals *P<.05

29. Client survey: Initial session Goals (contd) *P<.05

30. Client survey: Follow-up session Goals *P<.05

31. Client Surveys: Client Participation *P<.05

32. Chart reviews Goals *P<.05

33. 33 Significant changes seen in initial sessions GoalsObservation Client Report Documentation Introduction/Orientati on X Enhanced risk percept. XX Recent risk discussed XXX Reviewed past RR XX Sum up pattern of risk XX Risk reduction plan X Support and referral X X (2) X Test decision counseling X Appt and reminder XX

34. 34 Significant changes seen in follow up sessions GoalsObservation Client Report Documentation Orient and give results X Review RR efforts XX Risk reduction plan XX Support and referral XXX Summarize and close X

35. That you had to follow every single task even though they didn't all apply to everybody [is a problem]. Protocol doesn't allow for individual counseling styles or use of skills RRS's have received at prior trainings…it seems cumbersome and redundant to use this protocol with clients with very few risks - although it's easy enough to move through the protocol by saying this doesn't really apply to you [for certain tasks]. -Risk Reduction Counselor

36. [The protocol improved the quality of my counseling] because I had a structure to make sure I wasnt leaving anything out. -Risk Reduction Counselor

37. Observations: Initial Visit Use of Counseling Skills *P<.05

38. Observations: Follow-up Session Use of Counseling Skills *P<.05

39. Client Surveys: Client experience *P<.05

40. What were some of the facilitators and barriers of the implementation of the protocols?

41. Overall themes and feedback Delivery of protocol-driven prevention Delivery of protocol-driven prevention Provided structure Provided structure Improves with practice Improves with practice Aided in identifying risk behaviors and patterns Aided in identifying risk behaviors and patterns Protocol questions felt rigid Protocol questions felt rigid More training and TA is essential More training and TA is essential

42. Overall themes and feedback (contd) Spanish version tools are needed Spanish version tools are needed Supervisor buy-in is essential Supervisor buy-in is essential Supervisors other responsibilities need to be considered due to time constraints of quality assurance Supervisors other responsibilities need to be considered due to time constraints of quality assurance Difficult with certain clients (such as low risk, outreach, drug treatment and jail) Difficult with certain clients (such as low risk, outreach, drug treatment and jail)

43. Using the Counseling Tools Most of the RRS found the cards to be the most helpful of the tools (72%) Most of the RRS found the cards to be the most helpful of the tools (72%) Cards help ensure you cover everything in order (44%) Cards help ensure you cover everything in order (44%) Wheel was not as helpful (83%) Wheel was not as helpful (83%) Spanish version of the tools is needed Spanish version of the tools is needed

44. Counseling QA by RRS Most helpful Most helpful Observation by supervisor Observation by supervisor General feedback General feedback Role play Role play Observation by peer/peer observation/document review Observation by peer/peer observation/document review Case conference Case conference

45. Counseling QA by RRS (contd) Least helpful Least helpful Documentation review Documentation review Observation by peer Observation by peer Case conference/self assessment Case conference/self assessment

46. Counseling QA by Supervisor Most helpful Most helpful Observation by supervisor Observation by supervisor General feedback General feedback Document review Document review Case conferences Case conferences Least helpful Least helpful Observation by peer Observation by peer Self-assessment Self-assessment

47. QA Activities by RRS

48. [The supervisor observation form] is better because it is less subjective and more structured. …the priorities of the tasks are made clear by the forms and that feedback using these forms makes the whole process self-reinforcing [the process of understanding the expectations of the protocol, using the protocol, and getting feedbackall have the same language, structure, and expectations]…the new feedback is less stressful for everybody, including the observer for the stated reasons. ---Risk Reduction Specialist

49. I think this new protocol is greatfabulous! Before when they first told us about the program and we went to training, we were all iffy and said its not gonna work no way in heck it would be accepted by the people. Now that we are implementing it, we are doing a great job. When you have to write steps, the clients leave with RR plan in hand, a referral, an appointment card with the date on it in hand. As for review forms used by the supervisor on documentation, etc. You have the form yourself to be able to discuss met or not met. ---Risk Reduction Specialist

50. What has been done? Changes to training Changes to training Trainers have bought in Trainers have bought in Preceptorship is done first, then attend training Preceptorship is done first, then attend training More time for role play More time for role play Develop their own questions for each step Develop their own questions for each step Not a gripe session Not a gripe session Sites learning from each other Sites learning from each other Role plays/Peer observations for practice Role plays/Peer observations for practice Sign in waiting room for length of sessions Sign in waiting room for length of sessions Regularly scheduled QA sessions Regularly scheduled QA sessions

51. What Now? State-wide roll-out begins May 2, 2005 State-wide roll-out begins May 2, 2005 All DSHS HIV/STD contractors All DSHS HIV/STD contractors Roll-out completed by July 2006 Roll-out completed by July 2006 Protocol Based Counseling Training (PBCT) replaces PCPE by August 2006 as the state mandated prevention counseling course for risk reduction specialists Protocol Based Counseling Training (PBCT) replaces PCPE by August 2006 as the state mandated prevention counseling course for risk reduction specialists

52. HOW?? 11-week training and technical assistance cycles 11-week training and technical assistance cycles One month of supervisory training and development One month of supervisory training and development Two week employee preceptorship Two week employee preceptorship Three weeks of employee training and TA Three weeks of employee training and TA Two weeks of independent implementation and TA Two weeks of independent implementation and TA Ongoing technical assistance and support Ongoing technical assistance and support 7 training staff dedicated to rollout initially 7 training staff dedicated to rollout initially Field operations and regional staff support Field operations and regional staff support

53. When? Cycle 1 – May 2 Cycle 1 – May 2 SE Texas area agencies SE Texas area agencies Cycle 2 – July 18 Cycle 2 – July 18 Cycle 3 – September 19 Cycle 3 – September 19 Cycles 4-6 in 2006 Cycles 4-6 in 2006 Agency selection for cycle based on: Agency selection for cycle based on: Epi data Epi data Field Ops Field Ops Agency Readiness Agency Readiness

54. Programmatic PBC is the prevention counseling model that must be used if contracted to perform PCPE or an ILI as a component of your GLI. PBC is the prevention counseling model that must be used if contracted to perform PCPE or an ILI as a component of your GLI. Once committed - no going back to old PCPE. Once committed - no going back to old PCPE. Changes in the RFP and contract language. Changes in the RFP and contract language. o State 2005/6 o Federal 2006 Federal 2006 Big competitive RFP released Spring 2006 to start funding state 9/1/06 and federal 1/1/07 Big competitive RFP released Spring 2006 to start funding state 9/1/06 and federal 1/1/07

55. Points to Ponder… Structural - your program overall? Buy-in from your administration, capacity? Structural - your program overall? Buy-in from your administration, capacity? Staffing – Supervision requirements, hiring, vacancies, current staff Staffing – Supervision requirements, hiring, vacancies, current staff Time for supervision, time for staffing, time for QA activities, time to perform the sessions. Time for supervision, time for staffing, time for QA activities, time to perform the sessions. Who needs to be trained? PCM, EBI, TCADA, Case Managers? Who needs to be trained? PCM, EBI, TCADA, Case Managers?

56. Budgets Budgets – how to pay for upcoming trainings, how to compensate your staff? Budgets – how to pay for upcoming trainings, how to compensate your staff? Outreach workers conference, OraSure, salary savings Outreach workers conference, OraSure, salary savings One week of Austin training for supervisor and any team leaders. One week of Austin training for supervisor and any team leaders. All PCPE staff one week in possibly local area. All PCPE staff one week in possibly local area. Possible budget amendments Possible budget amendments Start thinking about new budgets for 2006 Start thinking about new budgets for 2006

57. Workplans Where are you doing your PC? What populations are you serving? Where are you doing your PC? What populations are you serving? What does your PC look like now? What does your PC look like now? How does this change the structure of the work that you do? How does this change the structure of the work that you do? What type of changes will you need to make in your workplan? What type of changes will you need to make in your workplan? Look at settings and time and how this will work? Look at settings and time and how this will work? Partner with your fellow providers to perform activities in various settings. Partner with your fellow providers to perform activities in various settings.

58. THINGS YOU CAN DO NOW! Look at the quality assurance guidelines and your contract. Are you doing as required now? Look at the quality assurance guidelines and your contract. Are you doing as required now? Review the PBC tools QA. Please do not use them until you are trained Review the PBC tools QA. Please do not use them until you are trained What does your orientation plan look like? Some do a type of preceptorship already. What does your orientation plan look like? Some do a type of preceptorship already. Are you using the TDH documentation guide and sample? Are you using a PCPE review tool? Get them off our web. Are you using the TDH documentation guide and sample? Are you using a PCPE review tool? Get them off our web.

59. Things to do now.. Review your objectives, what do you need to meet your return rates, link to EI, and PE? Review your objectives, what do you need to meet your return rates, link to EI, and PE? What does your counseling look like now? What does your counseling look like now? Clean RECN data Clean RECN data

60. TA and Monitoring FO staff will go through Mega-training with their program FO staff will go through Mega-training with their program TA provided with Training staff TA provided with Training staff Monitoring schedule will start six months after all trained date Monitoring schedule will start six months after all trained date

61. Q & A