1. National Immunization Partnership with the APA: A Quality Improvement Project Specific to HPV Vaccination (NIPA) Wave 2 CDC Grant # 1H23IP000950

2. AGENDA Attendance Introductions What to expect from this project… Project overview  Timeline  Monthly Learning Collaborative calls  Study Materials Questions?

3. NIPA Wave 2 CORNET QI Team Monthly Learning Collaborative Call QI Leads: –Cynthia Rand, MD, MPH –Paul Darden, MD –Stanley Schaffer, MD, MS –William Stratbucker, MD, MS Other Project Faculty: –Peter G. Szilagyi, MD, MPH –Sharon Humiston, MD, MPH –Janet Serwint, MD CORNET Research Assistant: –Holly Tyrrell, MSSW

4. Project Goals Develop a sustainable strategy to train residents and faculty about optimal adolescent immunization methods Contribute to increased rates of HPV vaccination Provide excellent QI training

5. Background & Context Studies show that what residents learn during training informs how they later practice Teaching residents skills and approaches can improve adolescent immunization delivery There are approximately 200 Pediatric residency training programs in the US Pediatric faculty need to be trained in order to provide effective training to residents The best training occurs in the context in which care is delivered, i.e. immunization delivery in clinic.

6. Specific Aims Develop and execute a national QI project addressing adolescent immunizations with a focus on HPV Identify and test strategies to improve health care delivery of HPV vaccines to adolescent males and females

7. QI Practice Change Strategies All sites incorporate Strong Provider Recommendations Practices can select 1 or more QI strategies to use: Prompts to providers to give vaccine at the time of the visit; specifically prompts by nurse/staff at ALL visits (including acute visits) Standing Order protocols that empower office personnel to vaccinate, if a patient is due, without direct doctor involvement at the time of patient care at ALL visits Reminder Recall systems to notify members of the target population that vaccinations are due or overdue Feedback from monthly chart reviews to assess process measures

8. Faculty Responsibilities Obtain IRB approval or equivalent letter for a quality improvement (QI) project from your institution Participate in Orientation and Monthly Learning Collaborative teleconferences over 6 month project. Faculty must attend at least 5 LC calls to receive MOC credit. Encourage resident participation – form a group of resident champions for project Work with residents to pull patient charts for baseline chart review data (96 charts reviews total) Work with residents to pull interim charts and reports for monthly chart reviews during QI Learning Collaborative (16 chart reviews per month) Attest resident MOC participation and completion Each site asked to keep a running record of practice changes and process assessments Learning Collaborative Plan

9. ACGME: Practice-based Learning & Improvement Residents must demonstrate the ability to investigate and evaluate their care of patients, to appraise and assimilate scientific evidence, and to continuously improve patient care based on constant self-evaluation and life-long learning. Resident Learning Requirement:

10. MILESTONES: - Practice based learning and improvement - Systematically analyze practice using quality improvement methods, and implement changes with the goal of practice improvement MOC for Residents: - Residents can now earn Part 4 MOC credit during residency for meaningful participation in QI activities. - Resident MOC credit will be “in the bank” for when they become certified. It will then be applied to their first MOC cycle. Resident Learning Requirement:

11. Resident Responsibilities Attend/Review Orientation Call and QI Education Training Modules Participate in chart reviews:  Review at least 16 charts for baseline chart review  Review at least 16 charts (1 month) for monthly chart reviews Participate in at least 5 monthly Learning Collaborative calls or review recordings Participate in review of Monthly Data Reports Brainstorm with clinic QI team as to how to make change most effective Utilize MOC tracking log to record participation throughout project Learning Collaborative Plan

12. Benefits to Participating Receive 25 credits toward MOC, Part 4, after meeting all requirements (faculty and residents) Participate in a national QI project for faculty and residents Enhanced education in QI principles Participate in a scholarly project Receive own site-specific data Site stipend for completing the project

13. Partnership with NIPN National Improvement Partnership Network (NIPN) Based at University of Vermont A network of over 20 State Improvement Partnerships developed to advance quality and transform healthcare for children and their families NIPN will conduct a similar project to CORNET across 11 State Improvement Partnerships: Alabama Arizona Iowa Indiana Florida Maine New Hampshire New Jersey Tennessee Vermont Massachusetts

14. NIPA Participation Map Wave 2 CORNET – 15 clinics NIPN – 11 states Wave 1 CORNET – 14 clinics NIPN – 6 states

15. Project Timeline March - April 2016 Orientation Phase Office Systems Inventory Practice Readiness Assessment Survey IRB QI proposal submissions Contact Form April 2016 Training Phase Orientation Call – Project Overview & Data Collection Begin Baseline Chart Reviews (96) & Patient Tally Form QI Module Training – 4 parts (about 60 minutes) Intro, Leading Change, AIM Statements, QI Measures Strong Provider Recommendation Training (about 10 minutes) May - October 2016 Learning Collaborative Phase Monthly Learning Collaborative Calls Monthly PDSA Self-Assessments Monthly Chart Reviews (16/month) Monthly Chart Review Patient Tally Form NIPA Staff Impact Survey (Midpoint) October 2016 Wrap-Up Phase NIPA Staff Impact Survey (Endpoint) Office Systems Inventory (Endpoint)

16. Responsibilities of Site Participants PRE-Implementation of Practice Change Participate in Orientation Call Obtain IRB approval from each institution Complete the Practice Readiness Assessment* Complete the Office Systems Inventory* Complete QI Education Training Modules Establish Baseline - review a total of 96 charts *Data entry online to be completed via REDCap. A site specific link will be provided to each site

17. Responsibilities of Site Participants Implementation of Practice Change Important dates: Monthly Data due: 5 th day of the month Monthly Calls scheduled: 3 rd week of the month May 2016 – October 2016 Learning Collaborative Phase: - Monthly Learning Collaborative Calls - Monthly PDSA Self Assessments: Barriers encountered and lessons learned - Monthly Chart Reviews (16/month) for process measures: missed opportunities - Monthly Chart Review Patient Tally Form - Staff Impact Survey (Midpoint)

18. Responsibilities of Site Participants POST-Implementation of Practice Change MOC attestation: Use MOC Tracking Log on Virtual Toolkit Instructions will be sent towards the end of the series of LC calls October 2016 Wrap Up Phase - Post-Office Systems Inventory - Staff Impact Survey (Endpoint) - MOC attestation

19. Patient Population Baseline (96) Chart Reviews: Patients age 11-17 years, 48 males/48 females Any visit reason (health maintenance, chronic disease, acute care) in 6-month timeframe ONLY IF Eligible for HPV vaccine at the visit Follow randomization scheme Retrospective: 6 months prior to intervention (November 1, 2015 – April 30, 2016) Monthly (16 charts/month) Chart Reviews: Patient age 11-17 years, 8 males/8 females Any visit type in the prior month ONLY IF Eligible for HPV vaccine at the visit Follow randomization scheme Prospective: 6 months of Intervention (May 1, 2016 – October 31, 2016)

20. Learning Collaborative Plan Tools to help you navigate Project Timeline A timeline to help move you through the project Listserv address A way to communicate with your fellow project colleagues NIPAWave2_QI@listserv1.academicpeds.org NIPAWave2_QI@listserv1.academicpeds.org Virtual Toolkit An easy way to reference all your project materials www.academicpeds.org/NIPA www.academicpeds.org/NIPA REDCap for all data entry Secure link to enter data (site specific)

21. All data will be entered electronically via REDCap All data collection and survey tools are at the clinic-level, not the provider level Main faculty contact for each site will receive email with clinic-specific links to access data collection tools & can share these with clinic team to coordinate data entry Do not send any forms or medical record information to the NIPA project team Data Submission

22. REDCap Tips Most fields are required and you cannot move to the next page without completing all required fields. You can save partially- completed data and return later to complete. SAVE the ‘Return Code’ to re-enter and complete data collection. Once you have completed your data entry, click ‘submit’, and SAVE the new ‘Return Code’, in case you need to re-enter and edit submission. You are given 2 Return Codes: 1) When you partially complete data entry 2) When you submit data Keep the final one in case the Project Team has a question about your data.

23. Your site will receive a link to complete this survey pre- intervention Purpose: Assess practice experience and comfort with QI to guide practice coaching Practice Readiness Assessment

24. Your site will receive a link to complete this survey pre- and post-intervention Purpose: Describe practice’s current systems for delivery of HPV vaccine and identify potential areas for system change It is a good idea to discuss survey questions with your clinic team before completing the survey Office Systems Inventory

25. Your site will receive an email with detailed instructions and a link to enter data in REDCap Purpose: Calculate rates of missed opportunities, HPV series initiation and completion Baseline Chart Review 96 charts total 48 males 48 females Can be completed at site’s convenience throughout the project

26. Eligible patients: Adolescents 11-17, seen for any visit type between November 1, 2015 – April 30, 2016 You will need: 1)List of all 11-17 year old patient visits during baseline time period (NOT alphabetical or by provider, but in consecutive order). 2)Baseline Patient Tracking Log 3)Baseline Chart Review Patient Tally Form Baseline Chart Review

27. Your site will receive a link to enter data in REDCap during each month of Learning Collaborative Purpose: Assess progress, evaluate system changes, evaluate missed opportunities on a monthly basis 8 charts per measurement period (16 charts per month) Measurement period defined as 2 periods within a give calendar month: Period A: Days 1-14 Period B: Days 15-31 Monthly Chart Review

28. Eligible patients: Adolescents 11-17 year old, seen for any visit type during measurement period (Period A: Days 1-14, Period B: Days 1- 31) You will need: 1)List of all 11-17 year old visits during measurement period (NOT alphabetical or by provider, but in consecutive order). 2)Monthly Chart Review Patient Tally Form Monthly Chart Review

29. Your site will receive a link to complete this survey in REDCap during each month of Learning Collaborative Purpose: Reflection on Plan-Do-Study-Act cycle Record successes and barriers you encounter, examine the progress you have been making through the strategy you selected, and make changes that will lead to maximal improvement of your immunization rates Helps to guide discussion on monthly LC calls PDSA Self-Assessment Tool

30. Monthly Data Report

31. Your site will receive a link to complete this survey in REDCap mid- and post-intervention Purpose: Balancing measure to evaluate whether the time and effort required to implement office systems interventions affect staff in planning, scheduling, and conducting visits for adolescents who are due/overdue for HPV vaccine Staff Impact Survey

32. Any questions? Please contact : Holly Tyrrell, MSSW Phone: 703/556 9222 x113 I Fax: 703/556 8729 hollyce@academicpeds.org Listserv: NIPAWave2_QI@listserv1.academicpeds.org NIPAWave2_QI@listserv1.academicpeds.org