VFC Site Visit Questionnaire and AFIX as Tools for Quality Assessment David Bibus, MPH Betsy Hubbard, RN, MN Lauren Greenfield, BSN, RN Krista Rietberg, MPH Jeff Duchin, MD Kyle Yasuda, MD Public Health – Seattle & King County 39th National Immunization Conference March 23, 2005

Objectives Use an enhanced VFC site visit questionnaire to assess quality of immunization practice. Examine association between quality of immunization practice and measurable immunization outcomes. Learn about potential quality improvement and cost-benefit of peer based education

Vaccines for Children (VFC) in King County Currently 307 clinic sites enrolled in VFC All children eligible for VFC in WA Public Health – Seattle & King County (PHSKC) has been conducting VFC site visits since 1995.

VFC Site Visit Team Public Health Nurse AFIX coordinator VFC program representative

Enhanced Questionnaire 2004-05 Thirty-one additional questions, focus on immunization practices and quality improvement Questions based on: Teaching points from previous visits Temperature incidents and storage problems Data collection for ongoing study

VFC Site Visit AFIX Assessment modified 2004-05 Follow-up letter 50 charts Consecutive method chart pulling 3-18 months Follow-up letter Addresses findings and recommended changes specific to clinic site

Methods Data collected from 73 sites in 2004, 25 pediatric and 48 family practice 3,174 charts reviewed Up-to-date status Immunization delay Site visit questionnaire Quality issues

If we are going to give the shots, let’s make sure the vaccines are good and that they are given on time.

What Did We Find?

Vaccine Storage and Handling 21% did not have protocols for power outages or temperature incidents 49% did not have a written list of “emergency responders” 47% did not protect MMR/VAR from light 51% did not check temperatures frequently

Immunization Practices and Policies - page 1 58% limit the number of injections Of these 78% limit due to provider preference or clinic policy 44% do not have an adverse reaction protocol posted or accessible 60% do not document all VFC-required elements

Immunization Practices and Policies – page 2 Invalid Contraindications: 32% do not immunize while on antibiotics 27% do not immunize with low grade fever or infectious disease exposure 22% do not immunize with recent acute illness 21% do not immunize with mild diarrhea 11% do not immunize with a cold

Immunization Practices and Policies – page 3 58% do not have a recall system 40% use 5/8” needle for IM injections in <2 month olds 29% use 5/8” needle for IM injection in >2 month olds

Up-to-Date at 12 Months Clinic mean 85%, median 89% Range 17%-100% 77% of clinics had UTD rates of 80% or greater 44% of clinics had UTD rates of 90% or greater

Does Up-to-Date = Quality? Clinics ≥ 80% UTD at 12 months: Generally require well child visits* Promote immunizations in their communities* Less likely to be delayed for immunizations*

Does Up-to-Date = Quality? Clinics ≥ 80% UTD at 12 months may tend to: Be pediatric practices Have system for ensuring vaccine stock current Have written protocols for: Power outages Ordering Inventory control Trained back-up person Contraindications to immunizations

Does Up-to-Date = Quality? Clinics ≥ 80% UTD at 12 months Check the immunization record at each visit Have access to immunization resources, e.g. Pink Book, Red Book, ACIP statements, internet access May be more likely to immunize: With a cold With mild diarrhea While on antibiotics

Dose not given within recommended age range. Immunization Delay Dose not given within recommended age range.

Benefits of Focusing on Immunization Delay Long periods of susceptibility to VPDs even if up-to-date (UTD) More precise measurement of status Quick response opportunity to make practice changes Practice change measured in a shorter period of time Underlying trends more visible Predictive value of delay on UTD status

Immunization Delay Clinic mean 32%, median 31% Range 4%-86% 80% of clinics had a delay in greater than 20% of their patients 52% of clinics had a delay in greater than 30% of their patients

Immunization Delay and Quality Clinics less than 30% delay Complete documentation* ≥ 90% UTD at 12 months* Pediatric practice May be more likely to immunize: With a cold With mild diarrhea Recent acute illness May be less likely to limit number of injections *statistically significant

Immunization Delay and Quality Clinics with less than 30% delay: Written protocols for: Contraindications for immunizations Immunization schedule Access to immunization resources Have a recall system May be less likely to have had out of range temperatures in the previous three months

VFC Site Visit Followed-up with TIPS

TIPS Peer based educational model One hour educational session focused on quality improvement Presented by a physician and public health nurse Presented to all clinic staff Focus on quality improvement and immunization delay Evaluation to be completed in 2006

Potential cost savings from improved immunization practice Reduction in vaccine storage incidents ex. $161,000 = 3 years storage incidents in King County (but cut by half in 2004). Incorrect needle length, revaccination cost ex. $442,000 Hepatitis B, 0-2 years of age per year estimate vaccine only in King Co. Improved child immunization rates and reduction in delay pts; Up to $27 saved for each dollar spent on immunization.

Conclusions Besides getting shots into kids’ arms, we also need to ensure the safety and effectiveness of vaccine being administered Emphasis on immunization coverage rates should not minimize the need to address issues related to quality. A focus on quality issues may lead to cost savings High immunization rates may not necessarily be associated with quality clinical practices

Conclusions A focus on delay as well as up-to-date status can provide important additional feedback for a clinic practice Adding questions to VFC site visit creates systematic opportunity for giving feedback and follow-up training about quality issues. AFIX Coordinators may want to consider adding quality improvement questions to the site visit tool Additional study with larger sample size needed

Limitations 73 sites = small sample size Many clinics do not document birth dose of Hepatitis B vaccine Delay may be overestimated Immunization registry not accessible all sites 77% of babies born in King County hospitals actually receive birth dose VFC site visit interview is often with only one clinic staff member

Contact Information david.bibus@metrokc.gov betsy.hubbard@metrokc.gov (206) 296-4774 Vaccineinfo@metrokc.gov