1. VFC Site Visit Questionnaire and AFIX as Tools for Quality Assessment
David Bibus, MPH
Betsy Hubbard, RN, MN
Lauren Greenfield, BSN, RN
Krista Rietberg, MPH
Jeff Duchin, MD
Kyle Yasuda, MD
Public Health – Seattle & King County
39th National Immunization Conference
March 23, 2005

2. Objectives
Use an enhanced VFC site visit questionnaire to assess quality of immunization practice.
Examine association between quality of immunization practice and measurable immunization outcomes.
Learn about potential quality improvement and cost-benefit of peer based education

3. Vaccines for Children (VFC) in King County
Currently 307 clinic sites enrolled in VFC
All children eligible for VFC in WA
Public Health – Seattle & King County (PHSKC) has been conducting VFC site visits since 1995.

4. VFC Site Visit Team Public Health Nurse AFIX coordinator
VFC program representative

5. Enhanced Questionnaire 2004-05
Thirty-one additional questions, focus on immunization practices and quality improvement
Questions based on:
Teaching points from previous visits
Temperature incidents and storage problems
Data collection for ongoing study

6. VFC Site Visit AFIX Assessment modified 2004-05 Follow-up letter
50 charts
Consecutive method chart pulling
3-18 months
Follow-up letter
Addresses findings and recommended changes specific to clinic site

7. Methods
Data collected from 73 sites in 2004, 25 pediatric and 48 family practice
3,174 charts reviewed
Up-to-date status
Immunization delay
Site visit questionnaire
Quality issues

8. If we are going to give the shots, let’s make sure the vaccines are good and that they are given on time.

9. What Did We Find?

10. Vaccine Storage and Handling
21% did not have protocols for power outages or temperature incidents
49% did not have a written list of “emergency responders”
47% did not protect MMR/VAR from light
51% did not check temperatures frequently

11. Immunization Practices and Policies - page 1
58% limit the number of injections
Of these 78% limit due to provider preference or clinic policy
44% do not have an adverse reaction protocol posted or accessible
60% do not document all VFC-required elements

12. Immunization Practices and Policies – page 2
Invalid Contraindications:
32% do not immunize while on antibiotics
27% do not immunize with low grade fever or infectious disease exposure
22% do not immunize with recent acute illness
21% do not immunize with mild diarrhea
11% do not immunize with a cold

13. Immunization Practices and Policies – page 3
58% do not have a recall system
40% use 5/8” needle for IM injections in <2 month olds
29% use 5/8” needle for IM injection in >2 month olds

14. Up-to-Date at 12 Months Clinic mean 85%, median 89% Range 17%-100%
77% of clinics had UTD rates of 80% or greater
44% of clinics had UTD rates of 90% or greater

15. Does Up-to-Date = Quality?
Clinics ≥ 80% UTD at 12 months:
Generally require well child visits*
Promote immunizations in their communities*
Less likely to be delayed for immunizations*

16. Does Up-to-Date = Quality?
Clinics ≥ 80% UTD at 12 months may tend to:
Be pediatric practices
Have system for ensuring vaccine stock current
Have written protocols for:
Power outages
Ordering
Inventory control
Trained back-up person
Contraindications to immunizations

17. Does Up-to-Date = Quality?
Clinics ≥ 80% UTD at 12 months
Check the immunization record at each visit
Have access to immunization resources, e.g. Pink Book, Red Book, ACIP statements, internet access
May be more likely to immunize:
With a cold
With mild diarrhea
While on antibiotics

18. Dose not given within recommended age range.
Immunization Delay
Dose not given within recommended age range.

19. Benefits of Focusing on Immunization Delay
Long periods of susceptibility to VPDs even if up-to-date (UTD)
More precise measurement of status
Quick response opportunity to make practice changes
Practice change measured in a shorter period of time
Underlying trends more visible
Predictive value of delay on UTD status

20. Immunization Delay Clinic mean 32%, median 31% Range 4%-86%
80% of clinics had a delay in greater than 20% of their patients
52% of clinics had a delay in greater than 30% of their patients

21. Immunization Delay and Quality
Clinics less than 30% delay
Complete documentation*
≥ 90% UTD at 12 months*
Pediatric practice
May be more likely to immunize:
With a cold
With mild diarrhea
Recent acute illness
May be less likely to limit number of injections
*statistically significant

22. Immunization Delay and Quality
Clinics with less than 30% delay:
Written protocols for:
Contraindications for immunizations
Immunization schedule
Access to immunization resources
Have a recall system
May be less likely to have had out of range temperatures in the previous three months

23. VFC Site Visit Followed-up with TIPS

24. TIPS Peer based educational model
One hour educational session focused on quality improvement
Presented by a physician and public health nurse
Presented to all clinic staff
Focus on quality improvement and immunization delay
Evaluation to be completed in 2006

25. Potential cost savings from improved immunization practice
Reduction in vaccine storage incidents
ex. $161,000 = 3 years storage incidents in King County (but cut by half in 2004).
Incorrect needle length, revaccination cost
ex. $442,000 Hepatitis B, 0-2 years of age per year estimate vaccine only in King Co.
Improved child immunization rates and reduction in delay pts; Up to $27 saved for each dollar spent on immunization.

26. Conclusions
Besides getting shots into kids’ arms, we also need to ensure the safety and effectiveness of vaccine being administered
Emphasis on immunization coverage rates should not minimize the need to address issues related to quality.
A focus on quality issues may lead to cost savings
High immunization rates may not necessarily be associated with quality clinical practices

27. Conclusions
A focus on delay as well as up-to-date status can provide important additional feedback for a clinic practice
Adding questions to VFC site visit creates systematic opportunity for giving feedback and follow-up training about quality issues.
AFIX Coordinators may want to consider adding quality improvement questions to the site visit tool
Additional study with larger sample size needed

28. Limitations 73 sites = small sample size
Many clinics do not document birth dose of Hepatitis B vaccine
Delay may be overestimated
Immunization registry not accessible all sites
77% of babies born in King County hospitals actually receive birth dose
VFC site visit interview is often with only one clinic staff member

29. Contact Information
(206)