1. Kim Salisbury-Keith, MBA Rhode Island Department of Health October 2003 Provider Based Assessment of Data Quality The AFIX Model CDC 4th Immunization Registry Conference Provider Based Assessment of Data Quality The AFIX Model CDC 4th Immunization Registry Conference

2. Vaccine Management Provider Quality Assurance Service Delivery Support of Consumer Information Support of Surveillance Support of Assessment The Importance of Data Quality

3. Systems Approach Programming Edit Checks Data Cleaning Matching, Merging and Deduplication Evaluation of Individual Provider Data Quality Provider Confidence in Data Quality

4. A ssessment of Data Quality F eedback I ncentives E X change AFIX

5. Accuracy Completeness Timeliness AFIX Assessment of Data Quality

6. What to Look for: Improbable CPT Codes Vaccines Not in Use Vaccines Used at Inappropriate Ages Assessing Data Accuracy

7. What to Look for: Improbable Dates Vaccines with Less than Minimum Interval Vaccines Given on Holidays/Sundays Accuracy of the Denominator Assessing Data Accuracy

8. Data Queries Errors in Submission or Processing Chart Review Accuracy of Data vs. Practice Behavior Identify Source of Inaccuracy Methods For Assessing Data Accuracy

9. What to Look for: Time Gaps Missing Vaccine Codes Missing Histories Volume and Ratios of Vaccine Usage Assessing Data Completeness

10. Data Queries Data Gaps for Time or Vaccine Codes Vaccine Volume and Ratios Chart Review Missing Histories Missed Reporting or Data Entry Methods for Assessing Data Completeness

11. DTP806Expected Ratio5 Actual 5.0 HBV279Expected Ratio2 Actual 1.7 HIB363Expected Ratio3 Actual 2.3 MMR398Expected Ratio2 Actual 2.5 PCV383Expected Ratio4 Actual 2.4 POLIO656Expected Ratio4 Actual 4.1 VAR177Expected Ratio1 Actual 1.1 Ratio of Vaccine Usage Ratio for shots given by 83 Provider XYZ (Ratio formula treats the count of DTAP shots as 5 and pro-rates all other shot counts.) July 2002 to June 2003

12. Shot Count by Month of Administration

13. What to look for: Regularity of Data Submissions Recentness of Submitted Data Assessing Timeliness

14. Methods for Assessing Data Timeliness Data Queries % in Registry in 14 Days % in Registry in 30 Days Data Submission Record Provider Interview Review Office Processes

15. Written Report Discuss Comments and Recommendations Identify Problems Identify Solutions AFIX Feedback of Data Quality

16. Recognition in Newsletters Data Quality Awards - Annual Immunization Breakfast Contests Future CASA Assessments AFIX Incentives for Data Quality

17. Face to Face Meetings Frequent Office Visits Educational Materials Problem Solve – More than Registries Identify Office Practices Coding and Documentation Practices Office Work Flow AFIX Information Exchange

18. Schedule CASA / Data Assessment Visit Run Data Queries Prepare Written Data Assessment Report Identify Charts to be Reviewed Visit Office and Review Charts Meet with Provider for Feedback Provide Updated Written Report Process

19. Compare CASA Rate Before and After Chart Review Does Not Fully Assess Timeliness Best Single Indicator of Data Quality Provider Level

20. Principle: By enabling access to complete immunization records, immunization registries support providers in delivering age-appropriate and timely immunizations, and in reducing both under- and over-immunization. PROW Registry Standards of Excellence in Support of Provider Quality Assurance

21. Contact Information: Kim Salisbury-Keith, MBA KIDSNET Rhode Island Department of Health 3 Capitol Hill Providence, RI 02908 Telephone: 401- 222-5925 Fax: 401-222-1442 E-Mail: kims@doh.state.ri.us