Using Ohio’s Impact SIIS Data in Assessing Immunization Rates Carolyn A. Parry, MPH Public Health Advisor Ohio Department of Health Immunization Program

Objectives Determine when a practice is ready to use registry data for an assessment. Determine the methods to use for conducting assessments using Impact SIIS data. Number of records that matched the list of “active” patients What type of additional information in located in the charts Data quality

Methods Two sites were selected Site A began using the registry in January 2003 with pre-loaded historical information Site B began using the registry in March 2003 without pre-loaded historical information Three methods of extraction were researched Impact SIIS’s CASA Report function Electronic name and date of birth matching Random selection of 100 names and manually entering into CASA Two age ranges assessed Assessment conducted September 2003 Assessed as of January 1, 2003

Impact SIIS’s CASA Report function Currently utilized by some local health department How a practice is set up in Impact SIIS determines whether this function can be utilized File size and time limits do not allow this function to be used for all practices Not available for Site A or Site B

Electronic Name Matching An electronic list of names and dates of birth is used to electronically search for matches in the registry In this case, patient ID numbers were utilized to facilitate the matching process Conducted by computer staff, not an automated process within the registry

Results - Electronic Name Matching Records Found % Match MOGE Records Reviewed UTD from Name Match UTD from 100 Sample Site A 24-35 months 1614/1881 85.8% 204 1410 51.06% 62.63% 12-23 months 1483/1800 82.3% 61 1422 72.29% 67.00% Site B 205/380 53.9% 18 187 45.45% 42.42% 186/352 52.8% 41 145 60.00% 57.14%

Electronic Name Matching Questions/Issues to be resolved Names of those who did not match were excluded from the Impact SIIS extract Moved or Going Elsewhere (MOGE) When modified it will be a valuable way to pull assessment data for larger practices and possibly HMO groups

Random Selection of 100 Names Manually Entered into CASA 100 records randomly selected from each age group Records manually retrieved from Impact SIIS and entered into CASA Pulled charts of those records with missing immunizations, entered missing chart data into CASA

Site A Results – Before and After Chart Pull UTD Late UTD Missed Opps Last Imm > 12 months UTD with varicella UTD at 12 months 24-35 months Original 62.63% 16.16% 7.07% 12.12% 52.53% 75.76% After chart pull 66.67% 15.15% 9.09% 56.57% 77.78% 12-23 months 67.00% 17.00% 1.00% 73.40% 15.96% 1.06% 4.37 and 6.40 percentage point increases in UTD rate, respectively

Site B Results – Before and After Chart Pull UTD Late UTD Missed Opps Last Imm > 12 months UTD with varicella UTD at 12 months 24-35 months Original 42.42% 15.15% 12.12% 28.28% 36.36% 51.52% After chart pull 68.80% 18.48% 2.17% 10.87% 59.78% 73.33% 12-23 months 57.14% 13.27% 8.16% 78.82% 14.12% 0.00% 26.38 and 21.68 percentage point increases in UTD rate, respectively

Site A Results – Status in Registry and Additional Information Found in Charts 24-35 month age group Imms Phil/ Religious Exempt History of Disease MOGE Not found (2) 2 Found – no shots (2) 1 Found – not complete (26) -Missing varicella only (8) -Missing charts (2) *7 of 30 charts had additional information

Site B Results – Status in Registry and Additional Information Found in Charts 24-35 month age group Imms Phil/ Religious Exempt History of Disease MOGE Not found (7) 4 3 Found – no shots (12) 9 1 2 Found – not complete (33) -Missing varicella only (4) -Deleted (1) 17 *39 of 52 charts had additional information

Conclusions Flexibility in the choice of available methods of data extraction is crucial How sites are set up and utilize the registry impacts the ability to use certain methods All assessment methods still need to obtain a list of “active” patients In the beginning, sites using the registry real-time with pre-load historical information will have more complete assessments using registry only data

Conclusions continued… Conducting assessments with registry data serves as a quality check Limitations of the data being reviewed must be taken into consideration Electronic data does not necessarily eliminate the need to pull charts

Recommendations Site B would benefit from having more information in the registry Impact SIIS should review the algorithm for accuracy Documentation of history of disease and its effect on forecasting Replicate the study in 2004 Look more closely at data quality

Questions? Contact: Carolyn A. Parry, MPH Public Health Advisor Ohio Department of Health Immunization Program (614) 466-0249 cparry@gw.odh.state.oh.us