1. Case Completeness and Data Accuracy in the National Program of Cancer Registries KK Thoburn, CDC/NPCR Contractor RR German, M Lewis, P Nichols, F Ahmed, CDC J Jackson-Thompson, University of Missouri-Columbia 2007 Annual NAACCR Conference Detroit, Michigan Tuesday, June 5, 2007

2. NPCR Cancer Surveillance System (NPCR-CSS) Established in 2000 In collaboration with SEER, collects cancer incidence data for the entire US population NPCR-CSS data submissions are evaluated for case completeness (CC), percentage of death-certificate-only (DCO) cases, percentage of unknown/missing values for selected required data elements, and percentage of cases passing critical edits for data quality Measures incidence rates by various characteristics Data estimates CCR progress in meeting NPCR Program Standards

3. NPCR Technical Assistance and Audit Program (NPCR-TAA) ORC Macro conducts NPCR-TAA Performs case completeness (CC) and data accuracy (DA) audits at the level of the hospital reporting to the CCR Uses sample of cases from selected facilities for reabstraction – Lung and bronchus, colon and rectum, prostate, and female breast – These primary sites represent over 50% of cases reported to the NPCR-CSS Provides technical assistance to CCRs

4. NPCR Annual Program Evaluation Instrument (NPCR-APEI) Web-based instrument completed annually by all NPCR-funded CCRs Assessment of NPCR program objectives, registry operations, and data use

5. Present Study Usefulness of NPCR-CSS data depends on completeness of case reporting and accuracy of submitted information NPCR-CSS data are frequently used to measure incidence and survival – Race, sex, age, subsite, and stage at diagnosis Present national-level study: – Summary analysis of NPCR-TAA information by primary site – Covariate analysis of NPCR-TAA results with information from the NPCR-APEI, and CCR achievement of USCS publication and NAACCR certification standards

6. Methods NPCR-TAA audit results for 34 of 45 NPCR-supported CCRs audited at the time of analysis were included 1998 through 2001 audit diagnosis years were included Overall average case completeness and data accuracy rates, and site-specific data accuracy rates were calculated – DA rates calculated by site for each of the 13 data audited data Average CC and site-specific DA rates were examined across the covariates – Small-sample Student’s t tests (α = 0.05)

7. Methods CC rates (%) 100 - (# of missed cases / total # of cases identified) x 100 DA rates, overall and site-specific (%) = (# of data elements with no discrepancies / total # of data elements reabstracted) x 100 DA rates, audited data elements (%) = (# of reabstracted cases with no discrepancies on data element / total # of cases reabstracted) x 100

8. Methods: Sources of Variables NPCR-TAA: CC and DA rates Primary site Audited data elements Source of missed cases USCS publication: Met publication standards? NAACCR Web site: Certified by NAACCR? Certification level NPCR-APEI: Type of current funding? Ratio of FTE positions to central registry caseload? Ratio of CTRs to central registry caseload? Supplementary reporting sources reporting cases? Case-finding audits at reporting facilities? Reabstracting audits at reporting facilities? Annual report issued?

9. Results: Case Completeness 34 CCRs, diagnosis years 1998 to 2001 41,521 lung and bronchus, colon and rectum, prostate, and female breast were identified 1,503 cases were missed Overall case completeness rate of 96.4%

10. Lung and bronchus 19.9% (n = 299 ) Colon and rectum 18.8% (n = 283) Prostate 28.6% (n = 430) Female breast 32.7% (n = 491) Distribution of Missed Cases (N = 1, 503)

11. Other Autopsy record Cytology report Radiation therapy log Pathology report Medical records disease index Lung and bronchus n = 324* 62.0% 29.0% 2.2% 3.7% 3.1% Colon and rectum n = 315* 55.6% 43.2% 0.6% 0.3% Female breast n = 544* 52.0% 45.8% 0.2% 0.7% 1.3% Prostate n = 458* 53.9% 40.6% 0.2% 0.6% 4.6% *Counts include missed cases from more than one case-finding source. Distribution of Missed Cases by Type of Case-finding Source

12. Site-Specific Data Accuracy Lung and bronchus (n = 2,448) Female breast (n = 3,210) Prostate (n = 2,289) Colon and rectum (n = 2,063) Percentage of records containing no discrepancies 93.5 85.9 81.2 89.7 96.2 97.5 98.0 99.1 99.9 98.9 99.3 99.5 99.9 92.0 89.9 100.0 97.6 95.3 98.2 97.5 99.4 100.0 99.9 99.4 99.7 100.0 84.3 93.9 93.6 88.3 98.5 97.1 97.8 99.4 97.4 98.7 99.2 99.9 99.8 83.5 89.7 89.1 95.5 96.9 98.4 95.4 99.4 99.8 99.5 99.8 99.7 0102030405060708090100 SEER Summary Stage Grade Subsite Histology Diagnosis date Sequence number Race Laterality Primary site Behavior Date of birth State of residence Gender

13. Covariate Analysis Ratio of FTEs and CTRs to CCR Caseload Higher CC and DA rates were found for the more well-staffed registries (not significant; P >.05). Higher DA rates were found for registries staffed with a greater number of CTRs (significant for colon and rectum, prostate) Higher CC rate was found for registries staffed with a greater number of CTRs (not significant; P >.05).

14. Covariate Analysis Supplementary Reporting Sources Higher CC rates were found for registries with pathology laboratories and/or radiation therapy centers facilities reporting (P <.01). Higher site-specific DA rates were found for registries with pathology laboratories and/or radiation therapy centers facilities reporting (not significant; P >.05).

15. Covariate Analysis Met USCS Publication Standards/Achieved NAACCR Certification Higher CC and average site-specific DA rates were found for CCRs that achieved these 2 milestones (only colon and rectum significant; P =.02, USCS; P =.04, NAACCR certification) NAACCR-certified CCRs generally had higher data element-specific DA rates than non-certified CCRs

16. Conclusions Summary Analysis of NPCR-TAA Data Underscores importance and effectiveness of conducting CC and DA audits at reporting hospitals Enables identification of general and site-specific case- finding and abstracting issues Demonstrates overall high accuracy and completeness of NPCR-CSS incidence data on cancers of the lung and bronchus, colon and rectum, prostate, and female breast – Overall CC (96%) higher than NPCR-CSS 24-month standard (95%) – Overall DA rate (95%), and high DA rates found by demographic data elements and primary site lend confidence in incidence rates Provides guidance to users of the data

17. Conclusions Covariate Analysis Shows that combining data from the NPCR-TAA and the NPCR-APEI provides additional valuable information that neither program can provide individually Offers perspective on how CCR operations effect case completeness and data quality Underscores importance of CCRs having well-trained staff; having supplemental reporting sources; and attaining compliance with national data standards

18. Conclusions NPCR funding and technical assistance help CCRs in developing and enhancing effective registry operations—especially in areas such as staffing, training, and monitoring and in improving the completeness and quality of registry Present study demonstrates: – Positive outcome of enhanced registry operations – Benefit of NPCR support for a high-quality, statewide, population-based CCR

19. Special Appreciation Arizona Arkansas Colorado District of Columbia Florida Georgia Idaho Illinois Kansas Maine Maryland Massachusetts Michigan Minnesota Missouri Montana Nebraska Nevada New Hampshire New York North Carolina North Dakota Ohio Oklahoma Oregon Pennsylvania Rhode Island Tennessee Texas Vermont Virginia Washington West Virginia Wisconsin