Development and Validation of HealthImpactTM: An Incident Diabetes Prediction Model Based on Administrative Data Rozalina G. McCoy, M.D.1, Vijay S. Nori, Ph.D.2, Steven A. Smith, M.D.1, Christopher A. Hane, Ph.D.2 (1) Division of Endocrinology, Department of Medicine, Mayo Clinic, Rochester, MN; (2) Optum Labs, Cambridge, MA Abstract Data Source Results Performance of HealthImpact Comparison to “Pre-Diabetes” Background: Type 2 diabetes can be prevented or delayed if high risk individuals are identified early Current individual and population screening methods rely on laboratory detection of hyperglycemia, which can be burdensome and costly, particularly if applied on the population level. As a result, only 11% of people with diabetes are aware of their condition. Objective: To develop and validate HealthImpactTM to identify patients at high risk of developing type 2 diabetes. This model is based solely on administrative data and is not predicated on a face-to-face encounter or patient-provided information. Methods: We utilized the Optum Labs Data Warehouse (OLDW) administrative dataset of more 100 million commercial enrollees and Medicare Advantage beneficiaries across the U.S. Adults ≥18 years without diagnosed diabetes upon cohort entry were included. There were 473,049 adults in training cohort, 776,074 adults in the internal validation cohort, and 2,000,000 adults in the external validation cohort. Results: HealthImpact, scored on a linear scale 0-100, includes 47 demographic, medical, and medication variables obtained from administrative data; bias-corrected c-statistic 0.80815. In the training population, we identified HealthImpact scores of 50, 75, and 90 as indicative of low, intermediate, and high risk of incident diabetes. HealthImpact had very good discrimination in the internal (c- statistic 0.8270) and external (c-statistic 0.8171) validation cohorts. In the external validation dataset, the sensitivity, specificity, positive predictive value, and negative predictive value of HealthImpact >50 for incident diabetes at 3 years were 66.60%, 67.18%, 4.92%, and 98.7%; for HealthImpact >75, 32.84%, 90.98%, 8.49%, and 98.15%; and for HealthImpact >90, 14.43%, 97.23%, 11.73%, and 97.81%. Conclusions: HealthImpact is an efficient and effective method of risk stratification for incident diabetes that does not rely on patient-provided information or lab tests. It can be used by public health programs, health systems, and payers that utilize administrative data. Optum Labs Data Warehouse (OLDW) National de-identified dataset of >100 million privately insured individuals that is geographically and racially diverse, including individuals of all ages (including Medicare Advantage beneficiaries ≥65 years) and from all 50 states. Includes professional, facility, and outpatient prescription medication claims. HealthImpact Development (Figure 1) Nested case-control study design Diabetes ascertained by two diagnoses meeting HEDIS criteria within 180 days 12,482 variables were grouped using the agglomerative single-link Clustering algorithm  12,405 variables were passed to GLMNET to solve with a pure lasso regression model  8-way cross-validation to address overfitting of the data  final model includes 48 variables Development HealthImpact: constant+47 variables, bias-corrected c-statistic 0.80815 Scored 0-100 Thresholds based on risk of incident diabetes: Based on sensitivity/specificity of incident diabetes High (50) vs. Medium (75) vs. Low (90) Internal Validation C-statistic 0.8270 External Validation 2,000,000 adults (age 18-79) with continuous enrollment and no diagnosis of diabetes between 4/01/08 and 4/01/09  followed prospectively for up to 3 years. HealthImpact score was calculated at cohort entry (4/01/09). Point prevalence of diabetes ascertained at 12, 24, and 36 months. C-statistic: 0.8200 (1 year), 0.8171 (2 year), and 0.8171 (3 year) HealthImpact threshold >50 >75 >90 Training cohort (n=473,049)   Sensitivity 69.23 38.10 17.86 Specificity 76.49% 94.42 98.42 Internal validation cohort (n=303,025) 69.18 37.74 18.01 80.11 95.68 98.80 External validation cohort  Year 1 (n=1,383,743) 67.90 36.77 16.36 67.04 90.61 97.03 PPV 0.83 1.57 2.19 NPV 99.81 99.72 99.65 Year 2 (n=1,054,142) 67.62 34.63 15.60 90.75 97.11 2.55 4.56 6.45 99.39 99.09 98.90 Year 3 (n=827,969) 66.60 32.84 14.43 67.18 90.98 97.23 4.92 8.49 11.73 98.75 98.15 97.81 421,520 people in the external validation cohort had diabetes-related laboratory studies (glucose, HbA1c, glucose tolerance test) performed and available at baseline Laboratory pre-diabetes (n=87,977; 20.87%)  diabetes in 1.2% (1 yr), 6.7% (3 yr) HealthImpact >50 (n=136,874; 33.49%)  diabetes in 0.83% (1 yr), 4.92% (3 yr) HealthImpact >75 (n=39,480; 9.66%)  diabetes in 1.57% (1 yr), 8.49% (3 yr) HealthImpact >90 (n=12,629; 3.09%)  diabetes in 2.19% (1 yr), 11.73% (3 yr) HealthImpact >50 HealthImpact >75 HealthImpact >90 Pre-diabetes Year 1 (n=358,605)    Sensitivity 67.90 36.77 16.36 65.4 Specificity 67.04 90.61 97.03 78.5 Year 3 (n=239,079)  66.60 32.84 14.43 59.0 67.18 90.98 97.23 78.9 Discussion Diabetes prevention and treatment efforts are hindered by the challenges of identifying high risk patients in a reliable and cost-effective manner that could be implemented at local and population levels. HealthImpact is a novel type 2 diabetes risk prediction algorithm based entirely on administrative data, obviating need for patient-provided information, face-to-face encounter, or blood draw HealthImpact has good discrimination (c-statistic >0.8) in the general population of commercially insured adults in the U.S. Different score thresholds can be used depending on the context and objective. Specifically, higher score thresholds (>90) can be used to identify patients at very high risk and proactively reach out to them to schedule a clinical encounter and diagnostic testing. Intermediate thresholds (>75) can identify patients receiving unrelated care who would benefit from diabetes screening. Finally, low thresholds (>50) could be used to flag patients in whom the diagnosis of diabetes should be entertained if presenting with a constellation of clinically related symptoms or conditions. Study Cohorts Incident diabetes within 3 years External validation cohort Cumulative incidence of diabetes at 3 years HealthImpact ≤50: 1.24% HealthImpact 50-75: 5.80% HealthImpact 75-90: 12.24% HealthImpact >90: 22.25%   Training (N=473,049) Internal validation (N=303,025) External validation (N=2,000,000) Age, years, mean (SD) 45.76 (13.65) 44.21 (13.67) 44.12 (12.66) Gender, male, N (%) 211,638 (44.74) 145,148 (47.90) 967,243 (48.36) Race/ethnicity, N (%) High (>24%) minority 168,124 (35.54) 110,110 (36.34) 761,026 (38.05) Medium (9-24%) minority 175,568 (37.11) 111,461 (36.78) 718,080 (35.90) Low (<9%) minority 520,894 (26.04) Comorbidities, N (%) Ischemic heart disease 29,678 (6.27) 14,343 (4.73) 85,142 (4.26) Cerebrovascular disease 16,189 (3.42) 7,967 (2.63) 43,496 (2.17) Peripheral vascular disease 10,225 (2.16) 4,921 (1.62) 27,334 (1.37) Hypertension 140,297 (29.66) 72,234 (23.84) 427,801 (21.39) Hyperlipidemia 172,103 (36.38) 93,438 (30.84) 563,822 (28.19) Obesity 34,036 (7.20) 18,238 (6.02) 92,694 (4.63) Elevated blood glucose 16,441 (3.48) 7,883 (2.60) 28,572 (1.43) Gestational diabetes 3,522 (0.74) 1,847 (0.61) 10,001 (0.50) Polycystic ovarian syndrome 3,413 (0.72) 1,761 (0.58) 10,750 (0.54) Conclusions HealthImpact is a reliable and cost-effective way of identifying patients at risk for type 2 diabetes that can be adapted to all health plans and systems that bill for or codify their services. Different score thresholds can be used depending on the clinical context and goal of HealthImpact use Advanced analytic methods implemented in this study can be extended to other clinical questions, and exemplify how health information technology can be leveraged to promote patient and population health © 2015 Mayo Foundation for Medical Education and Research