1. Development and Validation of HealthImpactTM: An Incident Diabetes Prediction Model Based on Administrative Data
Rozalina G. McCoy, M.D.1, Vijay S. Nori, Ph.D.2, Steven A. Smith, M.D.1, Christopher A. Hane, Ph.D.2 (1) Division of Endocrinology, Department of Medicine, Mayo Clinic, Rochester, MN; (2) Optum Labs, Cambridge, MA
Abstract
Data Source
Results
Performance of HealthImpact
Comparison to “Pre-Diabetes”
Background: Type 2 diabetes can be prevented or delayed if high risk individuals are identified early
Current individual and population screening methods rely on laboratory detection of hyperglycemia, which can be burdensome and costly, particularly if applied on the population level.
As a result, only 11% of people with diabetes are aware of their condition.
Objective: To develop and validate HealthImpactTM to identify patients at high risk of developing type 2 diabetes.
This model is based solely on administrative data and is not predicated on a face-to-face encounter or patient-provided information.
Methods: We utilized the Optum Labs Data Warehouse (OLDW) administrative dataset of more 100 million commercial enrollees and Medicare Advantage beneficiaries across the U.S. Adults ≥18 years without diagnosed diabetes upon cohort entry were included.
There were 473,049 adults in training cohort, 776,074 adults in the internal validation cohort, and 2,000,000 adults in the external validation cohort.
Results: HealthImpact, scored on a linear scale 0-100, includes 47 demographic, medical, and medication variables obtained from administrative data; bias-corrected c-statistic
In the training population, we identified HealthImpact scores of 50, 75, and 90 as indicative of low, intermediate, and high risk of incident diabetes.
HealthImpact had very good discrimination in the internal (c- statistic ) and external (c-statistic ) validation cohorts.
In the external validation dataset, the sensitivity, specificity, positive predictive value, and negative predictive value of HealthImpact >50 for incident diabetes at 3 years were 66.60%, %, 4.92%, and 98.7%; for HealthImpact >75, 32.84%, %, 8.49%, and 98.15%; and for HealthImpact >90, %, 97.23%, 11.73%, and 97.81%.
Conclusions: HealthImpact is an efficient and effective method of risk stratification for incident diabetes that does not rely on patient-provided information or lab tests.
It can be used by public health programs, health systems, and payers that utilize administrative data.
Optum Labs Data Warehouse (OLDW) National de-identified dataset of >100 million privately insured individuals that is geographically and racially diverse, including individuals of all ages (including Medicare Advantage beneficiaries ≥65 years) and from all 50 states.
Includes professional, facility, and outpatient prescription medication claims.
HealthImpact Development (Figure 1)
Nested case-control study design
Diabetes ascertained by two diagnoses meeting HEDIS criteria within 180 days
12,482 variables were grouped using the agglomerative single-link Clustering algorithm  12,405 variables were passed to GLMNET to solve with a pure lasso regression model  8-way cross-validation to address overfitting of the data  final model includes 48 variables
Development
HealthImpact: constant+47 variables, bias-corrected c-statistic
Scored 0-100
Thresholds based on risk of incident diabetes:
Based on sensitivity/specificity of incident diabetes
High (50) vs. Medium (75) vs. Low (90)
Internal Validation
C-statistic
External Validation
2,000,000 adults (age 18-79) with continuous enrollment and no diagnosis of diabetes between 4/01/08 and 4/01/09  followed prospectively for up to 3 years.
HealthImpact score was calculated at cohort entry (4/01/09).
Point prevalence of diabetes ascertained at 12, 24, and 36 months.
C-statistic: (1 year), (2 year), and (3 year)
HealthImpact threshold
>50
>75
>90
Training cohort (n=473,049)
Sensitivity
69.23
38.10
17.86
Specificity
76.49%
94.42
98.42
Internal validation cohort (n=303,025)
69.18
37.74
18.01
80.11
95.68
98.80
External validation cohort 
Year 1 (n=1,383,743)
67.90
36.77
16.36
67.04
90.61
97.03
PPV
0.83
1.57
2.19
NPV
99.81
99.72
99.65
Year 2 (n=1,054,142)
67.62
34.63
15.60
90.75
97.11
2.55
4.56
6.45
99.39
99.09
98.90
Year 3 (n=827,969)
66.60
32.84
14.43
67.18
90.98
97.23
4.92
8.49
11.73
98.75
98.15
97.81
421,520 people in the external validation cohort had diabetes-related laboratory studies (glucose, HbA1c, glucose tolerance test) performed and available at baseline
Laboratory pre-diabetes (n=87,977; 20.87%)  diabetes in 1.2% (1 yr), 6.7% (3 yr)
HealthImpact >50 (n=136,874; 33.49%)  diabetes in 0.83% (1 yr), 4.92% (3 yr)
HealthImpact >75 (n=39,480; 9.66%)  diabetes in 1.57% (1 yr), 8.49% (3 yr)
HealthImpact >90 (n=12,629; 3.09%)  diabetes in 2.19% (1 yr), 11.73% (3 yr)
HealthImpact >50
HealthImpact >75
HealthImpact >90
Pre-diabetes
Year 1 (n=358,605) 
Sensitivity
67.90
36.77
16.36
65.4
Specificity
67.04
90.61
97.03
78.5
Year 3 (n=239,079) 
66.60
32.84
14.43
59.0
67.18
90.98
97.23
78.9
Discussion
Diabetes prevention and treatment efforts are hindered by the challenges of identifying high risk patients in a reliable and cost-effective manner that could be implemented at local and population levels.
HealthImpact is a novel type 2 diabetes risk prediction algorithm based entirely on administrative data, obviating need for patient-provided information, face-to-face encounter, or blood draw
HealthImpact has good discrimination (c-statistic >0.8) in the general population of commercially insured adults in the U.S.
Different score thresholds can be used depending on the context and objective. Specifically, higher score thresholds (>90) can be used to identify patients at very high risk and proactively reach out to them to schedule a clinical encounter and diagnostic testing. Intermediate thresholds (>75) can identify patients receiving unrelated care who would benefit from diabetes screening. Finally, low thresholds (>50) could be used to flag patients in whom the diagnosis of diabetes should be entertained if presenting with a constellation of clinically related symptoms or conditions.
Study Cohorts
Incident diabetes within 3 years
External validation cohort
Cumulative incidence of diabetes at 3 years
HealthImpact ≤50: 1.24%
HealthImpact 50-75: 5.80%
HealthImpact 75-90: 12.24%
HealthImpact >90: 22.25%
Training
(N=473,049)
Internal validation
(N=303,025)
External validation
(N=2,000,000)
Age, years, mean (SD)
45.76 (13.65)
44.21 (13.67)
44.12 (12.66)
Gender, male, N (%)
211,638 (44.74)
145,148 (47.90)
967,243 (48.36)
Race/ethnicity, N (%)
High (>24%) minority
168,124 (35.54)
110,110 (36.34)
761,026 (38.05)
Medium (9-24%) minority
175,568 (37.11)
111,461 (36.78)
718,080 (35.90)
Low (<9%) minority
520,894 (26.04)
Comorbidities, N (%)
Ischemic heart disease
29,678 (6.27)
14,343 (4.73)
85,142 (4.26)
Cerebrovascular disease
16,189 (3.42)
7,967 (2.63)
43,496 (2.17)
Peripheral vascular disease
10,225 (2.16)
4,921 (1.62)
27,334 (1.37)
Hypertension
140,297 (29.66)
72,234 (23.84)
427,801 (21.39)
Hyperlipidemia
172,103 (36.38)
93,438 (30.84)
563,822 (28.19)
Obesity
34,036 (7.20)
18,238 (6.02)
92,694 (4.63)
Elevated blood glucose
16,441 (3.48)
7,883 (2.60)
28,572 (1.43)
Gestational diabetes
3,522 (0.74)
1,847 (0.61)
10,001 (0.50)
Polycystic ovarian syndrome
3,413 (0.72)
1,761 (0.58)
10,750 (0.54)
Conclusions
HealthImpact is a reliable and cost-effective way of identifying patients at risk for type 2 diabetes that can be adapted to all health plans and systems that bill for or codify their services.
Different score thresholds can be used depending on the clinical context and goal of HealthImpact use
Advanced analytic methods implemented in this study can be extended to other clinical questions, and exemplify how health information technology can be leveraged to promote patient and population health
© 2015 Mayo Foundation for Medical Education and Research