1. SA Lorch1, JH Silber1, GE Escobar2, D Small3
Readmissions as a measure of quality for neonatal intensive care units (NICU) and outpatient practices
SA Lorch1, JH Silber1, GE Escobar2, D Small3
1 Center for Outcomes Research, Dept of Pediatrics, The Children’s Hospital of Philadelphia
2 Perinatal Research Center, Kaiser Permanente Medical System, Oakland, CA
3 Dept of Statistics, The Wharton School
Thank you very much for this opportunity to present our work. This study will share characteristics of the previous presentations, with some unique differences because of the patient population.

2. Background
There is increased interest in measuring the quality of inpatient care by insurers, public agencies, and patients.
One proposed measure: risk-adjusted readmission rates
There is increased interest in measuring the quality of inpatient care by insurers, public agencies such as state departments of health, and patients. One proposed measure has been risk-adjusted readmission rates.

3. Conceptual Framework Poor inpatient quality of care
Incomplete Evaluation or Management of Condition
Increased Readmission Rates
This slide shows a conceptual framework for the use of readmission rates to assess the quality of inpatient care. Poor inpatient care results in the incomplete evaluation or management of a specific condition. When this patient is discharged from the hospital, they have a higher risk of readmission, resulting in increased readmission rates.

4. Prior Work Conflicting data in literature
Ashton (1995): Meta-Analysis, 13 studies
OR 1.55 for readmission if care received at hospital with sub-standard quality
Wide range of metrics and time frame for readmissions
More recent literature did not find this association
Congestive Heart Failure
Myocardial Infarction
The literature on the use of readmission rates as a quality of care measure is conflicting. Ashton in 1995 presented a meta-analysis of 13 studies of readmission rates, and found a composite odds ratio of 1.55 for readmission if a patient received care at a hospital with “sub-standard” quality. Of note, the definition of “sub-standard” care was highly variable – both explicit and implicit criteria were used, and the quality thresholds were very different between studies. However, recent literature in congestive heart failure and ischemic myocardial infarction have not found such an association between readmission rates and inpatient quality of care. Why?

5. Conceptual Framework Poor inpatient quality of care
Incomplete Evaluation or Management of Condition
Increased Readmission Rates
If we examine our conceptual framework, we notice something missing – other reasons that inpatient hospitals may have higher readmission rates. For example, patients with higher illness severity should have increased readmission rates. Illness severity can also be a function of a given hospital’s criteria for admission; a more stringent threshold, which only admits the sickest of patients, could increase a hospital’s rate of readmission. Finally, the quality of an outpatient facility may influence readmission rates. Few, if any studies, account for these outpatient sites of care in their analyses.
Illness Severity
Quality of Outpatient Facility

6. Goals of Study
Aim 1: Determine the role of NICUs in predicting variations in risk-adjusted readmission rates
Aim 2: Determine how this role changes when site of outpatient care is accounted for
Aim 3: Define characteristics of facilities associated with high risk-adjusted readmission rates
There were three goals for this study. First, we determined the role of NICUs in predicting variations in risk-adjusted readmission rates. We then determined how this role changed when we accounted for the site of outpatient care. Finally, we defined facility characteristics associated with higher risk-adjusted readmission rates.

7. Patient Population 5 Northern California Kaiser Permanente hospitals
Gestational age < 32 weeks at delivery
Survive to hospital discharge
Received care at 1 of 32 outpatient clinics affiliated with the Kaiser Permanente health system
To answer these questions, we constructed a population cohort of all infants born at a gestational age of 32 weeks or less at five Kaiser Permanente hospitals between 1998 and For the non-clinicians in the audience, this gestational age cutoff is 2 months before the “usual” time of delivery for an infant, and all infants at these gestational ages require care at a NICU. All infants must have survived to hospital discharge, and all infants received care at 1 of 32 outpatient clinics affiliated with the Kaiser system.

8. Exclusion Criteria Major congenital anomalies
Need for home ventilation
Loss to follow-up within 1 year of discharge
Typically from leaving the Kaiser system
This slide shows the 3 exclusion criteria. Most eligible infants were excluded because of loss to follow-up within the time frame after discharge. These infants were typically lost because they left the Kaiser system. Compared to included infants in the study, these excluded infants had a slightly heavier birth weight but had similar illness severity during their NICU course, as measured by either the percentage of infants with a complications of premature birth or the need for mechanical ventilatory support.

9. Study Definitions Readmission Ambulatory-care Sensitive Condition
Any unplanned rehospitalization within specified time period
Ambulatory-care Sensitive Condition
Any readmission for condition “sensitive” to care provided in outpatient setting
Pneumonia
Asthma
Cellulitis
Failure-to-Thrive
Time Frame: 0-1 month; 0-3 months; 3-12 months
We defined a readmission as any unplanned readmission within a specified time frame, which is listed at the bottom of the slide. For premature infants, planned readmissions were overwhelmingly for the surgical repair of inguinal hernias in male infants. We also were interested in readmissions for a group of conditions considered “sensitive” to the care provided in an outpatient setting. We included this separate group to determine whether the types of conditions influenced the validity of our quality metric. For this study, we included typical pediatric ACS conditions: pneunomia, asthma, cellulits, and failure-to-thrive.

10. Data Collection Neonatal data
Neonatal Minimal Data Set: prospective collection of 250 clinical variables, including
Maternal history
Birth history
Complications occurring in NICU
Outpatient data
Cost Management Information System tracked all resources used in the outpatient setting, including
medications and laboratory studies
readmissions, outpatient visits, and emergency room visits
Demographic data used to define area-level socioeconomic data based on zip code
Data were collected in 2 datasets. Inpatient data came from the Neonatal Minimal Data set, which prospectively collected data for over 250 clinical variables, includes information on maternal and birth history and clinical information from the NICU stay. Outpatient data, specifically information on readmissions, were collected through administrative records.

11. Facility Characteristics
Outpatient facility assigned to child based on site of usual care for well-child visits.
Characteristics:
Use of oral albuterol (poor quality)
Use of inhaled albuterol for asthma symptoms (good quality)
Use of antibiotics for viral illness (poor quality)
Facilities divided into high or low quality for each characteristic.
For our last goal, we were interested in determining the association between various facility characteristics and the readmission measure. Children were assigned to the outpatient facility where they received the majority of well-child visits. Facilities received a risk-adjusted rate of 3 process measures: use of oral albuterol; use of inhaled albuterol for persistent asthma symptoms; and use of antibiotics for apparent viral illness. Facilities were divided into those that were high or low quality based on whether they were higher or lower than the expected rate for each characteristic.

12. Statistical Analysis Multivariable poisson regression models
2 sets of models
Fixed NICU effects included alone (Aim 1)
Random outpatient effects added to the fixed NICU effects (Aim 2)
Random outpatient effects accounts for smaller numbers of patients at a given outpatient center.
2 separate multivariable poisson regression models were developed; the first only included fixed NICU effects – to replicate previous studies – while the second included both fixed NICU effects and random outpatient effects. This random variable allows for the statistical interpretation of outpatient centers that cared for a few prematurely-born patients in this cohort.

13. Demographics 892 infants at 5 NICUs and 32 outpatient facilities
Each NICU discharged to 9-17 outpatient facilities
Each outpatient facility received infants from 1-3 NICUs
Gestational Age 29.5 ± 2.2 wks
Racial/Ethnic Distribution:
45.5% White non-Hispanic
20.5% Hispanic
11.2% Black
22.8% Asian or Multi-Racial
16.6% with BPD, 1.8% with NEC
Our results. There were 892 infants captured in this study. The average gestational age was 29.5 weeks. Almost half of the infants were white non-hispanics; 20% were hispanic, nearly all white hispanic infants; 11% Black; and 22% other races, primarily Asian infants. About 1/6 of the population had chronic premature lung disease, abbreviated as “BPD”, and about 2% had gastrointestinal injury related to prematurity, abbreviated as “NEC”.

14. Timing of Readmissions
This slide shows the timing of each readmission in the study. The month after discharge is shown in the x axis, and the number of readmissions are shown in the y-axis. We can see that the majority of readmissions occur in the first 4 months after discharge, after which there are a fairly consistent number of readmissions out to 1 year after discharge.

15. All Readmissions: Medical Factors
0-1 mos
0-3 mos
3-12 mos
Gestational Age
< 26 wks
8.65***
5.43**
2.80***
27-28 wks
6.48***
3.36**
1.98**
29-30 wks
6.17***
2.79**
1.00
31-32 wks
Reference
NEC
2.43
3.55***
2.60*
BPD
2.20*
1.17
1.21
Home on oxygen
0.75
1.36
1.39
The following slides show the results of our multivariable models. The following 2 slides show the incident rate ratios for various patient-level factors in predicting the number of readmission within 1 month or 3 months after discharge, or 3-12 months after discharge. These factors make up the “risk adjustment” for our readmission rates and did not change appreciably whether we included NICUs alone, or added outpatient sites of care to the analysis. We can see from this slide, which presents the medical factors associated with readmissions, that younger GA is the primary factor predicting readmissions. BPD and NEC are also associated with readmissions.
All values report incident rate ratios for the given risk factor
* P < 0.05; ** P < 0.01; *** P < 0.001

16. All Readmissions: Sociodemographic Factors
0-1 mos
0-3 mos
3-12 mos
Each sibling at home
1.13
1.06
1.34***
Median Area Income, per $10,000
0.98
1.03
0.95
Racial/Ethnic Status
Reference
Asian/Other
1.01
0.84
0.99
Black
1.15
1.53
1.18
Hispanic
1.22
0.79
0.91
White
Maternal Age < 18 yrs
1.24
0.70
1.65
Male Sex
0.96
1.21
1.14
Fewer sociodemographic factors were associated with readmission rates for any reason, except for the number of siblings at home.
All values report incident rate ratios for the given risk factor
* P < 0.05; ** P < 0.01; *** P < 0.001

17. All Readmissions: NICU and Outpatient Facilities
0-1 mos
0-3 mos
3-12 mos
NICU measured alone
NICU Variation
0.24
0.02
< 0.001
NICU measured with outpatient facility
0.25
0.10
0.07
Outpatient Facility Variation
1.00
0.011
After including the previous factors, we then determined whether NICUs were associated with variations in readmission rates. The first row shows the results when NICUs are evaluated by themselves. We can see that readmissions 0-3 and 3-12 mos after discharge were associated with NICUs…but once we included the outpatient facility in our analyses, we see that NICUs did not significantly predict a change in readmission rates, but rather outpatient facilities took this variation.

18. Example of Attributable Variation: All readmissions 0-3 months
Patient
NICU
Program
This slide presents our results in another fashion. We have created Venn diagrams, to represent the amount of variation in our readmission metric that is attributable to patient-level factors, shown in the red circle; NICUs, shown in the yellow circle; and the outpatient site, shown in the green circle. The amount of variation is represented by the area of the circle. We can see that
Area of each circle represents the proportional amount of variation attributed to each group of factors.

19. ACS Readmissions: Medical Factors
0-1 mos
0-3 mos
3-12 mos
Gestational Age
< 26 wks
15.66***
7.43***
2.89**
27-28 wks
15.32***
3.13**
1.72
29-30 wks
21.01***
3.79***
0.72
31-32 wks
Reference
NEC
2.44
2.25
2.80***
BPD
2.10
1.12
1.78
Home on oxygen
0.59
1.33
0.81
The following three slides show our results when we limited the analysis to ACS readmissions. Again, we found that younger GA was strongly associated with higher readmission rates.
All values report incident rate ratios for the given risk factor
* P < 0.05; ** P < 0.01; *** P < 0.001

20. ACS Readmissions: Sociodemographic Factors
0-1 mos
0-3 mos
3-12 mos
Each sibling at home
1.04
1.27*
1.43***
Median Area Income, per $10,000
0.96
0.99
0.90
Racial/Ethnic Status
Reference
Asian/Other
1.44
0.97
0.80
Black
1.31
1.91
1.13
Hispanic
1.21
1.03
White
Maternal Age < 18 yrs
1.35
0.64
2.64
Male Sex
0.54
0.84
0.68
Again, only the number of sibilings at home were associated with increased readmissions among the following sociodemographic factors.
All values report incident rate ratios for the given risk factor
* P < 0.05; ** P < 0.01; *** P < 0.001

21. ACS Readmissions: NICU and Outpatient Facilities
0-1 mos
0-3 mos
3-12 mos
NICU measured alone
NICU Variation
0.63
0.06
< 0.001
NICU measured with outpatient facility
0.24
0.41
Outpatient Facility Variation
0.50
1.00
And, as with all readmissions, the site of NICU care predicted ACS readmissions between 3-12 mos, but these facilities lost their predictive ability once we included outpatient facilities in the model.

22. Facility Characteristics and Readmissions
This graph shows the results of our third aim: what facility characteristics were associated with higher readmission rates? The specific outcome is shown on the horizontal axis, and the associated incident rate ratio between being a higher-than-expected facility for one of the characteristics and the outcome are shown in as points on the vertical axis. 95% CI are shown as brackets. Oral albuterol is always shown on the left of a specific outcome, as the red circle; inhaled albuterol is shown as the green diamond; and antibiotics for viral conditions are shown as the orange triangles on the right. We can see that higher use of oral albuterol is associated with higher readmissions at 0-3 and 3-12 mos, both for all readmissions and ACS readmissions. Similarly, higher use of antibiotics for viral conditions was associated with higher readmissions at 3-12 mos. Note that higher use of either treatment characteristic could be considered poorer quality of care.
Oral Albuterol
Inhaled Albuterol
Viral antibiotics

23. Limitations Data from one health system
NICUs and outpatient facilities with different practices and outcomes
No direct information on family income and socioeconomic status
Cohort more homogeneous than other NICUs, especially academic centers
There are two significant limitations. First, all data came from patients treated within a single health system, who may have similar practice patterns and guidelines. However, the NICUs and outpatient facilities did differ in many features, including their forms that they used. Also, there were no direct information on family income or socioeconomic status. Finally, the cohort was likely more homogenous than a cohort from other NICUs, especially academic medical centers.

24. Conclusions
Patient-level factors were the primary determinants for readmissions after NICU discharge.
NICU measured alone:
Significant variations between sites.
NICU measured with outpatient facilities:
No independent variation between NICUs
In conclusion, patient-level factors – especially younger GA – were the primary determinants for readmission after NICU discharge. When we used the measure on NICUs alone, without accounting for the site of outpatient care, there was significant variation between NICU sites, especially 0-3 and 3-12 mos after discharge. Once we included the site of outpatient care in our analyses, though, the variation moved to the outpatient site; NICUs lost their association with readmission rates.

25. Conclusions
Outpatient facility characteristics associated with poor quality are also associated with higher readmission rates:
High oral albuterol use: 0-3 mos and 3-12 mos
High antibiotic use: mos
Time frame does matter when examining readmission rates.
As further evidence for the role of outpatient facilities in readmission rates, we found that characteristics associated with poor quality were also associated with higher readmission rates. The time frame does matter when examining the various factors associated with variation in rates.

26. Implications for Policy
Readmission rates appear to measure the quality of outpatient facilities, not inpatient hospitals.
Associations with NICU  typical sites of outpatient care to which a NICU discharges.
Thus, readmission rates appear to measure the quality of outpatient facilities, not inpatient hospitals. Any associations with the inpatient service, here the NICU, were explained by the group of outpatient centers to which a NICU discharges.

27. Acknowledgements Funded by MCHB R40 MC00238
Thanks to Marla Gardner and John Greene at Northern California Kaiser Permanente health system.

28. Why Study NICUs and Premature Infants?
Prematurely-born infants are uniformly admitted to NICUs.
Relatively consistent discharge practices based on development of physiologic skills and weight gain.
Readmission rates after discharge are high, but do not occur in all patients.
Allows for variation among NICUs and outpatient settings
Why study NICUs and prematurely-born infants? Prematurely born infants are uniformly admitted to NICUs, avoiding one of the potential deficits I presented in the last slide. There are relatively consistent discharge practices based on the development of physiologic skills and weight gain. Finally, there is signal in this high-risk population: readmission rates after discharge are high, but do not occur in all patients. This allows for potential variations among the inpatient and outpatient settings.

29. Deficits in Literature
Many conditions do not have validated admission criteria
Wide variations in time frame
Which time frames are valid?
No control for site of outpatient care
Ignoring these factors may lead to faulty assessment of the care provided by inpatient services, such as neonatal intensive care units (NICUs)
These two issues are important to understanding the deficits in the current literature. Many conditions do not have validated admission criteria; as a result, hospitals with more lenient or stringent admission criteria may have different rates of readmission…and we may not be able to control for these differences. There are also wide variations in the time frame used to monitor for readmissions, without data to validate these time frames. Finally, few studies even acknowledge the difference in the site of outpatient care experienced by patients at the same, or different inpatient settings. Ignoring these factors may lead to a faulty assessment of the care provided by inpatient services, such as neonatal intensive care units.