1. Overview of the Pediatric Indicator Module Presenters: Kathryn McDonald and Sheryl Davies, Stanford University AHRQ QI User Meeting September 26-27, 2005

2. Acknowledgements Pediatric Module Development: Kathryn McDonald, Stanford University Patrick Romano, UC-Davis Sheryl Davies, Stanford University Amy Ku, Stanford University Kavita Choudhry, Stanford University Jeffrey Geppert, Battelle Health and Life Sciences Corinna Haberland, Stanford University Support for Quality Indicators II (Contract No. 290-04-0020): Mamatha Pancholi, AHRQ Project Officer Marybeth Farquhar, AHRQ Mark Gritz and Jeffrey Geppert, Project Directors, Battelle Health and Life Sciences

3. spinningwheelalpacas.comchkd.com/images/HospitalVisit.jpg Children’s Hospitalizations, US 2000 6.3 million $46 billion 36% of 1-17 yr olds in Children’s hospitals

4. Unique Population Dependent on adults Dependent on adults Constantly developing Constantly developing Demographics Demographics Epidemiology Epidemiology Coding in pediatrics Coding in pediatrics Simpson LA, al DDe. Measures of Children's Health Care Quality: Building towards Consensus. Manuscript in preparation: Background paper prepared for National Quality Forum; 2003 September 19.

5. Current Measurement State Simpson and colleagues search Simpson and colleagues search Simpson LA, et al. Measures of Children's Health Care Quality: Building towards Consensus. Manuscript in preparation: Background paper prepared for National Quality Forum; 2003 September 19. Pediatric indicators Inpatient Small subset (~10) feasible with restricted data

6. Pediatric Applications of AHRQ QIs Miller et al., Sedman et al., NACHRI chart reviews Miller et al., Sedman et al., NACHRI chart reviews Lessons learned Complications DO occur in children Complications DO occur in children Some complications clinically different Some complications clinically different Some indicators perform differently in kids or rare with current exclusions Some indicators perform differently in kids or rare with current exclusions Death related PSIs seemed less useful as defined in kids Death related PSIs seemed less useful as defined in kids

7. Indicator Module Development Literature Actual Use Concept SOURCES Candidate Indicators Evaluation Selection

8. Framework for Assessing Pediatric Indicator Validity Face validity/consensual validity Face validity/consensual validity – Does the indicator capture an aspect of quality that is important and subject to provider control? Precision Precision – Is there substantial “true” provider-level variation? Minimum bias Minimum bias – Is it possible to account for differences in severity of illness that could potentially confound comparisons across providers? Construct validity Construct validity – Does the indicator identify quality of care problems that are flagged or suspected using other methods? Fosters real quality improvement Fosters real quality improvement – Is the indicator unlikely to be gamed or cause perverse incentives? Application/experience Application/experience – Is there reason to believe the indicator will be feasible and useful?

9. Literature review Literature review – To identify quality concepts and indicators – To determine previous work on indicator validity Hospital ICD-9-CM coding review Hospital ICD-9-CM coding review – To ensure proper definition (correspondence between clinical concept and coding practice) Clinical panel reviews Clinical panel reviews – To refine indicator definition and risk groupings – To establish face validity when minimal literature Empirical analyses Empirical analyses – To explore alternative definitions – To assess nationwide rates, hospital variation, relationships among indicators – To develop appropriate methods to account for differences in underlying risk Indicator Development

10. Phased Evaluation Phase I Phase I – Current AHRQ QIs Eliminate QIs covering adult only chronic illnesses or those with questionable validity for kids Eliminate QIs covering adult only chronic illnesses or those with questionable validity for kids Phase II Phase II – Novel indicators Require development or updating Require development or updating

11. Example Indicator Evaluation PANEL EVALUATION FURTHER EMPIRICAL ANALYSES REFINED DEF. FURTHER REVIEW? FINAL DEFINITION INITIAL EMPRICAL ANALYSES AND DEFINITION LITERATURE REVIEW USER DATA

13. Decubitus ulcer Patients with secondary dx 707.0 per 1000 patients Exclude high risk patients: Transfers from long term care facility, paralysis EXCLUDE SPINA BIFIDA PATIENTS Literature Review and User Data.

14. Initial Empirical Results Rates by age group and high risk groups Rates by age group and high risk groups – Higher rate in higher age groups – Ulcers occur more frequently in high risk groups but some occur in traditionally low risk – Lower rate in premature neonates Rates are provided without commentary to panelists prior to conference Rates are provided without commentary to panelists prior to conference.

15. Medical/Surgical Panel Composition SpecialtyLocation Pediatric Emergency MedicineDallas, TX Pediatric Emergency MedicineDallas, TX Thoracic Surgery, Congenital Heart SurgeryWashington, DC Thoracic Surgery, Congenital Heart SurgeryWashington, DC NeonatologySeattle, WA NeonatologySeattle, WA Neonatal & Pediatric NursingSan Francisco, CA Neonatal & Pediatric NursingSan Francisco, CA Pediatric Surgery, Surgical Critical CareNew Haven, CT Pediatric Surgery, Surgical Critical CareNew Haven, CT Pediatric Critical CareLouisville, KY Pediatric Critical CareLouisville, KY Pediatric Infectious DiseaseAugusta, GA Pediatric Infectious DiseaseAugusta, GA Pediatric General SurgeryNashville, TN Pediatric General SurgeryNashville, TN PediatricsValhalla, NY PediatricsValhalla, NY Pediatric Radiology, Diagnostic RadiologySeattle, WA Pediatric Radiology, Diagnostic RadiologySeattle, WA Pediatric OncologyNew York, NY Pediatric OncologyNew York, NY HospitalistPhiladelphia, PA HospitalistPhiladelphia, PA.

16. Panel Evaluation Expand population to INCLUDE high risk populations Expand population to INCLUDE high risk populations Prefer stratification scheme Prefer stratification scheme Skin breakdown in neonates Skin breakdown in neonates.

17. Post-Panel Investigation Empirical analyses Empirical analyses – Examine rates of decubitus ulcer in potentially high risk groups. – Identify similar risk strata Coding consult Coding consult – Understand coding guidelines for infants with “skin breakdown” or decubiti.

18. Example Evaluation Revised Definition for Decubitus Ulcer Patients with secondary dx of 707x per 1000 patients Exclude patients transferred from long term care facility and another acute care facility Stratify by: - Low Risk - High risk (paralysis, spina bifida, anoxic brain damage).

21. Results Overarching Themes High risk populations are important in children High risk populations are important in children Bias and risk groups Bias and risk groups Expanded data Expanded data Application of indicators key Application of indicators key Feedback and validity testing key Feedback and validity testing key

22. Types of Modifications Made to QIs Expand population at risk Expand population at risk – Decubitus ulcer, postoperative sepsis Restricted age range Restricted age range – Transfusion reaction, Diabetes, Asthma, Perforated appendix – Exclusion of normal newborns Stratification/split Stratification/split – Iatrogenic pneumothorax, Accidental puncture laceration, Post-op hemorrhage/hematoma Added exclusion criteria Added exclusion criteria – Post-op wound dehiscence, Post-op respiratory failure, UTI Modified numerator Modified numerator – Gastroenteritis

23. Indicators Not Recommended Clinically different in children Clinically different in children Likely to occur in complex cases in children/ preventability questionable Likely to occur in complex cases in children/ preventability questionable Coding concerns Coding concerns – Bacterial pneumonia – PO physiologic and metabolic derangement Combined with other indicator, remaining cases not useful Combined with other indicator, remaining cases not useful – Dehydration

24. Rates per 1000 Procedure-related Complications

25. Rates per 1000 Complications in All Patients

26. Rates per 1000 Postoperative Complications

27. Rates per 100,000 population Potentially Avoidable Hospitalizations

28. Rates (%) Mortality Indicators

29. Dealing with Bias Stratification Stratification – Clinically transparent, actual numbers – Low numbers, overwhelming number of results Risk adjustment Risk adjustment – Allows for comparisons – Full adjustment impossible, black box Exclusions Exclusions – Easy comparisons, complex cases avoided – Low numbers, leaves out cases important to prevent

30. Risk Adjustment Reason for admission/ type of procedure Reason for admission/ type of procedure – DRGs Comorbidity Comorbidity – Must develop de novo SES risk adjustment SES risk adjustment – Not unique to kids, but may over-adjust

31. Phase II: Novel Indicators Literature review Literature review Organization contact Organization contact – Federal agencies, professional organizations, advocacy groups, provider organizations – 100+ contacted – Most indicators submitted not feasible given data constraints

32. Indicators Under Consideration Ambulatory Care Cellulitis hospitalization rate Cellulitis hospitalization rate Hospital admissions for influenza-related conditions, age 6-23 months Hospital admissions for influenza-related conditions, age 6-23 months Immunizable condition hospitalization rate Immunizable condition hospitalization rate

33. Indicators Under Consideration Neonatal Intraventricular hemorrhage Intraventricular hemorrhage Respiratory distress syndrome Respiratory distress syndrome Chronic respiratory disease Chronic respiratory disease Meconium aspiration syndrome rate Meconium aspiration syndrome rate Nectrotizing enterocolitis Nectrotizing enterocolitis Neonatal mortality Neonatal mortality Nosocomial bacteremia Nosocomial bacteremia Proportion of VLBW infants born at Level III centers Proportion of VLBW infants born at Level III centers Retinopathy of prematurity Retinopathy of prematurity

34. Indicators Under Consideration Patient Safety and Mortality Aspiration pneumonia Aspiration pneumonia Postoperative pneumonia Postoperative pneumonia Catheter-associated venous thrombosis Catheter-associated venous thrombosis Other postoperative metabolic derangements (hyponatremia, hypernatremia) Other postoperative metabolic derangements (hyponatremia, hypernatremia) Trauma mortality Trauma mortality

35. Phase II: Next Steps Literature reviews Literature reviews Update existing definitions Update existing definitions Develop and test definitions using administrative data Develop and test definitions using administrative data Panel review Panel review Reformulation of indicators Reformulation of indicators Development and release of new software Development and release of new software

36. Timeline January 2006 January 2006 – PedQI software release with current AHRQ QIs adapted for pediatric cases Fall/Winter 2005 Fall/Winter 2005 – PQI, IQI, PSI updates converted to adult population focus Early 2007 Early 2007 – PedQI update with new indicators

37. Implications AHRQ PSIs, IQIs and PQIs AHRQ PSIs, IQIs and PQIs – No longer apply to children, though concepts retained in PedQI Children’s vs. community hospitals Children’s vs. community hospitals – Focus on strata for stratified indicators – Compare results within peer groups Request to users Request to users – Monitoring of coding practices essential – Communication to AHRQ about early experiences

38. Acknowledgments Funded by AHRQ Support for Quality Indicators II (Contract No. 290-04-0020) Mamatha Pancholi, AHRQ Project Officer Marybeth Farquhar, AHRQ QI Senior Advisor Mark Gritz and Jeffrey Geppert, Project Directors, Battelle Health and Life Sciences Data used for analyses: Nationwide Inpatient Sample (NIS), 1995-2000. Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality State Inpatient Databases (SID), 1997-2002 (36 states). Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality

39. Acknowledgements We gratefully acknowledge the data organizations in participating states that contributed data to HCUP and that we used in this study: the Arizona Department of Health Services; California Office of Statewide Health Planning & Development; Colorado Health & Hospital Association; Connecticut - Chime, Inc.; Florida Agency for Health Care Administration; Georgia: An Association of Hospitals & Health Systems; Hawaii Health Information Corporation; Illinois Health Care Cost Containment Council; Iowa Hospital Association; Kansas Hospital Association; Kentucky Department for Public Health; Maine Health Data Organization; Maryland Health Services Cost Review; Massachusetts Division of Health Care Finance and Policy; Michigan Health & Hospital Association; Minnesota Hospital Association; Missouri Hospital Industry Data Institute; Nebraska Hospital Association; Nevada Department of Human Resources; New Jersey Department of Health & Senior Services; New York State Department of Health; North Carolina Department of Health and Human Services; Ohio Hospital Association; Oregon Association of Hospitals & Health Systems; Pennsylvania Health Care Cost Containment Council; Rhode Island Department of Health; South Carolina State Budget & Control Board; South Dakota Association of Healthcare Organizations; Tennessee Hospital Association; Texas Health Care Information Council; Utah Department of Health; Vermont Association of Hospitals and Health Systems; Virginia Health Information; Washington State Department of Health; West Virginia Health Care Authority; Wisconsin Department of Health & Family Services. We gratefully acknowledge the data organizations in participating states that contributed data to HCUP and that we used in this study: the Arizona Department of Health Services; California Office of Statewide Health Planning & Development; Colorado Health & Hospital Association; Connecticut - Chime, Inc.; Florida Agency for Health Care Administration; Georgia: An Association of Hospitals & Health Systems; Hawaii Health Information Corporation; Illinois Health Care Cost Containment Council; Iowa Hospital Association; Kansas Hospital Association; Kentucky Department for Public Health; Maine Health Data Organization; Maryland Health Services Cost Review; Massachusetts Division of Health Care Finance and Policy; Michigan Health & Hospital Association; Minnesota Hospital Association; Missouri Hospital Industry Data Institute; Nebraska Hospital Association; Nevada Department of Human Resources; New Jersey Department of Health & Senior Services; New York State Department of Health; North Carolina Department of Health and Human Services; Ohio Hospital Association; Oregon Association of Hospitals & Health Systems; Pennsylvania Health Care Cost Containment Council; Rhode Island Department of Health; South Carolina State Budget & Control Board; South Dakota Association of Healthcare Organizations; Tennessee Hospital Association; Texas Health Care Information Council; Utah Department of Health; Vermont Association of Hospitals and Health Systems; Virginia Health Information; Washington State Department of Health; West Virginia Health Care Authority; Wisconsin Department of Health & Family Services.

40. Questions?