This presentation will probably involve audience discussion, which will create action items. Use PowerPoint to keep track of these action items during your presentation In Slide Show, click on the right mouse button Select “Meeting Minder” Select the “Action Items” tab Type in action items as they come up Click OK to dismiss this box This will automatically create an Action Item slide at the end of your presentation with your points entered. Designing A New Performance Measurement System for Maternal and Child Health in the US Michael D. Kogan, PhD Director, Office of Epidemiology and Research Maternal and Child Health Bureau Health Resources and Services Administration Early Childhood National Conference September 2014

Outline Background on the Maternal and Child Health Bureau Background on the Maternal and Child Health Block Grant Definitions and history of performance measures Why a transformation was needed Guiding principles for change The new performance measure system Moving towards evidence-based systems

Background on Why We are Your Long-Lost Cousins at the Maternal and Child Health Bureau

Early Childhood Programs Home Visiting Autism Systems of Care for Children with Special Health Care Needs Selected on the basis of measurability, impact, Title V priority activities

Data and Research The National Survey of Children’s Health About 33,000 children from 0-5 Focus on the health and well-being of children Childhealthdata.org Selected on the basis of measurability, impact, Title V priority activities

The Title V Maternal and Child Health Block Grant

Transformation of the MCH Services Block Grant Mission To improve the health and well-being of all of America’s mothers, infants, children and youth, including children and youth with special healthcare needs, and their families I’ve been at the vision for a year now, my vision has gotten clearer but it’s still not quite 20/20 yet. But here’s my working vision statement: We are going to improve maternal and child health in our nation by improving access, quality, integration, accountability, and equity. 7

History Began in 1935 as part of the Social Security Act Became a block grant in 1981 Greater standards for accountability in 1993 (Government Performance Results Act) First National Performance Measures and National Outcome Measures in 1998

The Current Block Grant Process Yearly funding from Congress States conduct needs assessment every 5 years States identify their MCH priority needs and develop state performance measures Report on national and state performance measures and legislatively required health data States meet with MCHB yearly to review progress

Definitions of Performance Measurement

The process of quantifying the efficiency and effectiveness of past actions (Neely) The process of evaluating how well organizations are managed and the value they deliver for customers and other stakeholders (Moullin)

Definitions of Performance Measurement (GAO) The ongoing monitoring and reporting of program accomplishments, particularly progress towards pre-established goals. Performance measures may address the type or level of program activities conducted (process), the direct products and services delivered by a program (outputs), and/or the results of those products and services (outcomes).

Why was a Transformation of the Performance Measure System Needed?

Changes in MCH Data Systems

The National Survey of Children’s Health The American Community Survey The revised US birth certificates The Pregnancy Risk Assessment Monitoring System

Changes in Performance Measurement

Healthy People 2020 Children’s Health Insurance Program Reauthorization Act’s Quality Improvement Measures The National Quality Forum

Changes in MCH Research

Changes in MCH Risk Factors and Outcomes

Preterm birth rates are 13% higher than in 1990. Teen birth rates have declined over 50% since 1991. The infant mortality rate has declined since 1990, but the US ranks 26th in the world. The percent of children with chronic conditions has increased, particularly for developmental and behavioral conditions. Childhood obesity is 60% greater than in 1990.

Changes in MCH Health Services

Cesarean section rates have increased over 50% since 1996. The percent of uninsured children declined from 13.9% in 1997 to 6.5% in 2013. The percent of children reported to be underinsured has increased slightly between 2003 and 2012, especially for children with special health care needs.

Budgetary Constraints

Transformation

Guiding Principals for Change

Reduce the reporting burden of States Increase flexibility Improve accountability

Current Title V Performance Measures and Evaluation Since 1998, States have reported annually on both National and State Performance Measures Tracked performance on MCH issues across populations 18 National Performance Measures, used by States for evaluation programs Data reported by States made available publicly in the Title V Information System Web Reports https://mchdata.hrsa.gov/tvisreports/

More Challenges There was not reliable data for some measures. The 6 National Outcome Measures weren’t tied to the National Performance Measures It was difficult to tie the national Title V measures to the State Title V programs. Comparability across States was impossible for many measures because of different data sources.

Transformation Reduce burden Reducing data reporting States can choose 8 out of 15 National Performance Measures (NPMs) MCHB will provide data for NPMs and National Outcome Measures (NOMs), when possible Increase flexibility Choice in NPMs State-specific performance measures (SPMs) State-developed structural/process measures (S/PMs)

Transformation 3. Improve accountability and document impact Measurable Title V activities directly addressing the chosen performance measures.

Performance Measure Framework National Outcome Measures National Performance Measures State-Initiated Structure / Process Measures

Criteria for National Outcome Measures Data collection mandated by Title V legislation. Considered sentinel health marker. Focus of either Title V legislation or activities. Important health condition to monitor because prevalence is increasing. Recognized need to move the field forward.

National Outcome Measures Infant mortality, preterm-related mortality, neonatal mortality, post-neonatal mortality, perinatal mortality, sleep-related SUID mortality Low birthweight, moderately low birthweight, very low birthweight Preterm birth, early preterm birth, late preterm birth, early term birth, early elective delivery Children in excellent or very good health Immunizations for children and adolescents Overweight and obesity

National Outcome Measures Child mortality Children without health insurance Children’s oral health conditions Adolescent mortality, adolescent motor vehicle mortality, adolescent suicide Systems of care for children with special health care needs (CSHCN) Prevalence of CSHCN, autism spectrum disorders, attention deficit disorders, mental/behavioral conditions Maternal morbidity and mortality Healthy and ready to learn

Performance Measure Domains Women’s/Maternal Health Perinatal/Infant Health Child Health Adolescent Health CYSHCN Cross-cutting or Life Course Selected on the basis of measurability, impact, Title V priority activities

Criteria for National Performance Measures Large investment of resources. Considered modifiable by Title V activities. State could delineate measurable activities. Significant disparities existed. Condition had large societal costs. Associated with at least one NOM.

Women’s/Maternal Health Well-woman visit (BRFSS) Definition: % of women 18-44 with past-year preventive visit Low-risk cesarean (Birth certificate) Definition: % cesarean among term, singleton, vertex, first births

Perinatal/Infant Health Perinatal Regionalization (Linked Birth – AAP Directory) Definition: % VLBWs born in facilities with level III+ NICUs Breastfeeding (NIS) Definition: % infants ever breastfed Safe Sleep (PRAMS) Definition: % infants placed to sleep in a safe sleep environment

Young Children’s Health Developmental Screening (NSCH) Definition: % children ages 9-71 months receiving a developmental screening using a parent-completed screening tool Potential outcomes Healthy and Ready to Learn Children with special health care needs Autism spectrum disorder, attention deficit disorder, developmental and behavioral conditions

Adolescent Health Adolescent well-visit (NSCH) Definition: % of adolescents aged 12-17 with a well-visit in the past year Bullying (YRBSS and/or NSCH) Definition: % adolescents who report being bullied

Child and Adolescent Health Injury (HCUP – State Inpatient Databases) Definition: Rate of injury-related hospitalizations per population aged 0-19 Potential outcome Child death rate Adequate insurance is defined by these criteria: child currently has health insurance coverage AND benefits usually or always meet child's needs AND usually or always allow child to see needed providers AND either no out-of-pocket expenses or out-of-pocket expenses are usually or always reasonable.

Child and Adolescent Health Physical Activity (YRBSS and NSCH) Definition: % of children ages 6-17 who are physically active at least 60 minutes per day

Children with Special Health Care Needs Medical Home (NSCH) Definition: % children with and without CSHCN that have a medical home Transition (NSCH) Definition: % adolescents ages 12-17 with and without CSHCN who received services necessary to make transitions to adult health care Potential outcome Percent of children and youth with special health care needs (CYSHCN) receiving care in a well-functioning system Medical home is accessible, family-centered, continuous, comprehensive, coordinated, compassionate and culturally effective Indicator 4.9: Personal doctor or nurse - pdn - Indicator 4.9a: Usual source for sick and well care - usual - Indicator 4.9b: Family-centered care - famcent - Indicator 4.9c: Problems getting needed referrals - norefprob - Indicator 4.9d: Effective Care Coordination when needed - carecoor. System of services for CSHCN (partners in decision making and satisfied with services, coordinated ongoing care in medical home, adequate services to pay for needed services, community-based service systems are organized so they can use them effectively, received services to make transitions to all aspects of adult life, healthcare, work, and independence)

Cross-cutting or Life Course Oral Health (NSCH and PRAMS) Definitions: % of women who had a dental visit during pregnancy and % children ages 1-6 with a past-year preventive dental visit

Cross-cutting or Life Course Smoking (NSCH and NVSS) Definition: % of women who smoke during pregnancy and % children in households where someone smokes

Cross-cutting or Life Course Adequate Insurance Coverage (NSCH) Definition: % children who are adequately insured (continuous)

Criteria for State-Initiated Structural / Process Measures Activities had to be measurable. Evidence that the activity was related to the performance measure chosen. Development should be guided through an examination of the evidence-based best practices.

JUST BECAUSE YOU HAVE DATA DOESN’T MEAN YOU KNOW WHAT TO DO WITH IT

Definition of Evidence-Based Public Health “EBPH is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of communities and populations in the domain of health protection, disease prevention, health maintenance and improvement.” Jenicek (1997) Key terms in this definition of EBPH are… EXPLICIT: Use defined methods JUDICIOUS: Use judgment BEST EVIDENCE: Sort through the literature and identify what is useful Jenicek M. J Epidemiol 1997;7:187-97

So what is “best evidence”?

Best Evidence Makes sense (it’s relevant) Unbiased Available Statistically significant Significant to public health Leads to correct decisions

Evidence Statistical significance Meaningful to Public Health BOTH good best fair We have been taught to accept statistical significance. If large samples (as in many cases), we are bound to have it, even if it is not meaningful.

Steps of Evidence-Based Public Health Develop an initial statement of the issue Search the scientific literature and organize information Quantify the issue using sources of existing data Develop and prioritize program options; implement interventions Evaluate the program or policy These are the steps of Evidence-Based Public Health (EBPH) as described by Brownson et al. Brownson RC. J Public Health Manag Pract 1999;5:86-87

Different Sources of Evidence in Public Health “Soft information”: review processes, personal information, gut feelings “Adequate information”: routinely collected information, case review programs “Strong information”: active surveillance, and some clinical studies “Very strong”: randomized control trials

Evidence-Based Maternal and Child Health True or false? For women who are experiencing problems with their pregnancy, bed rest is effective in preventing preterm labor.

Evidence-Based Maternal and Child Health FALSE: Obstetric practices for which there is little evidence of effectiveness in preventing or treating preterm labor include bed rest. Goldenberg, Obstetrics and Gynecology, 2002

Are evidence-based approaches only applicable to the health field?

Evidence-Based Baseball Evidence-based approach by Oakland Athletics Relied on theoretically relevant statistics and scientific approach to baseball. Achieved winning seasons despite being burdened with severe budget constraints.

Evidence-Based Baseball What is the biggest predictor of runs scored by a team over a season: Number of home-runs? Team batting average? On-base percentage? Number of steals?

Are evidence-based approaches sufficient?

Not always Sometimes MCH outcomes are affected by issues in other areas

Day of the Week: Delivery Route Index of Occurrence of Delivery Route: Florida 2004-2006* Singletons, 34-41 Weeks, No Previous Cesarean, Low Documented Risk, and No Medical Induction (N=263,326) 140 120 100 Index of Occurrence 80 Vaginal 60 Cesarean with Labor Further evidence to suggest more active management is seen in this slide. Index of occurrence of delivery route by day of the week. The red line represents the average number of deliveries on a given day of the week by delivery route. Points above mean higher than average, and points below mean lower than average. You can see that cesareans without a trial of labor follow vaginal deliveries very closely, but that cesareans without a trial of labor have a sharp peak on Fridays. Cesarean without Labor 40 20 Sun Mon Tue Wed Thu Fri Sat Day of Week Goodman, et al, 2008.

Day of the Week: Late Preterm Index of Occurrence of Late Preterm: Florida 2004-2006* Singletons, 34-41 Weeks, No Previous Cesarean, Low Documented Risk, and No Medical Induction (N=263,326) 140 120 100 Index of Occurrence 80 Vaginal 60 Cesarean with Labor And to highlight the association of cesarean without trial of labor with late-preterm, this is the index of occurrence of late-preterm, by delivery route. You can see it follows a similar pattern to that on the previous slide. Cesarean without Labor 40 20 Sun Mon Tue Wed Thu Fri Sat Day of Week Goodman, et al, 2008.

Framework Measure Example NOM: Infant and Postneonatal Mortality, Sudden Unexpected Infant Deaths NPM: Percent of infants placed to sleep on their backs (Healthy People 2020 indicator) Possible State-Initiated S/PMs: Number of education sessions on safe sleep practices conducted in clinics or by the health department Number and percent of birthing hospitals that have received formal training from the MCH Department on safe sleep position Implementation of public service announcements (PSA) to raise awareness of safe sleep broadly and/or through partner organizations Number of “train the trainer” sessions on safe sleep conducted in each health district in the state

Contact Information Michael D. Kogan, Ph.D. 301-443-3145 mkogan@hrsa.gov