1. This presentation will probably involve audience discussion, which will create action items. Use PowerPoint to keep track of these action items during your presentation
In Slide Show, click on the right mouse button
Select “Meeting Minder”
Select the “Action Items” tab
Type in action items as they come up
Click OK to dismiss this box
This will automatically create an Action Item slide at the end of your presentation with your points entered.
Designing A New Performance Measurement System for Maternal and Child Health in the US
Michael D. Kogan, PhD
Director, Office of Epidemiology and Research
Maternal and Child Health Bureau
Health Resources and Services Administration
Early Childhood National Conference
September 2014

2. Outline Background on the Maternal and Child Health Bureau
Background on the Maternal and Child Health Block Grant
Definitions and history of performance measures
Why a transformation was needed
Guiding principles for change
The new performance measure system
Moving towards evidence-based systems

3. Background on Why We are Your Long-Lost Cousins at the Maternal and Child Health Bureau

4. Early Childhood Programs
Home Visiting
Autism
Systems of Care for Children with Special Health Care Needs
Selected on the basis of measurability, impact, Title V priority activities

5. Data and Research The National Survey of Children’s Health
About 33,000 children from 0-5
Focus on the health and well-being of children
Childhealthdata.org
Selected on the basis of measurability, impact, Title V priority activities

6. The Title V Maternal and Child Health Block Grant

7. Transformation of the MCH Services Block Grant Mission
To improve the health and well-being of all of America’s mothers, infants, children and youth, including children and youth with special healthcare needs, and their families
I’ve been at the vision for a year now, my vision has gotten clearer but it’s still not quite 20/20 yet. But here’s my working vision statement:
We are going to improve maternal and child health in our nation by improving access, quality, integration, accountability, and equity. 7

8. History Began in 1935 as part of the Social Security Act
Became a block grant in 1981
Greater standards for accountability in 1993 (Government Performance Results Act)
First National Performance Measures and National Outcome Measures in 1998

9. The Current Block Grant Process
Yearly funding from Congress
States conduct needs assessment every 5 years
States identify their MCH priority needs and develop state performance measures
Report on national and state performance measures and legislatively required health data
States meet with MCHB yearly to review progress

10. Definitions of Performance Measurement

11. The process of quantifying the efficiency and effectiveness of past actions (Neely)
The process of evaluating how well organizations are managed and the value they deliver for customers and other stakeholders (Moullin)

12. Definitions of Performance Measurement (GAO)
The ongoing monitoring and reporting of program accomplishments, particularly progress
towards pre-established goals. Performance measures may address the type or level of program activities conducted (process), the direct products and services delivered by a program (outputs), and/or the results of those products and services (outcomes).

13. Why was a Transformation of the Performance Measure System Needed?

14. Changes in MCH Data Systems

15. The National Survey of Children’s Health
The American Community Survey
The revised US birth certificates
The Pregnancy Risk Assessment Monitoring System

16. Changes in Performance Measurement

17. Healthy People 2020
Children’s Health Insurance Program Reauthorization Act’s Quality Improvement Measures
The National Quality Forum

18. Changes in MCH Research

20. Changes in MCH Risk Factors and Outcomes

21. Preterm birth rates are 13% higher than in 1990.
Teen birth rates have declined over 50% since 1991.
The infant mortality rate has declined since 1990, but the US ranks 26th in the world.
The percent of children with chronic conditions has increased, particularly for developmental and behavioral conditions.
Childhood obesity is 60% greater than in 1990.

22. Changes in MCH Health Services

23. Cesarean section rates have increased over 50% since 1996.
The percent of uninsured children declined from 13.9% in 1997 to 6.5% in 2013.
The percent of children reported to be underinsured has increased slightly between 2003 and 2012, especially for children with special health care needs.

24. Budgetary Constraints

25. Transformation

26. Guiding Principals for Change

27. Reduce the reporting burden of States
Increase flexibility
Improve accountability

28. Current Title V Performance Measures and Evaluation
Since 1998, States have reported annually on both National and State Performance Measures
Tracked performance on MCH issues across populations
18 National Performance Measures, used by States for evaluation programs
Data reported by States made available publicly in the Title V Information System Web Reports

29. More Challenges There was not reliable data for some measures.
The 6 National Outcome Measures weren’t tied to the National Performance Measures
It was difficult to tie the national Title V measures to the State Title V programs.
Comparability across States was impossible for many measures because of different data sources.

30. Transformation Reduce burden Reducing data reporting
States can choose 8 out of 15 National Performance Measures (NPMs)
MCHB will provide data for NPMs and National Outcome Measures (NOMs), when possible
Increase flexibility
Choice in NPMs
State-specific performance measures (SPMs)
State-developed structural/process measures (S/PMs)

31. Transformation
3. Improve accountability and document impact
Measurable Title V activities directly addressing the chosen performance measures.

32. Performance Measure Framework
National Outcome Measures National Performance Measures State-Initiated Structure / Process Measures

33. Criteria for National Outcome Measures
Data collection mandated by Title V legislation.
Considered sentinel health marker.
Focus of either Title V legislation or activities.
Important health condition to monitor because prevalence is increasing.
Recognized need to move the field forward.

34. National Outcome Measures
Infant mortality, preterm-related mortality, neonatal mortality, post-neonatal mortality, perinatal mortality, sleep-related SUID mortality
Low birthweight, moderately low birthweight, very low birthweight
Preterm birth, early preterm birth, late preterm birth, early term birth, early elective delivery
Children in excellent or very good health
Immunizations for children and adolescents
Overweight and obesity

35. National Outcome Measures
Child mortality
Children without health insurance
Children’s oral health conditions
Adolescent mortality, adolescent motor vehicle mortality, adolescent suicide
Systems of care for children with special health care needs (CSHCN)
Prevalence of CSHCN, autism spectrum disorders, attention deficit disorders, mental/behavioral conditions
Maternal morbidity and mortality
Healthy and ready to learn

36. Performance Measure Domains
Women’s/Maternal Health
Perinatal/Infant Health
Child Health
Adolescent Health
CYSHCN
Cross-cutting or Life Course
Selected on the basis of measurability, impact, Title V priority activities

37. Criteria for National Performance Measures
Large investment of resources.
Considered modifiable by Title V activities.
State could delineate measurable activities.
Significant disparities existed.
Condition had large societal costs.
Associated with at least one NOM.

38. Women’s/Maternal Health
Well-woman visit (BRFSS) Definition: % of women with past-year preventive visit Low-risk cesarean (Birth certificate) Definition: % cesarean among term, singleton, vertex, first births

39. Perinatal/Infant Health
Perinatal Regionalization (Linked Birth – AAP Directory) Definition: % VLBWs born in facilities with level III+ NICUs Breastfeeding (NIS) Definition: % infants ever breastfed Safe Sleep (PRAMS) Definition: % infants placed to sleep in a safe sleep environment

40. Young Children’s Health
Developmental Screening (NSCH)
Definition: % children ages 9-71 months receiving a developmental screening using a parent-completed screening tool
Potential outcomes
Healthy and Ready to Learn
Children with special health care needs
Autism spectrum disorder, attention deficit disorder, developmental and behavioral conditions

41. Adolescent Health
Adolescent well-visit (NSCH) Definition: % of adolescents aged with a well-visit in the past year Bullying (YRBSS and/or NSCH) Definition: % adolescents who report being bullied

42. Child and Adolescent Health
Injury (HCUP – State Inpatient Databases)
Definition: Rate of injury-related hospitalizations per population aged 0-19
Potential outcome
Child death rate
Adequate insurance is defined by these criteria: child currently has health insurance coverage AND benefits usually or always meet child's needs AND usually or always allow child to see needed providers AND either no out-of-pocket expenses or out-of-pocket expenses are usually or always reasonable.

43. Child and Adolescent Health
Physical Activity (YRBSS and NSCH) Definition: % of children ages 6-17 who are physically active at least 60 minutes per day

44. Children with Special Health Care Needs
Medical Home (NSCH)
Definition: % children with and without CSHCN that have a medical home
Transition (NSCH)
Definition: % adolescents ages with and without CSHCN who received services necessary to make transitions to adult health care
Potential outcome
Percent of children and youth with special health care needs (CYSHCN) receiving care in a well-functioning system
Medical home is accessible, family-centered, continuous, comprehensive, coordinated, compassionate and culturally effective
Indicator 4.9: Personal doctor or nurse - pdn - Indicator 4.9a: Usual source for sick and well care - usual - Indicator 4.9b: Family-centered care - famcent - Indicator 4.9c: Problems getting needed referrals - norefprob - Indicator 4.9d: Effective Care Coordination when needed - carecoor.
System of services for CSHCN (partners in decision making and satisfied with services, coordinated ongoing care in medical home, adequate services to pay for needed services, community-based service systems are organized so they can use them effectively, received services to make transitions to all aspects of adult life, healthcare, work, and independence)

45. Cross-cutting or Life Course
Oral Health (NSCH and PRAMS) Definitions: % of women who had a dental visit during pregnancy and % children ages 1-6 with a past-year preventive dental visit

46. Cross-cutting or Life Course
Smoking (NSCH and NVSS)
Definition: % of women who smoke during pregnancy and % children in households where someone smokes

47. Cross-cutting or Life Course
Adequate Insurance Coverage (NSCH)
Definition: % children who are adequately insured (continuous)

48. Criteria for State-Initiated Structural / Process Measures
Activities had to be measurable.
Evidence that the activity was related to the performance measure chosen.
Development should be guided through an examination of the evidence-based best practices.

49. JUST BECAUSE YOU HAVE DATA DOESN’T MEAN YOU KNOW WHAT TO DO WITH IT

50. Definition of Evidence-Based Public Health
“EBPH is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of communities and populations in the domain of health protection, disease prevention, health maintenance and improvement.”
Jenicek (1997)
Key terms in this definition of EBPH are…
EXPLICIT: Use defined methods
JUDICIOUS: Use judgment
BEST EVIDENCE: Sort through the literature and identify what is useful
Jenicek M. J Epidemiol 1997;7:187-97

51. So what is “best evidence”?

52. Best Evidence Makes sense (it’s relevant) Unbiased Available
Statistically significant
Significant to public health
Leads to correct decisions

53. Evidence Statistical significance Meaningful to Public Health BOTH
good best fair
We have been taught to accept statistical significance. If large samples (as in many cases), we are bound to have it, even if it is not meaningful.

54. Steps of Evidence-Based Public Health
Develop an initial statement of the issue
Search the scientific literature and organize information
Quantify the issue using sources of existing data
Develop and prioritize program options; implement interventions
Evaluate the program or policy
These are the steps of Evidence-Based Public Health (EBPH) as described by Brownson et al.
Brownson RC. J Public Health Manag Pract 1999;5:86-87

55. Different Sources of Evidence in Public Health
“Soft information”: review processes, personal information, gut feelings
“Adequate information”: routinely collected information, case review programs
“Strong information”: active surveillance, and some clinical studies
“Very strong”: randomized control trials

56. Evidence-Based Maternal and Child Health
True or false?
For women who are experiencing problems with their pregnancy, bed rest is effective in preventing preterm labor.

57. Evidence-Based Maternal and Child Health
FALSE:
Obstetric practices for which there is little evidence of effectiveness in preventing or treating preterm labor include bed rest.
Goldenberg, Obstetrics and Gynecology, 2002

58. Are evidence-based approaches only applicable to the health field?

59. Evidence-Based Baseball
Evidence-based approach by Oakland Athletics
Relied on theoretically relevant statistics and scientific approach to baseball.
Achieved winning seasons despite being burdened with severe budget constraints.

60. Evidence-Based Baseball
What is the biggest predictor of runs scored by a team over a season:
Number of home-runs?
Team batting average?
On-base percentage?
Number of steals?

61. Are evidence-based approaches sufficient?

62. Not always
Sometimes MCH outcomes are affected by issues in other areas

63. Day of the Week: Delivery Route
Index of Occurrence of Delivery Route: Florida *
Singletons, Weeks, No Previous Cesarean, Low Documented Risk, and No Medical Induction (N=263,326)
140
120
100
Index of Occurrence 80
Vaginal 60
Cesarean with Labor
Further evidence to suggest more active management is seen in this slide. Index of occurrence of delivery route by day of the week. The red line represents the average number of deliveries on a given day of the week by delivery route.
Points above mean higher than average, and points below mean lower than average.
You can see that cesareans without a trial of labor follow vaginal deliveries very closely, but that cesareans without a trial of labor have a sharp peak on Fridays.
Cesarean without Labor 40 20
Sun
Mon
Tue
Wed
Thu
Fri
Sat
Day of Week
Goodman, et al, 2008.

64. Day of the Week: Late Preterm
Index of Occurrence of Late Preterm: Florida *
Singletons, Weeks, No Previous Cesarean, Low Documented Risk, and No Medical Induction (N=263,326)
140
120
100
Index of Occurrence 80
Vaginal 60
Cesarean with Labor
And to highlight the association of cesarean without trial of labor with late-preterm, this is the index of occurrence of late-preterm, by delivery route. You can see it follows a similar pattern to that on the previous slide.
Cesarean without Labor 40 20
Sun
Mon
Tue
Wed
Thu
Fri
Sat
Day of Week
Goodman, et al, 2008.

65. Framework Measure Example
NOM: Infant and Postneonatal Mortality, Sudden Unexpected Infant Deaths
NPM: Percent of infants placed to sleep on their backs (Healthy People 2020 indicator)
Possible State-Initiated S/PMs:
Number of education sessions on safe sleep practices conducted in clinics or by the health department
Number and percent of birthing hospitals that have received formal training from the MCH Department on safe sleep position
Implementation of public service announcements (PSA) to raise awareness of safe sleep broadly and/or through partner organizations
Number of “train the trainer” sessions on safe sleep conducted in each health district in the state

66. Contact Information
Michael D. Kogan, Ph.D.