designing a new performance measurement system for maternal and child health in the us michael d....
TRANSCRIPT
Designing A New Performance Measurement System for
Maternal and Child Health in the US
Michael D. Kogan, PhDDirector, Office of Epidemiology and Research
Maternal and Child Health BureauHealth Resources and Services Administration
Early Childhood National ConferenceSeptember 2014
Outline• Background on the Maternal and Child Health
Bureau• Background on the Maternal and Child Health
Block Grant• Definitions and history of performance
measures• Why a transformation was needed• Guiding principles for change• The new performance measure system• Moving towards evidence-based systems
Background on Why We are Your Long-Lost
Cousins at the Maternal and Child Health Bureau
Early Childhood Programs
• Home Visiting• Autism• Systems of Care for Children with Special
Health Care Needs
Data and Research
• The National Survey of Children’s Health• About 33,000 children from 0-5• Focus on the health and well-being of children
• Childhealthdata.org
The Title V Maternal and Child Health Block Grant
Transformation of the MCH Services Block Grant
Mission
To improve the health and well-being of all of America’s mothers, infants, children and youth, including children and youth with special healthcare needs, and their families
History• Began in 1935 as part of the Social Security Act
• Became a block grant in 1981
• Greater standards for accountability in 1993 (Government Performance Results Act)
• First National Performance Measures and National Outcome Measures in 1998
The Current Block Grant Process• Yearly funding from Congress• States conduct needs assessment every 5
years• States identify their MCH priority needs and
develop state performance measures • Report on national and state performance
measures and legislatively required health data• States meet with MCHB yearly to review
progress
Definitions of Performance Measurement
The process of quantifying the efficiency and effectiveness of past actions (Neely)
The process of evaluating how well organizations are managed and the value they deliver for customers and other stakeholders (Moullin)
Definitions of Performance Measurement (GAO)
The ongoing monitoring and reporting of program accomplishments, particularly progress
towards pre-established goals. Performance measures may address the type or level of program activities conducted (process), the direct products and services delivered by a program (outputs), and/or the results of those products and services (outcomes).
Why was a Transformation of the Performance Measure System
Needed?
Changes in MCH Data Systems
• The National Survey of Children’s Health• The American Community Survey• The revised US birth certificates• The Pregnancy Risk Assessment Monitoring
System
Changes in Performance Measurement
• Healthy People 2020• Children’s Health Insurance Program
Reauthorization Act’s Quality Improvement Measures
• The National Quality Forum
Changes in MCH Research
Changes in MCH Risk Factors and Outcomes
• Preterm birth rates are 13% higher than in 1990.
• Teen birth rates have declined over 50% since 1991.
• The infant mortality rate has declined since 1990, but the US ranks 26th in the world.
• The percent of children with chronic conditions has increased, particularly for developmental and behavioral conditions.
• Childhood obesity is 60% greater than in 1990.
Changes in MCH Health Services
• Cesarean section rates have increased over 50% since 1996.
• The percent of uninsured children declined from 13.9% in 1997 to 6.5% in 2013.
• The percent of children reported to be underinsured has increased slightly between 2003 and 2012, especially for children with special health care needs.
Budgetary Constraints
Transformation
Guiding Principals for Change
• Reduce the reporting burden of States• Increase flexibility• Improve accountability
Current Title V Performance Measures and Evaluation
• Since 1998, States have reported annually on both National and State Performance Measures
• Tracked performance on MCH issues across populations
• 18 National Performance Measures, used by States for evaluation programs
• Data reported by States made available publicly in the Title V Information System Web Reports
• https://mchdata.hrsa.gov/tvisreports/
More Challenges
• There was not reliable data for some measures.
• The 6 National Outcome Measures weren’t tied to the National Performance Measures
• It was difficult to tie the national Title V measures to the State Title V programs.
• Comparability across States was impossible for many measures because of different data sources.
Transformation1. Reduce burden
– Reducing data reporting– States can choose 8 out of 15 National
Performance Measures (NPMs)– MCHB will provide data for NPMs and National
Outcome Measures (NOMs), when possible
2. Increase flexibility– Choice in NPMs– State-specific performance measures (SPMs)– State-developed structural/process measures
(S/PMs)
Transformation
3. Improve accountability and document impact –Measurable Title V activities directly
addressing the chosen performance measures.
Performance Measure Framework
National Outcome Measures
National Performance Measures
State-Initiated Structure / Process Measures
Criteria for National Outcome Measures
• Data collection mandated by Title V legislation.• Considered sentinel health marker.• Focus of either Title V legislation or activities.• Important health condition to monitor because
prevalence is increasing.• Recognized need to move the field forward.
National Outcome Measures
• Infant mortality, preterm-related mortality, neonatal mortality, post-neonatal mortality, perinatal mortality, sleep-related SUID mortality
• Low birthweight, moderately low birthweight, very low birthweight
• Preterm birth, early preterm birth, late preterm birth, early term birth, early elective delivery
• Children in excellent or very good health• Immunizations for children and adolescents• Overweight and obesity
National Outcome Measures• Child mortality• Children without health insurance• Children’s oral health conditions• Adolescent mortality, adolescent motor vehicle
mortality, adolescent suicide• Systems of care for children with special health care
needs (CSHCN)• Prevalence of CSHCN, autism spectrum disorders,
attention deficit disorders, mental/behavioral conditions
• Maternal morbidity and mortality• Healthy and ready to learn
Performance Measure Domains
• Women’s/Maternal Health• Perinatal/Infant Health• Child Health• Adolescent Health• CYSHCN• Cross-cutting or Life Course
Criteria for National Performance Measures
• Large investment of resources.• Considered modifiable by Title V activities.• State could delineate measurable activities.• Significant disparities existed.• Condition had large societal costs.• Associated with at least one NOM.
Women’s/Maternal Health
Well-woman visit (BRFSS)
Definition: % of women 18-44 with past-year preventive visit
Low-risk cesarean (Birth certificate)
Definition: % cesarean among term, singleton, vertex, first births
Perinatal/Infant HealthPerinatal Regionalization (Linked Birth – AAP Directory)
Definition: % VLBWs born in facilities with level III+ NICUs
Breastfeeding (NIS)
Definition: % infants ever breastfed
Safe Sleep (PRAMS)
Definition: % infants placed to sleep in a safe sleep environment
Young Children’s HealthDevelopmental Screening (NSCH)
Definition: % children ages 9-71 months receiving a developmental screening using a parent-completed screening tool
• Potential outcomes Healthy and Ready to Learn Children with special health care needs Autism spectrum disorder, attention deficit
disorder, developmental and behavioral conditions
Adolescent Health
Adolescent well-visit (NSCH)
Definition: % of adolescents aged 12-17 with a well-visit in the past year
Bullying (YRBSS and/or NSCH)
Definition: % adolescents who report being bullied
Child and Adolescent Health
Injury (HCUP – State Inpatient Databases)
Definition: Rate of injury-related hospitalizations per population aged 0-19
• Potential outcome Child death rate
Child and Adolescent Health
Physical Activity (YRBSS and NSCH)
Definition: % of children ages 6-17 who are physically active at least 60 minutes per day
Children with Special Health Care Needs
Medical Home (NSCH)Definition: % children with and without CSHCN that have a medical home
Transition (NSCH)Definition: % adolescents ages 12-17 with and without CSHCN who received services necessary to make transitions to adult health care
• Potential outcome Percent of children and youth with special health care
needs (CYSHCN) receiving care in a well-functioning system
Cross-cutting or Life Course
Oral Health (NSCH and PRAMS)
Definitions: % of women who had a dental visit during pregnancy and % children ages 1-6 with a past-year preventive dental visit
Cross-cutting or Life Course
Smoking (NSCH and NVSS)
Definition: % of women who smoke during pregnancy and % children in households where someone smokes
Cross-cutting or Life Course
Adequate Insurance Coverage (NSCH)
Definition: % children who are adequately insured (continuous)
Criteria for State-Initiated Structural / Process Measures
• Activities had to be measurable.• Evidence that the activity was related to the
performance measure chosen.• Development should be guided through an
examination of the evidence-based best practices.
JUST BECAUSE YOU HAVE DATA DOESN’T MEAN YOU KNOW WHAT TO DO WITH
IT
Jenicek M. J Epidemiol 1997;7:187-97
Definition of Evidence-Based Public Health
• “EBPH is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of communities and populations in the domain of health protection, disease prevention, health maintenance and improvement.”
Jenicek (1997)
So what is “best evidence”?
Best Evidence
• Makes sense (it’s relevant)• Unbiased• Available• Statistically significant• Significant to public health• Leads to correct decisions
Evidence
Statistical significance
Meaningful to Public HealthBOTH
good best fair
We have been taught to accept statistical significance. If large samples (as in many cases), we are bound to have it, even if it is not meaningful.
Brownson RC. J Public Health Manag Pract 1999;5:86-87
Steps of Evidence-Based Public Health
• Develop an initial statement of the issue• Search the scientific literature and organize
information• Quantify the issue using sources of existing
data• Develop and prioritize program options;
implement interventions• Evaluate the program or policy
Different Sources of Evidence in Public Health
• “Soft information”: review processes, personal information, gut feelings
• “Adequate information”: routinely collected information, case review programs
• “Strong information”: active surveillance, and some clinical studies
• “Very strong”: randomized control trials
Evidence-Based Maternal and Child Health
• True or false?• For women who are experiencing problems
with their pregnancy, bed rest is effective in preventing preterm labor.
Evidence-Based Maternal and Child Health
• FALSE:• Obstetric practices for which there is little
evidence of effectiveness in preventing or treating preterm labor include bed rest.– Goldenberg, Obstetrics and Gynecology, 2002
•Are evidence-based approaches only applicable to the
health field?
Evidence-Based Baseball
• Evidence-based approach by Oakland Athletics – Relied on theoretically relevant statistics and
scientific approach to baseball.– Achieved winning seasons despite being burdened
with severe budget constraints.
Evidence-Based Baseball
• What is the biggest predictor of runs scored by a team over a season:– Number of home-runs?– Team batting average?– On-base percentage?– Number of steals?
Are evidence-based approaches sufficient?
Not always•Sometimes MCH outcomes are affected by issues in other areas
Day of the Week: Delivery Route20
4060
8010
012
014
0
Ind
ex o
f O
ccu
rren
ce
Sun Mon Tue Wed Thu Fri SatDay of Week
Index of Occurrence of Delivery Route: Florida 2004-2006*Singletons, 34-41 Weeks, No Previous Cesarean, Low Documented Risk, and No Medical Induction (N=263,326)
Vaginal
Cesarean with Labor
Cesarean without Labor
Goodman, et al, 2008.
Day of the Week: Late PretermIndex of Occurrence of Late Preterm: Florida 2004-2006*Singletons, 34-41 Weeks, No Previous Cesarean, Low Documented Risk, and No Medical Induction (N=263,326)
2040
6080
100
120
140
Ind
ex o
f O
ccu
rren
ce
Sun Mon Tue Wed Thu Fri SatDay of Week
Vaginal
Cesarean with Labor
Cesarean without Labor
Goodman, et al, 2008.
Framework Measure Example• NOM: Infant and Postneonatal Mortality, Sudden Unexpected
Infant Deaths
• NPM: Percent of infants placed to sleep on their backs (Healthy People 2020 indicator)
• Possible State-Initiated S/PMs:1) Number of education sessions on safe sleep practices
conducted in clinics or by the health department
2) Number and percent of birthing hospitals that have received formal training from the MCH Department on safe sleep position
3) Implementation of public service announcements (PSA) to raise awareness of safe sleep broadly and/or through partner organizations
4) Number of “train the trainer” sessions on safe sleep conducted in each health district in the state
