"What gets measured better gets done better": The landscape of validation of global maternal and newborn health indicators through key informant interviews

Accountability for maternal and newborn health: Why measuring and monitoring broader social, political, and health system determinants matters

This article offers four key lessons learned from a set of seven studies undertaken as part of the collection entitled, "Improving Maternal Health Measurement to Support Efforts toward Ending Preventable Maternal Mortality". These papers were aimed at validating ten of the Ending Preventable Maternal Mortality initiative indicators that capture information on distal causes of maternal mortality. These ten indicators were selected through an inclusive consultative process, and the research designs adhere to global recommendations on conducting indicator validation studies. The findings of these papers are timely and relevant given growing recognition of the role of macro-level social, political, and economic factors in maternal and newborn survival. The four key lessons include: 1) Strengthen efforts to capture maternal and newborn health policies to enable global progress assessments while reducing multiple requests to countries for similar data; 2) Monitor indicator "bundles" to understand degree of policy implementation, inconsistencies between laws and practices, and responsiveness of policies to individual and community needs; 3) Promote regular monitoring of a holistic set of human resource metrics to understand how to effectively strengthen the maternal and newborn health workforce; and 4) Develop and disseminate clear guidance for countries on how to assess health system as well as broader social and political determinants of maternal and newborn health. These lessons are consistent with the Kirkland principles of focus, relevance, innovation, equity, global leadership, and country ownership. They stress the value of indicator sets to understand complex phenomenon related to maternal and newborn health, including small groupings of complementary indicators for measuring policy implementation and health workforce issues. They also stress the fundamental ethos that maternal and newborn health indicators should only be tracked if they can drive actions at global, regional, national, or sub-national levels that improve lives.

Improvements in Obstetric and Newborn Health Information Documentation following the Implementation of the Safer Births Bundle of Care at 30 Facilities in Tanzania

This paper examines changes in the completeness of documentation in clinical practice before and during the implementation of the Safer Births Bundle of Care (SBBC) project. This observational study enrolled parturient women with a gestation age of at least 28 weeks at the onset of labour. Data collectors extracted information from facility registers and then a central data manager summarised and reported weekly statistics. Variables of clinical significance for CQI were selected, and the proportion of non-documentation was analysed over time. A Pearson chi-square test was used to test for significant differences in non-documentation between the periods. Between 1 March 2021 and 31 July 2022, a total of 138,442 deliveries were recorded. Overall, 75% of all patient cases had at least one missing variable among the selected variables across both periods. A lack of variable documentation occurred more frequently at the district hospital level (81% of patient cases) and health centres (74%) than at regional referral hospitals (56%) (p < 0.001). Non-documentation decreased significantly from 79% to 70% after the introduction of the SBBC (p < 0.001). A tendency towards negative correlations was noted for most variables. We noted an increased attention to data quality and use which may have a positive impact on the completeness of documentation. However, halfway through the project's implementation, the completeness of documentation was still low. Our findings support the recommendation to establish short-spaced feedback loops of locally collected data using one data platform.

Intrapartum care measures and indicators for monitoring the implementation of WHO recommendations for a positive childbirth experience: a scoping review

OBJECTIVE

We aimed to identify all available studies describing measures or indicators used to monitor 41 intrapartum care practices described in the 2018 WHO intrapartum care recommendations, with a view to informing development of standardised measurement of implementing these recommendations.

DESIGN

Systematic scoping review.

METHODS

We conducted a scoping review to identify studies reporting measures of intrapartum care published between 1 January 2000 and 28 June 2021. Primary and secondary outcome measures included study characteristics (publication year, journal, country and World Bank classification) and intrapartum care measure characteristics (definition, numerator, denominator, measurement level and measurement approach). We searched MEDLINE, EMBASE, CINAHL, Cochrane Library, the Maternity and Infant Care Database, Global Index Medicus and grey literature using structured search terms related to included recommendations, focusing on respectful and supportive care, and clinical practices performed throughout labour and birth. The measures identified were classified by the WHO recommendation and their characteristics reported.

RESULTS

We identified 150 studies which described 1331 intrapartum care measures. These measures corresponded to 35 of the 41 included WHO recommendations, and represented all domains of the WHO recommendations (care throughout labour and birth, first stage of labour, second stage of labour, third stage of labour). A total of 40.1% (534 of 1331 measures) of measures were related to respectful maternity care. Most studies used a questionnaire or survey measurement approach (522 of 1331 measures, 39.2%).

CONCLUSION

This scoping review presents a database of existing intrapartum care measures used to monitor the quality of intrapartum care globally. There is no clear consensus on a core set of measures for evaluating the practice of the WHO's intrapartum care recommendations. This review provides a foundation to support the development of a core set of internationally standardised intrapartum care measures for the WHO intrapartum care recommendations, highlighting key areas requiring consensus and validation, and measure development.

A scoping review, mapping, and prioritisation process for emergency obstetric and neonatal quality of care indicators: Focus on provision and experience of care

Background

Globally, approximately 800 women and 6400 newborns die around the time of childbirth each day. Many of these deaths could be prevented with high-quality emergency obstetric and newborn care (EmONC). The Monitoring Emergency Obstetric Care: A handbook guides strengthening EmONC services. However, the handbook contains limited quality of care measures. Our study identified and prioritised quality of care indicators for potential inclusion in the handbook, which is undergoing revision.

Methods

We conducted a consultative scoping review, mapping, and prioritisation exercise to select a short list of indicators on facility-based maternal and newborn quality of care. Indicators were identified from literature searches and expert suggestions and organised by the categories of structure, process, and outcomes as defined in the World Health Organization's Standards for Improving Quality of Maternal and Newborn Care in Health Facilities. We focused on process indicators, encompassing the provision of care and experience of care during the intrapartum period, and developed a priority list of indicators using the selection criteria of relevance and feasibility. Experience of care indicators were also mapped against the Person-Centered Maternity Care (PCMC) scale.

Results

We extracted a total of 3023 quality of care indicators. After removing out-of-scope and duplicate indicators and applying our selection criteria, we identified 20 provision of care indicators for possible inclusion in the revised EmONC handbook. We recommend including a score for experience of care that could be measured with the 30-item or the 13-item PCMC scale. We also identified 29 experience of care items not covered by the PCMC scale that could be used. Provider experience, patient safety, and quality of abortion care were identified as areas for which no or few indicators were found through our scoping review.

Conclusions

Through a rigorous, consultative, and multi-step process, we selected a short list of process-related, facility-based quality of care indicators for emergency obstetric and newborn care. This list could be included in the EmONC handbook or used for other monitoring purposes. Country consultations to assess the utility and feasibility of the proposed indicators and their adaptation to local contexts will support their refinement and uptake.

Registration

https://osf.io/msxbd (Open Science Framework).

Validating indicators for monitoring availability and geographic distribution of emergency obstetric and newborn care (EmoNC) facilities: A study triangulating health system, facility, and geospatial data

Availability of emergency obstetric and newborn care (EmONC) is a strong supply side measure of essential health system capacity that is closely and causally linked to maternal mortality reduction and fundamentally to achieving universal health coverage. The World Health Organization's indicator "Availability of EmONC facilities" was prioritized as a core indicator to prevent maternal death. The indicator focuses on whether there are sufficient emergency care facilities to meet the population need, but not all facilities designated as providing EmONC function as such. This study seeks to validate "Availability of EmONC" by comparing the value of the indicator after accounting for key aspects of facility functionality and an alternative measure of geographic distribution. This study takes place in four subnational geographic areas in Argentina, Ghana, and India using a census of all birthing facilities. Performance of EmONC in the 90 days prior to data collection was assessed by examining facility records. Data were collected on facility operating hours, staffing, and availability of essential medications. Population estimates were generated using ArcGIS software using WorldPop to estimate the total population, and the number of women of reproductive age (WRA), pregnancies and births in the study areas. In addition, we estimated the population within two-hours travel time of an EmONC facility by incorporating data on terrain from Open Street Map. Using these data sources, we calculated and compared the value of the indicator after incorporating data on facility performance and functionality while varying the reference population used. Further, we compared its value to the proportion of the population within two-hours travel time of an EmONC facility. Included in our study were 34 birthing facilities in Argentina, 51 in Ghana, and 282 in India. Facility performance of basic EmONC (BEmONC) and comprehensive EmONC (CEmONC) signal functions varied considerably. One facility (4.8%) in Ghana and no facility in India designated as BEmONC had performed all seven BEmONC signal functions. In Argentina, three (8.8%) CEmONC-designated facilities performed all nine CEmONC signal functions, all located in Buenos Aires Region V. Four CEmONC-designated facilities in Ghana (57.1%) and the three CEmONC-designated facilities in India (23.1%) evidenced full CEmONC performance. No sub-national study area in Argentina or India reached the target of 5 BEmONC-level facilities per 20,000 births after incorporating facility functionality yet 100% did in Argentina and 50% did in India when considering only facility designation. Demographic differences also accounted for important variation in the indicator's value. In Ghana, the total population in Tolon within 2 hours travel time of a designated EmONC facility was estimated at 99.6%; however, only 91.1% of women of reproductive age were within 2 hours travel time. Comparing the value of the indicator when calculated using different definitions reveals important inconsistencies, resulting in conflicting information about whether the threshold for sufficient coverage is met. This raises important questions related to the indicator's validity. To provide a valid measure of effective coverage of EmONC, the construct for measurement should extend beyond the most narrow definition of availability and account for functionality and geographic accessibility.

Authorization of midwives to perform basic emergency obstetric and newborn care signal functions in Argentina, Ghana, and India: A multi-country validation study of a key global maternal and newborn health indicator

BACKGROUND

Midwives' authorization to deliver the seven basic emergency obstetric and newborn care (BEmONC) functions is a core policy indicator in global monitoring frameworks, yet little evidence supports whether such data are captured accurately, or whether authorization demonstrates convergence with midwives' skills and actual provision of services. In this study, we aimed to validate the data reported in global monitoring frameworks (criterion validity) and to determine whether a measure of authorization is a valid indicator for BEmONC availability (construct validity).

METHODS

We conducted a validation study in Argentina, Ghana, and India. To assess accuracy of the reported data on midwives' authorization to provide BEmONC services, we reviewed national regulatory documents and compared with reported country-specific data in Countdown to 2030 and the World Health Organization Maternal, Newborn, Child and Adolescent Health Policy Survey. To assess whether authorization demonstrates convergent validity with midwives' skills, training, and performance of BEmONC signal functions, we surveyed 1257 midwives/midwifery professionals and assessed variance.

RESULTS

We detected discrepancies between data reported in the global monitoring frameworks and the national regulatory framework in all three countries. We found wide variations between midwives' authorization to perform signal functions and their self-reported skills and actual performance within the past 90 days. The percentage of midwives who reported performing all signal functions for which they were authorized per country-specific regulations was 17% in Argentina, 23% in Ghana, and 31% in India. Additionally, midwives in all three countries reported performing some signal functions that the national regulations did not authorize.

CONCLUSION

Our findings suggest limitations in criterion and construct validity for this indicator in Argentina, Ghana, and India. Some signal functions such as assisted vaginal delivery may be obsolete based on current practice patterns. Findings suggest the need to re-examine the emergency interventions that should be included as BEmONC signal functions.

Demand Forecasting Approaches for New Contraceptive Technologies: A Landscape Review and Recommendations for Alignment

INTRODUCTION

Market size estimations and demand forecasts use a variety of methodological approaches to inform decision-making around new (and lesser-used) contraceptive methods. For contraceptive products already available at scale in a market, historical procurement and consumption data can help to inform these forecasts. However, little published guidance is available on appropriate approaches to estimating contraceptive demand in the absence of historical data.

METHODS

This landscape review aimed to describe the variety of approaches for modeling demand for new contraceptive methods, highlight opportunities for alignment around forecasting practices, and make recommendations to support more accurate forecasting and sound decision-making based on forecasts. We used the published scientific and gray literature to inform the development of a semistructured guide for key informant interviews. We conducted interviews with 29 experts representing a spectrum of interests in market size estimation and demand forecasting for new contraceptive methods (e.g., ministries of health, donors, manufacturers, technical assistance providers, and demand forecasting specialists). We coded notes from the interviews using thematic content analysis.

RESULTS

The purposes of market size estimation and demand forecasting for new contraceptive methods vary widely, as do associated model inputs and outputs. Key informants revealed a need for more standardized language around market size estimation and demand forecasting and highlighted key recommendations: select models that are fit-for-purpose, clearly articulate assumptions and uncertainty in model outputs, consider a broad range of contraceptive options in a forecast to capture the complete contraceptive supply environment, and perform a reality check of results and refresh assumptions.

CONCLUSION

We recommend following a simple decision pathway to ensure that forecasts are fit-for-purpose, with appropriate inputs, outputs, and assumptions clearly articulated. Common pitfalls around overestimating demand should be avoided. Incorporating best practices into forecasting exercises will ensure that models are useful for the stakeholders.

Measures to assess quality of postnatal care: A scoping review

High quality postnatal care is key for the health and wellbeing of women after childbirth and their newborns. In 2022, the World Health Organization (WHO) published global recommendations on maternal and newborn care for a positive postnatal care experience in a new WHO PNC guideline. Evidence regarding appropriate measures to monitor implementation of postnatal care (PNC) according to the WHO PNC guideline is lacking. This scoping review aims to document the measures used to assess the quality of postnatal care and their validity. The review was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR). Five electronic bibliographic databases were searched together with a grey literature search. Two reviewers independently screened and appraised identified articles. All data on PNC measures were extracted and mapped to the 2022 WHO PNC recommendations according to three categories: i) maternal care, ii) newborn care, iii) health system and health promotion interventions. We identified 62 studies providing measures aligning with the WHO PNC recommendations. For most PNC recommendations there were measures available and the highest number of recommendations were found for breastfeeding and the assessment of the newborn. No measures were found for recommendations related to sedentary behavior, criteria to be assessed before discharge, retention of staff in rural areas and use of digital communication. Measure validity assessment was described in 24 studies (39%), but methods were not standardized. Our review highlights a gap in existing PNC measures for several recommendations in the WHO PNC guideline. Assessment of the validity of PNC measures was limited. Consensus on how the quality of PNC should be measured is needed, involving a selection of priority measures and the development of new measures as appropriate.

Measuring the quality of antenatal care in a context of high utilisation: evidence from Telangana, India

BACKGROUND

Antenatal care coverage has dramatically increased in many low-and middle-income settings, including in the state of Telangana, India. However, there is increasing evidence of shortfalls in the quality of care women receive during their pregnancies. This study aims to examine dimensions of antenatal care quality in Telangana, India using four primary and secondary data sources.

METHODS

Data from two secondary statewide data sources (National Family Health Survey (NFHS-5), 2019-21; Health Management Information System (HMIS), 2019-20) and two primary data sources (a facility survey in 19 primary health centres and sub-centres in selected districts of Telangana; and observations of 36 antenatal care consultations at these facilities) were descriptively analysed.

RESULTS

NFHS-5 data showed about 73% of women in Telangana received all six assessed antenatal care components during pregnancy. HMIS data showed high coverage of antenatal care visits but differences in levels of screening, with high coverage of haemoglobin tests for anaemia but low coverage of testing for gestational diabetes and syphilis. The facility survey found missing equipment for several key antenatal care services. Antenatal care observations found blood pressure measurement and physical examinations had high coverage and were generally performed correctly. There were substantial deficiencies in symptom checking and communication between the woman and provider. Women were asked if they had any questions in 22% of consultations. Only one woman was asked about her mental health. Counselling of women on at least one of the ten items relating to birth preparedness and on at least one of six danger signs occurred in 58% and 36% of consultations, respectively.

CONCLUSION

Despite high coverage of antenatal care services and some essential maternal and foetal assessments, substantial quality gaps remained, particularly in communication between healthcare providers and pregnant women and in availability of key services. Progress towards achieving high quality in both content and experience of antenatal care requires addressing service gaps and developing better measures to capture and improve women's experiences of care.

Priority Indicators for Adolescent Health Measurement - Recommendations From the Global Action for Measurement of Adolescent Health (GAMA) Advisory Group

PURPOSE

This article describes the selection of priority indicators for adolescent (10-19 years) health measurement proposed by the Global Action for Measurement of Adolescent health advisory group and partners, building on previous work identifying 33 core measurement areas and mapping 413 indicators across these areas.

METHODS

The indicator selection process considered inputs from a broad range of stakeholders through a structured four-step approach: (1) definition of selection criteria and indicator scoring; (2) development of a draft list of indicators with metadata; (3) collection of public feedback through a survey; and (4) review of the feedback and finalization of the indicator list. As a part of the process, measurement gaps were also identified.

RESULTS

Fifty-two priority indicators were identified, including 36 core indicators considered to be most important for measuring the health of all adolescents, one alternative indicator for settings where measuring the core indicator is not feasible, and 15 additional indicators for settings where further detail on a topic would add value. Of these indicators, 17 (33%) measure health behaviors and risks, 16 (31%) health outcomes and conditions, eight (15%) health determinants, five (10%) systems performance and interventions, four (8%) policies, programmes, laws, and two (4%) subjective well-being.

DISCUSSION

A consensus list of priority indicators with metadata covering the most important health issues for adolescents was developed with structured inputs from a broad range of stakeholders. This list will now be pilot tested to assess the feasibility of indicator data collection to inform global, regional, national, and sub-national monitoring.

Multisite, mixed methods study to validate 10 maternal health system and policy indicators in Argentina, Ghana and India: a research protocol

INTRODUCTION

Most efforts to assess maternal health indicator validity focus on measures of service coverage. Fewer measures focus on the upstream enabling environment, and such measures are typically not research validated. Thus, methods for validating system and policy-level indicators are not well described. This protocol describes original multicountry research to be conducted in Argentina, Ghana and India, to validate 10 indicators from the monitoring framework for the 'Strategies toward Ending Preventable Maternal Mortality' (EPMM). The overall aim is to improve capacity to drive and track progress towards achieving the priority recommendations in the EPMM strategies. This work is expected to contribute new knowledge on validation methodology and reveal important information about the indicators under study and the phenomena they target for monitoring. Validating the indicators in three diverse settings will explore the external validity of results.

METHODS AND ANALYSIS

This observational study explores the validity of 10 indicators from the EPMM monitoring framework via seven discrete validation exercises that will use mixed methods: (1) cross-sectional review of policy data, (2) retrospective review of facility-level patient and administrative data and (3) collection of primary quantitative and qualitative cross-sectional data from health service providers and clients. There is a specific methodological approach and analytic plan for each indicator, directed by unique, relevant validation research questions.

ETHICS AND DISSEMINATION

The protocol was approved by the Office of Human Research Administration at Harvard University in November 2019. Individual study sites received approval via local institutional review boards by January 2020 except La Pampa, Argentina, approved June 2020. Our dissemination plan enables unrestricted access and reuse of all published research, including data sets. We expect to publish at least one peer-reviewed publication per validation exercise. We will disseminate results at conferences and engage local stakeholders in dissemination activities in each study country.

The prioritization and development of key social and structural indicators to address gaps in a framework for monitoring the Strategies toward Ending Preventable Maternal Mortality (EPMM): Results of an iterative expert technical consultation

Background

Since 2014, iterative technical work has captured stakeholder demand and channeled it toward improving maternal health measurement, to support SDG 3.1. Strategies toward Ending Preventable Maternal Mortality (EPMM) (2015) turned a broad lens on upstream systemic determinants of maternal health and survival highlighted in 11 Key Themes. A monitoring framework was developed to help countries track progress across these domains. This process yielded requests for additional indicators where stakeholders identified gaps for tracking EPMM Key Themes. In response, two technical consultations aimed at affirming the measurement gaps, specifying the constructs for measurement, and fully elaborating the metadata to allow them to be monitored.

Methods

Measures for development were prioritized based on multi-stakeholder dialogues in five countries, and data collected from government officials and UN partners in twenty countries on perceived need for proposed additional indicators. Sixty-one participants representing expertise in measure development and the topical areas covered took part across both consultations. Measures were developed through two simultaneous participatory online consultations stratified by focus area, comprising videos, discussion forums, polls, and live Zoom meetings.

Results

Eight candidate indicators relevant to priority recommendations in the EPMM Strategies are presented. Each includes a definition, numerator and denominator (if applicable), method of estimation, disaggregation factors, preferred data source(s), and expected periodicity. Four address gaps in measures of fundamental rights-related determinants of maternal health at national and subnational level, including women's reproductive autonomy; participative accountability for maternal health outcomes; and Respectful Maternity Care. Four strengthen the ability to count, track, and link births and maternal deaths and causes of death.

Conclusions

The proposed indicators correspond to specific EPMM Key Themes, filling gaps identified by multiple stakeholders, and respond to calls for a broadened approach to measurement and for indicators that track the social and health-systems determinants of maternal health. They reflect inputs and aspirations of numerous stakeholders, gathered over time and across various platforms. The iterative, discursive exploration of the concepts for measurement and the need for metrics to track them responds to recent calls for measure development to be carried out in more inclusive ways, and to be primarily concept- and user-driven.

Women's perceptions and self-reports of excessive bleeding during and after delivery: findings from a mixed-methods study in Northern Nigeria

OBJECTIVES

To explore lay perceptions of bleeding during and after delivery, and measure the frequency of self-reported indicators of bleeding.

SETTING

Yola, North-East Nigeria.

PARTICIPANTS

Women aged 15-49 years who delivered in the preceding 2 years of data collection period (2015-2016), and their family members who played key roles.

METHODS

Data on perceptions of bleeding were collected through 7 focus group discussions, 21 in-depth interviews and 10 family interviews. Sampling was purposive and data were analysed thematically. A household survey was then conducted with 640 women using cluster sampling on postpartum bleeding indicators developed from the qualitative data; data were analysed descriptively.

RESULTS

Perceptions of excessive bleeding fell under four themes: quantity of blood lost; rate/duration of blood flow; symptoms related to blood loss and receiving birth interventions/hearing comments from birth attendants. Young and less educated rural women had difficulty quantifying blood loss objectively, including when shown quantities using bottles. Respondents felt that acceptable blood loss levels depended on the individual woman and whether the blood is 'good' or 'diseased/bad.' Respondents believed that 'diseased' blood was a normal result of delivery and universally took steps to help it 'come out.' In the quantitative survey, indicators representing less blood loss were reported more frequently than those representing greater loss, for example, more women reported staining their clothes (33.6%) than the bed (18.1%) and the floor (6.2%). Overall, indicators related to quantity and rate of blood flow had higher frequencies compared with symptom and intervention-related/comment-related indicators.

CONCLUSION

Women quantify bleeding during and after delivery in varied ways and some women do not see bleeding as problematic. This suggests the need for standard messaging to address subjectivity. The range of indicators and varied frequencies highlight the challenges of measuring excessive bleeding from self-reports. More work is needed in improving and testing validity of questions.

A Scoping Review of Adolescent Health Indicators

PURPOSE

A host of recent initiatives relating to adolescent health have been accompanied by varying indicator recommendations, with little stakeholder coordination. We assessed currently included adolescent health-related indicators for their measurement focus, identified overlap across initiatives, and determined measurement gaps.

METHODS

We conducted a scoping review to map the existing indicator landscape as depicted by major measurement initiatives. We classified indicators as per 33 previously identified core adolescent health measurement areas across five domains and by age groups. We also identified indicators common across measurement initiatives even if differing in details.

RESULTS

We identified 413 indicators across 16 measurement initiatives, with most measuring health outcomes and conditions (162 [39%]) and health behaviors and risks (136 [33%]); followed by policies, programs, and laws (49 [12%]); health determinants (44 [11%]); and system performance and interventions (22 [5%]). Age specification was available for 221 (54%) indicators, with 51 (23%) focusing on the full adolescent age range (10-19 years), 1 (<1%) on 10-14 years, 27 (12%) on 15-19 years, and 142 (64%) on a broader age range including adolescents. No definitional information, such as numerator and denominator, was available for 138 indicators. We identified 236 distinct indicators after accounting for overlap.

CONCLUSION

The adolescent health measurement landscape is vast and includes substantial variation among indicators purportedly assessing the same concept. Gaps persist in measuring systems performance and interventions; policies, programs, and laws; and younger adolescents' health. Addressing these gaps and harmonizing measurement is fundamental to improve program implementation and accountability for adolescent health globally.

Women's progression through the maternal continuum of care in Guinea: Evidence from the 2018 Guinean Demographic and Health Survey

Objective

To examine women's progression through the antenatal, birth, and post‐partum maternal care in Guinea in 2018.

Methods

Using the Guinea Demographic and Health Survey of 2018, we analysed data on most recent live births in the 24 months preceding the survey among women aged 15–49 and the determinants (health system, quality of care, reproductive and sociodemographic factors) of women's progression through three steps of the continuum of care, using multivariable logistic regression.

Results

In the sample of 3,018 women, 87% reported at least one ANC visit (ANC1) with a health professional and 36% reported ANC4+, at least one of which was with a health professional. In the study, 26% of women reported ANC4+ plus birth in a health facility, and 20% reported ANC4+, birth in a health facility, plus post‐partum check‐up.

Predictors of woman's progression from ANC1 to ANC4+ visits included living in the administrative regions of Kindia (AOR: 1.96, 95% CI: 1.23–3.14) and Nzérékoré (AOR: 0.50, 95% CI: 0.32–0.79) vs. Kankan, being aged 15 to 17 (AOR: 0.55, 95% CI: 0.35–0.86) vs. aged 25 to 34, having primary or more education (AOR: 1.37, 95% CI: 1.09–1.72), and being from a middle (AOR: 1.52, 95% CI: 1.18–1.96) or wealthier (AOR: 2.38, 95% CI: 1.67–3.39) household vs. a poor household. Living in the administrative regions of Nzérékoré (AOR: 6.27, 95% CI: 1.57–25.05) vs. Kankan, in a middle (AOR: 1.64, 95% CI: 1.05–2.57) or wealthier (AOR: 3.23, 95% CI: 1.98–5.29) household vs. a poor household, nulliparity (AOR: 1.75, 95% CI: 1.03–2.97) vs. 2–4 previous births, the distance to health facility perceived as not being a problem (AOR: 1.75, 95% CI: 1.23–2.50), and higher ANC content score (AOR: 1.29, 95% CI: 1.10–1.52) remained independently associated with progression from ANC4+ to birth in a health facility. Predictors of progression from birth in the health facility to post‐partum check‐up included residing in the administrative regions of Labé (AOR: 0.22, 95% CI: 0.09–0.51) or Faranah (AOR: 0.43, 95% CI: 0.19–0.96) vs. Kankan, higher ANC content score (AOR: 1.76, 95% CI: 1.36–2.28), skin‐to‐skin contact after birth (AOR: 3.00, 95% CI: 1.70–5.31), and being attended at birth by a health professional (AOR: 17.52, 95% CI: 4.68–65.54).

Conclusions

Removing financial barriers and improving quality of care appear to be important to increase the percentage of women receiving the full maternal continuum of care.

Development and testing of a composite index to monitor the continuum of maternal health service delivery at provincial and district level in South Africa

INTRODUCTION

The continuum of care is a recommended framework for comprehensive health service delivery for maternal health, and it integrates health system and social determinants of health. There is a current lack of knowledge on a measurement approach to monitor performance on the framework. In this study we aim to develop and test a composite index for assessing the maternal health continuum in a province in South Africa with the possibility of nationwide use.

MATERIALS AND METHODS

The composite index was computed as a geometric mean of four dimensions of adequacy of the continuum of care. Data was sourced from the district health information system, household surveys and the census. The index formula was tested for robustness when alternative inputs for indicators and standardization methods were used. The index was used to assess performance in service delivery in the North West province of South Africa, as well as its four districts over a five-year period (2013-2017). The index was validated by assessing associations with maternal health and other outcomes. And factor analysis was used to assess the statistical dimensions of the index.

RESULTS

The provincial level index score increased from 62.3 in 2013 to 74 in 2017, showing general improvement in service delivery over time. The district level scores also improved over time, and our analysis identified areas for performance improvement. These include social determinants of health in some districts, and access and linkages to care in others. The provincial index was correlated with institutional maternal mortality rates (rs = -0.90, 90% CI = (-1.00, -0.25)) and the Human Development Index (r = 0.97, 95% CI = (0.63, 0.99). It was robust to alternative approaches including z-score standardization of indicators. Factor analysis showed three groupings of indicators for the health system and social determinants of health.

CONCLUSIONS

This study demonstrated the development and testing of a composite index to monitor and assess service delivery on the continuum of care for maternal health. The index was shown to be robust and valid, and identified potential areas for service improvement. A contextualised version can be tested in other settings within and outside of South Africa.

Translating international guidelines for use in routine maternal and neonatal healthcare quality measurement

BACKGROUND

Improving facility-based quality for maternal and neonatal care is the key to reducing morbidity and mortality rates in low- and middle-income countries. Recent guidance from WHO and others has produced a large number of indicators to choose from to track quality.

OBJECTIVE

To explore how to translate complex global maternal and neonatal health standards into actionable application at the facility level.

METHODS

We applied a two-step process as an example of how the 352 indicators in WHO's 2016 Standards for Improving Quality of Maternal and Newborn Care in Health Facilities might be reduced to only those with the strongest evidence base, associated with outcomes, and actionable by facility managers. We applied Hill criteria and assessed whether indicators were within the control of facility managers. We next conducted a rapid review of supporting literature and applied GRADE analysis, retaining those with scores of 'moderate' or 'high'. To understand the utility and barriers to measuring this limited set of indicators in practice, we undertook a case study of hypothetical measurement application in two districts in Bangladesh, interviewing 25 clinicians, managers, and other stakeholders.

RESULTS

From the initial 352 indicators, 56 were retained. The 56 indicators were used as a base for interviews. Respondents emphasized the practical challenges to the use of complex guides and the need for parsimonious and actionable sets of quality indicators.

CONCLUSIONS

This work offers one way to move towards a reduced quality indicator set, beginning from current WHO guidance. Despite study limitations, this work provides evidence of the need for reduced and evidence-based sets of quality indicators if guides are to be used to improve quality in practice. We hope that future research will build on and refine our efforts. Measuring quality effectively so that evidence guides and improves practice is the first step to assuring safe maternal and neonatal care.

Assessment of the validity of the measurement of newborn and maternal health-care coverage in hospitals (EN-BIRTH): an observational study

BACKGROUND

Progress in reducing maternal and neonatal deaths and stillbirths is impeded by data gaps, especially regarding coverage and quality of care in hospitals. We aimed to assess the validity of indicators of maternal and newborn health-care coverage around the time of birth in survey data and routine facility register data.

METHODS

Every Newborn-BIRTH Indicators Research Tracking in Hospitals was an observational study in five hospitals in Bangladesh, Nepal, and Tanzania. We included women and their newborn babies who consented on admission to hospital. Exclusion critiera at admission were no fetal heartbeat heard or imminent birth. For coverage of uterotonics to prevent post-partum haemorrhage, early initiation of breastfeeding (within 1 h), neonatal bag-mask ventilation, kangaroo mother care (KMC), and antibiotics for clinically defined neonatal infection (sepsis, pneumonia, or meningitis), we collected time-stamped, direct observation or case note verification data as gold standard. We compared data reported via hospital exit surveys and via hospital registers to the gold standard, pooled using random effects meta-analysis. We calculated population-level validity ratios (measured coverage to observed coverage) plus individual-level validity metrics.

FINDINGS

We observed 23 471 births and 840 mother-baby KMC pairs, and verified the case notes of 1015 admitted newborn babies regarding antibiotic treatment. Exit-survey-reported coverage for KMC was 99·9% (95% CI 98·3-100) compared with observed coverage of 100% (99·9-100), but exit surveys underestimated coverage for uterotonics (84·7% [79·1-89·5]) vs 99·4% [98·7-99·8] observed), bag-mask ventilation (0·8% [0·4-1·4]) vs 4·4% [1·9-8·1]), and antibiotics for neonatal infection (74·7% [55·3-90·1] vs 96·4% [94·0-98·6] observed). Early breastfeeding coverage was overestimated in exit surveys (53·2% [39·4-66·8) vs 10·9% [3·8-21·0] observed). "Don't know" responses concerning clinical interventions were more common in the exit survey after caesarean birth. Register data underestimated coverage of uterotonics (77·9% [37·8-99·5] vs 99·2% [98·6-99·7] observed), bag-mask ventilation (4·3% [2·1-7·3] vs 5·1% [2·0-9·6] observed), KMC (92·9% [84·2-98·5] vs 100% [99·9-100] observed), and overestimated early breastfeeding (85·9% (58·1-99·6) vs 12·5% [4·6-23·6] observed). Inter-hospital heterogeneity was higher for register-recorded coverage than for exit survey report. Even with the same register design, accuracy varied between hospitals.

INTERPRETATION

Coverage indicators for newborn and maternal health care in exit surveys had low accuracy for specific clinical interventions, except for self-report of KMC, which had high sensitivity after admission to a KMC ward or corner and could be considered for further assessment. Hospital register design and completion are less standardised than surveys, resulting in variable data quality, with good validity for the best performing sites. Because approximately 80% of births worldwide take place in facilities, standardising register design and information systems has the potential to sustainably improve the quality of data on care at birth.

FUNDING

Children's Investment Fund Foundation and Swedish Research Council.

Measuring experiences of facility-based care for pregnant women and newborns: a scoping review

BACKGROUND

Access to high-quality, person-centred care during pregnancy and childbirth is a global priority. Positive experience of care is key in particular, because it is both a fundamental right and can influence health outcomes and future healthcare utilisation. Despite its importance for accountability and action, systematic guidance on measuring experience of care is limited.

METHODS

We conducted a scoping review of published literature to identify measures/instruments for experience of facility-based pregnancy and childbirth (abortion, antenatal, intrapartum, postnatal and newborn) care. We systematically searched five bibliographic databases from 1 January 2007 through 1 February 2019. Using a predefined evidence template, we extracted data on study design, data collection method, study population and care type as reported in primary quantitative articles. We report results narratively.

RESULTS

We retrieved 16 528 unique citations, including 171 eligible articles representing, 157 unique instruments and 144 unique parent instruments across 56 countries. Half of the articles (90/171) did not use a validated instrument. While 82% (n=141) of articles reported on labour and childbirth care, only one reported on early pregnancy/abortion care. The most commonly reported sub-domains of user experience were communication (84%, 132/157) and respect and dignity (71%, 111/157). The primary purpose of most papers was measurement (70%, 119/171), largely through cross-sectional surveys.

CONCLUSION

There are alarming gaps in measurement of user experience for abortion, antenatal, postnatal and newborn care, including lack of validated instruments to measure the effects of interventions and policies on user experience.

PROTOCOL REGISTRATION DETAILS

This review was registered and published on PROSPERO (CRD42017070867). PROSPERO is an international database of prospectively registered systematic reviews in health and social care.

What is meant by validity in maternal and newborn health measurement? A conceptual framework for understanding indicator validation

BACKGROUND

Rigorous monitoring supports progress in achieving maternal and newborn mortality and morbidity reductions. Recent work to strengthen measurement for maternal and newborn health highlights the existence of a large number of indicators being used for this purpose. The definitions and data sources used to produce indicator estimates vary and challenges exist with completeness, accuracy, transparency, and timeliness of data. The objective of this study is to create a conceptual overview of how indicator validity is defined and understood by those who develop and use maternal and newborn health indicators.

METHODS

A conceptual framework of validity was developed using mixed methods. We were guided by principles for conceptual frameworks and by a review of the literature and key maternal and newborn health indicator guidance documents. We also conducted qualitative semi-structured interviews with 32 key informants chosen through purposive sampling.

RESULTS

We categorised indicator validity into three main types: criterion, convergent, and construct. Criterion or diagnostic validity, comparing a measure with a gold standard, has predominantly been used to assess indicators of care coverage and content. Studies assessing convergent validity quantify the extent to which two or more indicator measurement approaches, none of which is a gold-standard, relate. Key informants considered construct validity, or the accuracy of the operationalisation of a concept or phenomenon, a critical part of the overall assessment of indicator validity.

CONCLUSION

Given concerns about the large number of maternal and newborn health indicators currently in use, a more consistent understanding of validity can help guide prioritization of key indicators and inform development of new indicators. All three types of validity are relevant for evaluating the performance of maternal and newborn health indicators. We highlight the need to establish a common language and understanding of indicator validity among the various global and local stakeholders working within maternal and newborn health.

Review of health and non-health sector indicators for monitoring service provision along the continuum of care for maternal health

Objective

This study uses health and non-health sector data sources to select and assess available indicators for service provision along the continuum of care for maternal health at subnational levels in South Africa. It applies the adequacy approach established in another study to assess the multi-dimensionality of available indicators. Using adequacy and the process of assessment in the study, the comprehensiveness of the continuum of care for improving maternal health outcomes can be assessed.

Results

We found 27 indicators of care utilization and access, linkages of care, and quality of care from the routine district health information system. The General Household Survey contained 11 indicators for the social determinants of health on the continuum of care framework. Indicator gaps include health promotion during and after pregnancy, maternal nutrition, empowerment and quality of care. At present, the available indicators measure about 74% of the interventions on the continuum of care framework. We make recommendations regarding improvements needed to better measure and monitor the continuum of care for maternal health. These involve actions within the health system and include integration of non-health system indicators.

Improving the validity, relevance and feasibility of the continuum of care framework for maternal health in South Africa: a thematic analysis of experts' perspectives

BACKGROUND

The continuum of care is a key strategy for ensuring comprehensive service delivery for maternal health, while acknowledging the role of the social determinants of health. However, there is little research on the operationalisation of the framework by decision-makers and implementers to address maternal health challenges. The framework should be measurable and feasible for implementation in low- and middle-income country contexts. In this study, we explore experts' perspective on monitoring indicators for continuum of care and key issues related to their use in the South African context.

METHODS

We conducted key informant interviews with a range of experts in decision-making and programme implementation roles in the health system and relevant sectors. Key informants provided their perspectives on systematically selected, nationally representative monitoring indicators in terms of validity, relevance and feasibility. We interviewed 13 key informants and conducted a thematic analysis of their responses using multi-stage coding techniques in Atlas.ti 8.4.

RESULTS

Experts believed that the continuum of care framework and monitoring indicators offer a multisectoral perspective for maternal health intervention missing in current programmes. To improve validity of monitoring indicators, experts suggested reflection on the use of proxy indicators and improvement of data to allow for equity analysis. In terms of relevance and feasibility, experts believe there was potential to foster co-accountability using continuum of care indicators. However, as experts stated, new indicators should be integrated that directly measure intersectoral collaboration for maternal health. In addition, experts recommended that the framework and indicators should evolve over time to reflect evolving policy priorities and public health challenges.

CONCLUSION

Experts, as decision-makers and implementers, helped identify key issues in the application of the continuum of care framework and its indicators. The use of local indicators can bring the continuum of care framework from an under-utilised strategy to a useful tool for action and decision-making in maternal health. Our findings point to measurement issues and systematic changes needed to improve comprehensive monitoring of maternal health interventions in South Africa. Our methods can be applied to other low- and middle-income countries using the continuum of care framework and locally available indicators.