External Validation Study of First Trimester Obstetric Prediction Models (Expect Study I): Research Protocol and Population Characteristics

Development and validation of a prognostic model to predict birth weight: individual participant data meta-analysis

Objective

To predict birth weight at various potential gestational ages of delivery based on data routinely available at the first antenatal visit.

Design

Individual participant data meta-analysis.

Data sources

Individual participant data of four cohorts (237 228 pregnancies) from the International Prediction of Pregnancy Complications (IPPIC) network dataset.

Eligibility criteria for selecting studies

Studies in the IPPIC network were identified by searching major databases for studies reporting risk factors for adverse pregnancy outcomes, such as pre-eclampsia, fetal growth restriction, and stillbirth, from database inception to August 2019. Data of four IPPIC cohorts (237 228 pregnancies) from the US (National Institute of Child Health and Human Development, 2018; 233 483 pregnancies), UK (Allen et al, 2017; 1045 pregnancies), Norway (STORK Groruddalen research programme, 2010; 823 pregnancies), and Australia (Rumbold et al, 2006; 1877 pregnancies) were included in the development of the model.

Results

The IPPIC birth weight model was developed with random intercept regression models with backward elimination for variable selection. Internal-external cross validation was performed to assess the study specific and pooled performance of the model, reported as calibration slope, calibration-in-the-large, and observed versus expected average birth weight ratio. Meta-analysis showed that the apparent performance of the model had good calibration (calibration slope 0.99, 95% confidence interval (CI) 0.88 to 1.10; calibration-in-the-large 44.5 g, -18.4 to 107.3) with an observed versus expected average birth weight ratio of 1.02 (95% CI 0.97 to 1.07). The proportion of variation in birth weight explained by the model (R2) was 46.9% (range 32.7-56.1% in each cohort). On internal-external cross validation, the model showed good calibration and predictive performance when validated in three cohorts with a calibration slope of 0.90 (Allen cohort), 1.04 (STORK Groruddalen cohort), and 1.07 (Rumbold cohort), calibration-in-the-large of -22.3 g (Allen cohort), -33.42 (Rumbold cohort), and 86.4 g (STORK Groruddalen cohort), and observed versus expected ratio of 0.99 (Rumbold cohort), 1.00 (Allen cohort), and 1.03 (STORK Groruddalen cohort); respective pooled estimates were 1.00 (95% CI 0.78 to 1.23; calibration slope), 9.7 g (-154.3 to 173.8; calibration-in-the-large), and 1.00 (0.94 to 1.07; observed v expected ratio). The model predictions were more accurate (smaller mean square error) in the lower end of predicted birth weight, which is important in informing clinical decision making.

Conclusions

The IPPIC birth weight model allowed birth weight predictions for a range of possible gestational ages. The model explained about 50% of individual variation in birth weights, was well calibrated (especially in babies at high risk of fetal growth restriction and its complications), and showed promising performance in four different populations included in the individual participant data meta-analysis. Further research to examine the generalisability of performance in other countries, settings, and subgroups is required.

Trial registration

PROSPERO CRD42019135045.

Development and validation of prediction models for fetal growth restriction and birthweight: an individual participant data meta-analysis

Background

Fetal growth restriction is associated with perinatal morbidity and mortality. Early identification of women having at-risk fetuses can reduce perinatal adverse outcomes.

Objectives

To assess the predictive performance of existing models predicting fetal growth restriction and birthweight, and if needed, to develop and validate new multivariable models using individual participant data.

Design

Individual participant data meta-analyses of cohorts in International Prediction of Pregnancy Complications network, decision curve analysis and health economics analysis.

Participants

Pregnant women at booking. External validation of existing models (9 cohorts, 441,415 pregnancies); International Prediction of Pregnancy Complications model development and validation (4 cohorts, 237,228 pregnancies).

Predictors

Maternal clinical characteristics, biochemical and ultrasound markers.

Primary outcomes

fetal growth restriction defined as birthweight <10th centile adjusted for gestational age and with stillbirth, neonatal death or delivery before 32 weeks' gestation birthweight.

Analysis

First, we externally validated existing models using individual participant data meta-analysis. If needed, we developed and validated new International Prediction of Pregnancy Complications models using random-intercept regression models with backward elimination for variable selection and undertook internal-external cross-validation. We estimated the study-specific performance (c-statistic, calibration slope, calibration-in-the-large) for each model and pooled using random-effects meta-analysis. Heterogeneity was quantified using τ2 and 95% prediction intervals. We assessed the clinical utility of the fetal growth restriction model using decision curve analysis, and health economics analysis based on National Institute for Health and Care Excellence 2008 model.

Results

Of the 119 published models, one birthweight model (Poon) could be validated. None reported fetal growth restriction using our definition. Across all cohorts, the Poon model had good summary calibration slope of 0.93 (95% confidence interval 0.90 to 0.96) with slight overfitting, and underpredicted birthweight by 90.4 g on average (95% confidence interval 37.9 g to 142.9 g). The newly developed International Prediction of Pregnancy Complications-fetal growth restriction model included maternal age, height, parity, smoking status, ethnicity, and any history of hypertension, pre-eclampsia, previous stillbirth or small for gestational age baby and gestational age at delivery. This allowed predictions conditional on a range of assumed gestational ages at delivery. The pooled apparent c-statistic and calibration were 0.96 (95% confidence interval 0.51 to 1.0), and 0.95 (95% confidence interval 0.67 to 1.23), respectively. The model showed positive net benefit for predicted probability thresholds between 1% and 90%. In addition to the predictors in the International Prediction of Pregnancy Complications-fetal growth restriction model, the International Prediction of Pregnancy Complications-birthweight model included maternal weight, history of diabetes and mode of conception. Average calibration slope across cohorts in the internal-external cross-validation was 1.00 (95% confidence interval 0.78 to 1.23) with no evidence of overfitting. Birthweight was underestimated by 9.7 g on average (95% confidence interval -154.3 g to 173.8 g).

Limitations

We could not externally validate most of the published models due to variations in the definitions of outcomes. Internal-external cross-validation of our International Prediction of Pregnancy Complications-fetal growth restriction model was limited by the paucity of events in the included cohorts. The economic evaluation using the published National Institute for Health and Care Excellence 2008 model may not reflect current practice, and full economic evaluation was not possible due to paucity of data.

Future work

International Prediction of Pregnancy Complications models' performance needs to be assessed in routine practice, and their impact on decision-making and clinical outcomes needs evaluation.

Conclusion

The International Prediction of Pregnancy Complications-fetal growth restriction and International Prediction of Pregnancy Complications-birthweight models accurately predict fetal growth restriction and birthweight for various assumed gestational ages at delivery. These can be used to stratify the risk status at booking, plan monitoring and management.

Study registration

This study is registered as PROSPERO CRD42019135045.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 17/148/07) and is published in full in Health Technology Assessment; Vol. 28, No. 14. See the NIHR Funding and Awards website for further award information.

Counseling pregnant women on calcium: effects on calcium intake

OBJECTIVES

To evaluate the effect of incorporating calcium advice into early pregnancy counseling on calcium intake during pregnancy in the Netherlands.

METHODS

A multicenter prospective before-after cohort study was conducted introducing risk-based care including calculating individual pre-eclampsia risk. Part of the intervention was to incorporate calcium advice into routine counseling. We calculated individual daily calcium intake and adequacy of calcium intake (≥1,000 mg/day) at 16, 24 and 34 weeks of pregnancy. We performed a multiple logistic regression adjusting for covariates to identify any differences in the risk of inadequate calcium intake between RC and CAC.

RESULTS

In regular care (RC, 2013-2015, n=2,477) 60% had inadequate calcium intake, compared to 49% during calcium advice care (CAC, 2017-2018, n=774) (aOR 0.75, 95% CI 0.64-0.88). Specific calcium supplements were used by 2% and 29% in RC and CAC, respectively (OR 25.1, 95% CI 17.8-36.0). Determinants of an inadequate calcium intake were lower age (aOR per additional year 0.96, 95% CI: 0.94-0.98), nulliparity (aOR 1.22, 95% CI: 1.03-1.45) and non-Caucasian origin (aOR 1.83, 95% CI 1.09-3.09). In CAC, risk of inadequate intake decreased with increasing predicted pre-eclampsia risk, which was a trend reversal compared to RC.

CONCLUSIONS

Incorporating calcium advice into early pregnancy counseling was shown to lead to a decrease in the risk of inadequate calcium intake during pregnancy, but still inadequate intake in half of the women suggesting the need for further study on improving implementation. Awareness of individual increased PE risk had positive effect on calcium intake.

Effects of race and ethnicity on perinatal outcomes in high-income and upper-middle-income countries: an individual participant data meta-analysis of 2 198 655 pregnancies

BACKGROUND

Existing evidence on the effects of race and ethnicity on pregnancy outcomes is restricted to individual studies done within specific countries and health systems. We aimed to assess the impact of race and ethnicity on perinatal outcomes in high-income and upper-middle-income countries, and to ascertain whether the magnitude of disparities, if any, varied across geographical regions.

METHODS

For this individual participant data (IPD) meta-analysis we used data from the International Prediction of Pregnancy Complications (IPPIC) Network of studies on pregnancy complications; the full dataset comprised 94 studies, 53 countries, and 4 539 640 pregnancies. We included studies that reported perinatal outcomes (neonatal death, stillbirth, preterm birth, and small-for-gestational-age babies) in at least two racial or ethnic groups (White, Black, south Asian, Hispanic, or other). For our two-step random-effects IPD meta-analysis, we did multiple imputations for confounder variables (maternal age, BMI, parity, and level of maternal education) selected with a directed acyclic graph. The primary outcomes were neonatal mortality and stillbirth. Secondary outcomes were preterm birth and a small-for-gestational-age baby. We estimated the association of race and ethnicity with perinatal outcomes using a multivariate logistic regression model and reported this association with odds ratios (ORs) and 95% CIs. We also did a subgroup analysis of studies by geographical region.

FINDINGS

51 studies from 20 high-income and upper-middle-income countries, comprising 2 198 655 pregnancies, were eligible for inclusion in this IPD meta-analysis. Neonatal death was twice as likely in babies born to Black women than in babies born to White women (OR 2·00, 95% CI 1·44-2·78), as was stillbirth (2·16, 1·46-3·19), and babies born to Black women were at increased risk of preterm birth (1·65, 1·46-1·88) and being small for gestational age (1·39, 1·13-1·72). Babies of women categorised as Hispanic had a three-times increased risk of neonatal death (OR 3·34, 95% CI 2·77-4·02) than did those born to White women, and those born to south Asian women were at increased risk of preterm birth (OR 1·26, 95% CI 1·07-1·48) and being small for gestational age (1·61, 1·32-1·95). The effects of race and ethnicity on preterm birth and small-for-gestational-age babies did not vary across regions.

INTERPRETATION

Globally, among underserved groups, babies born to Black women had consistently poorer perinatal outcomes than White women after adjusting for maternal characteristics, although the risks varied for other groups. The effects of race and ethnicity on adverse perinatal outcomes did not vary by region.

FUNDING

National Institute for Health and Care Research, Wellbeing of Women.

Perinatal factors related to pregnancy and childbirth satisfaction: a prospective cohort study

BACKGROUND

Satisfaction of pregnancy and childbirth is an important quality measure of maternity care. Satisfaction questionnaires generally result in high scores. However, it has been argued that dissatisfaction relies on a different construct. In response to a worldwide call for obstetric care that is more woman-centered, we identified and described the contributors to suboptimal satisfaction with pregnancy and childbirth.

METHODS

A prospective subcohort of 739 women from a larger cohort (Expect Study I, n = 2614) received a pregnancy and childbirth satisfaction questionnaire. Scores were transformed to a binary outcome whereby a score <100 points corresponded with less satisfied women. We performed a multiple logistic regression analysis to define independent perinatal factors related to suboptimal satisfaction.

RESULTS

Decreased perceived personal well-being, antenatal anxiety, and obstetrician-led care during labor were all independently associated with suboptimal pregnancy and childbirth satisfaction. No difference in satisfaction was found between antenatal care led by a midwife or an obstetrician, but midwife-led antenatal care reduced the odds of suboptimal satisfaction compared to women who were transferred to an obstetrician in the antenatal period. Antenatal anxiety was experienced by 25% of all women and is associated with decreased satisfaction scores.

DISCUSSION

Screening and treatment of women suffering from anxiety might improve pregnancy and childbirth satisfaction, but further research is necessary. Women's birthing experience may improve by reducing unnecessary secondary obstetric care.

Impact on perinatal health and cost-effectiveness of risk-based care in obstetrics: a before-after study

BACKGROUND

Obstetric health care relies on an adequate antepartum risk selection. Most guidelines used for risk stratification, however, do not assess absolute risks. In 2017, a prediction tool was implemented in a Dutch region. This tool combines first trimester prediction models with obstetric care paths tailored to the individual risk profile, enabling risk-based care.

OBJECTIVE

To assess impact and cost-effectiveness of risk-based care compared to care-as-usual in a general population.

METHODS

A before-after study was conducted using 2 multicenter prospective cohorts. The first cohort (2013-2015) received care-as-usual; the second cohort (2017-2018) received risk-based care. Health outcomes were (1) a composite of adverse perinatal outcomes and (2) maternal quality-adjusted life-years. Costs were estimated using a health care perspective from conception to 6 weeks after the due date. Mean costs per woman, cost differences between the 2 groups, and incremental cost effectiveness ratios were calculated. Sensitivity analyses were performed to evaluate the robustness of the findings.

RESULTS

In total 3425 women were included. In nulliparous women there was a significant reduction of perinatal adverse outcomes among the risk-based care group (adjusted odds ratio, 0.56; 95% confidence interval, 0.32-0.94), but not in multiparous women. Mean costs per pregnant woman were significantly lower for risk-based care (mean difference, -€2766; 95% confidence interval, -€3700 to -€1825). No differences in maternal quality of life, adjusted for baseline health, were observed.

CONCLUSION

In the Netherlands, risk-based care in nulliparous women was associated with improved perinatal outcomes as compared to care-as-usual. Furthermore, risk-based care was cost-effective compared to care-as-usual and resulted in lower health care costs.

External validation and clinical utility of prognostic prediction models for gestational diabetes mellitus: A prospective cohort study

Introduction

We performed an independent validation study of all published first trimester prediction models, containing non‐invasive predictors, for the risk of gestational diabetes mellitus. Furthermore, the clinical potential of the best performing models was evaluated.

Material and methods

Systemically selected prediction models from the literature were validated in a Dutch prospective cohort using data from Expect Study I and PRIDE Study. The predictive performance of the models was evaluated by discrimination and calibration. Clinical utility was assessed using decision curve analysis. Screening performance measures were calculated at different risk thresholds for the best model and compared with current selective screening strategies.

Results

The validation cohort included 5260 women. Gestational diabetes mellitus was diagnosed in 127 women (2.4%). The discriminative performance of the 12 included models ranged from 68% to 75%. Nearly all models overestimated the risk. After recalibration, agreement between the observed outcomes and predicted probabilities improved for most models.

Conclusions

The best performing prediction models showed acceptable performance measures and may enable more personalized medicine‐based antenatal care for women at risk of developing gestational diabetes mellitus compared with current applied strategies.

Low-dose-aspirin usage among women with an increased preeclampsia risk: A prospective cohort study

INTRODUCTION

Low-dose aspirin (LDA) prophylaxis has been shown to reduce women's preeclampsia risk. Evidence regarding LDA adherence rates of pregnant women is based almost exclusively on clinical trials, giving a potentially biased picture. Moreover, these studies do not report on determinants of adherence. Since 2017, obstetric healthcare professionals in a Dutch region have assessed women's preeclampsia risk by means of a prediction tool and counseled those with an above-population average risk on LDA as a prophylactic measure.

MATERIAL AND METHODS

From 2017 to 2018, 865 women were recruited in multiple centers and prospectively followed using web-based surveys (Expect Study II). Rates and determinants of LDA usage among women with an increased preeclampsia risk in daily practice were assessed. Results were compared with findings in a similar cohort from a care-as-usual setting lacking risk-based counseling (Expect Study I, n = 2614). Netherlands Trial Register NTR4143.

RESULTS

In total, 306 women had a predicted increased preeclampsia risk. LDA usage was higher for women receiving risk-based care than care-as-usual (29.4% vs 1.5%, odds ratio 19.1, 95% confidence interval 11.2-32.5). Daily LDA usage was positively correlated with both predicted risk and women's concerns regarding preeclampsia. Most reported reasons for non- or incomplete use were unawareness of LDA as a preventive intervention, concerns about potential adverse effects and doubts regarding the benefits.

CONCLUSIONS

Risk-based counseling was associated with a higher prevalence of LDA usage, but general usage rates were low. Future research regarding potential factors improving the usage of LDA during pregnancy is necessary.

External validation and clinical usefulness of first-trimester prediction models for small- and large-for-gestational-age infants: a prospective cohort study

Objective

To assess the external validity of all published first‐trimester prediction models based on routinely collected maternal predictors for the risk of small‐ and large‐for‐gestational‐age (SGA and LGA) infants. Furthermore, the clinical potential of the best‐performing models was evaluated.

Design

Multicentre prospective cohort.

Setting

Thirty‐six midwifery practices and six hospitals (in the Netherlands).

Population

Pregnant women were recruited at <16 weeks of gestation between 1 July 2013 and 31 December 2015.

Methods

Prediction models were systematically selected from the literature. Information on predictors was obtained by a web‐based questionnaire. Birthweight centiles were corrected for gestational age, parity, fetal sex, and ethnicity.

Main outcome measures

Predictive performance was assessed by means of discrimination (C‐statistic) and calibration.

Results

The validation cohort consisted of 2582 pregnant women. The outcomes of SGA <10th percentile and LGA >90th percentile occurred in 203 and 224 women, respectively. The C‐statistics of the included models ranged from 0.52 to 0.64 for SGA (n = 6), and from 0.60 to 0.69 for LGA (n = 6). All models yielded higher C‐statistics for more severe cases of SGA (<5th percentile) and LGA (>95th percentile). Initial calibration showed poor‐to‐moderate agreement between the predicted probabilities and the observed outcomes, but this improved substantially after recalibration.

Conclusion

The clinical relevance of the models is limited because of their moderate predictive performance, and because the definitions of SGA and LGA do not exclude constitutionally small or large infants. As most clinically relevant fetal growth deviations are related to ‘vascular’ or ‘metabolic’ factors, models predicting hypertensive disorders and gestational diabetes are likely to be more specific.

Tweetable abstract

The clinical relevance of prediction models for the risk of small‐ and large‐for‐gestational‐age is limited.

The clinical relevance of prediction models for the risk of small‐ and large‐for‐gestational‐age is limited.

External Validation and Clinical Usefulness of First Trimester Prediction Models for the Risk of Preeclampsia: A Prospective Cohort Study

INTRODUCTION

This study assessed the external validity of all published first trimester prediction models for the risk of preeclampsia (PE) based on routinely collected maternal predictors. Moreover, the potential utility of the best-performing models in clinical practice was evaluated.

MATERIAL AND METHODS

Ten prediction models were systematically selected from the literature. We performed a multicenter prospective cohort study in the Netherlands between July 1, 2013, and December 31, 2015. Eligible pregnant women completed a web-based questionnaire before 16 weeks' gestation. The outcome PE was established using postpartum questionnaires and medical records. Predictive performance of each model was assessed by means of discrimination (c-statistic) and a calibration plot. Clinical usefulness was evaluated by means of decision curve analysis and by calculating the potential impact at different risk thresholds.

RESULTS

The validation cohort contained 2,614 women of whom 76 developed PE (2.9%). Five models showed moderate discriminative performance with c-statistics ranging from 0.73 to 0.77. Adequate calibration was obtained after refitting. The best models were clinically useful over a small range of predicted probabilities.

DISCUSSION

Five of the ten included first trimester prediction models for PE showed moderate predictive performance. The best models may provide more benefit compared to risk selection as used in current guidelines.

Prediction models for the risk of spontaneous preterm birth based on maternal characteristics: a systematic review and independent external validation

Introduction

Prediction models may contribute to personalized risk‐based management of women at high risk of spontaneous preterm delivery. Although prediction models are published frequently, often with promising results, external validation generally is lacking. We performed a systematic review of prediction models for the risk of spontaneous preterm birth based on routine clinical parameters. Additionally, we externally validated and evaluated the clinical potential of the models.

Material and methods

Prediction models based on routinely collected maternal parameters obtainable during first 16 weeks of gestation were eligible for selection. Risk of bias was assessed according to the CHARMS guidelines. We validated the selected models in a Dutch multicenter prospective cohort study comprising 2614 unselected pregnant women. Information on predictors was obtained by a web‐based questionnaire. Predictive performance of the models was quantified by the area under the receiver operating characteristic curve (AUC) and calibration plots for the outcomes spontaneous preterm birth <37 weeks and <34 weeks of gestation. Clinical value was evaluated by means of decision curve analysis and calculating classification accuracy for different risk thresholds.

Results

Four studies describing five prediction models fulfilled the eligibility criteria. Risk of bias assessment revealed a moderate to high risk of bias in three studies. The AUC of the models ranged from 0.54 to 0.67 and from 0.56 to 0.70 for the outcomes spontaneous preterm birth <37 weeks and <34 weeks of gestation, respectively. A subanalysis showed that the models discriminated poorly (AUC 0.51–0.56) for nulliparous women. Although we recalibrated the models, two models retained evidence of overfitting. The decision curve analysis showed low clinical benefit for the best performing models.

Conclusions

This review revealed several reporting and methodological shortcomings of published prediction models for spontaneous preterm birth. Our external validation study indicated that none of the models had the ability to predict spontaneous preterm birth adequately in our population. Further improvement of prediction models, using recent knowledge about both model development and potential risk factors, is necessary to provide an added value in personalized risk assessment of spontaneous preterm birth.

Implementation and Effects of Risk-Dependent Obstetric Care in the Netherlands (Expect Study II): Protocol for an Impact Study

BACKGROUND

Recently, validated risk models predicting adverse obstetric outcomes combined with risk-dependent care paths have been made available for early antenatal care in the southeastern part of the Netherlands. This study will evaluate implementation progress and impact of the new approach in obstetric care.

OBJECTIVE

The objective of this paper is to describe the design of a study evaluating the impact of implementing risk-dependent care. Validated first-trimester prediction models are embedded in daily clinical practice and combined with risk-dependent obstetric care paths.

METHODS

A multicenter prospective cohort study consisting of women who receive risk-dependent care is being performed from April 2017 to April 2018 (Expect Study II). Obstetric risk profiles will be calculated using a Web-based tool, the Expect prediction tool. The primary outcomes are the adherence of health care professionals and compliance of women. Secondary outcomes are patient satisfaction and cost-effectiveness. Outcome measures will be established using Web-based questionnaires. The secondary outcomes of the risk-dependent care cohort (Expect II) will be compared with the outcomes of a similar prospective cohort (Expect I). Women of this similar cohort received former care-as-usual and were prospectively included between July 1, 2013 and December 31, 2015 (Expect I).

RESULTS

Currently, women are being recruited for the Expect Study II, and a total of 300 women are enrolled.

CONCLUSIONS

This study will provide information about the implementation and impact of a new approach in obstetric care using prediction models and risk-dependent obstetric care paths.

TRIAL REGISTRATION

Netherlands Trial Register NTR4143; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=4143 (Archived by WebCite at http://www.webcitation.org/6t8ijtpd9).