External validation of prognostic models to predict stillbirth using International Prediction of Pregnancy Complications (IPPIC) Network database: individual participant data meta-analysis

Validating the ratio of insulin like growth factor binding protein 4 to sex hormone binding globulin as a prognostic predictor of preterm birth in Viet Nam: a case-cohort study

OBJECTIVE

To validate a serum biomarker developed in the USA for preterm birth (PTB) risk stratification in Viet Nam.

METHODS

Women with singleton pregnancies (n = 5000) were recruited between 19+0-23+6 weeks' gestation at Tu Du Hospital, Ho Chi Minh City. Maternal serum was collected from 19+0-22+6 weeks' gestation and participants followed to neonatal discharge. Relative insulin-like growth factor binding protein 4 (IGFBP4) and sex hormone binding globulin (SHBG) abundances were measured by mass spectrometry and their ratio compared between PTB cases and term controls. Discrimination (area under the receiver operating characteristic curve, AUC) and calibration for PTB <37 and <34 weeks' gestation were tested, with model tuning using clinical factors. Measured outcomes included all PTBs (any birth ≤37 weeks' gestation) and spontaneous PTBs (birth ≤37 weeks' gestation with clinical signs of initiation of parturition).

RESULTS

Complete data were available for 4984 (99.7%) individuals. The cohort PTB rate was 6.7% (n = 335). We observed an inverse association between the IGFBP4/SHBG ratio and gestational age at birth (p = 0.017; AUC 0.60 [95% CI, 0.53-0.68]). Including previous PTB (for multiparous women) or prior miscarriage (for primiparous women) improved performance (AUC 0.65 and 0.70, respectively, for PTB <37 and <34 weeks' gestation). Optimal performance (AUC 0.74) was seen within 19-20 weeks' gestation, for BMI >21 kg/m2 and age 20-35 years.

CONCLUSION

We have validated a novel serum biomarker for PTB risk stratification in a very different setting to the original study. Further research is required to determine appropriate ratio thresholds based on the prevalence of risk factors and the availability of resources and preventative therapies.

Prioritisation of early pregnancy risk factors for stillbirth: An international multistakeholder modified e-Delphi consensus study

OBJECTIVE

To identify and prioritise early pregnancy risk factors for stillbirth to inform prognostic factor and model research.

STUDY DESIGN

We used a modified e-Delphi method and consultation meeting to achieve consensus. Risk factors for early, late and stillbirth at any gestation identified from an umbrella review of risk factors for stillbirth were entered into a two-stage online Delphi survey with an international group of stakeholders made up of healthcare professionals and researchers. The RAND/ University of California at Los Angeles appropriateness method was used to evaluate consensus. Responders voted on a scale of 1-9 for each risk factor in terms of importance for early, late, and stillbirth at any gestation. Consensus for inclusion was reached if the median score was in the top tertile and at least two thirds of panellists had scored the risk factor within the top tertile.

RESULTS

Twenty-six risk factors were identified from an umbrella review and presented to stakeholders in round 1 of our e-Delphi survey. Round 1 was completed by 68 stakeholders, 79% (54/68) of whom went on to complete the second round. Seventeen risk factors were discussed at the consensus meeting. From the twenty-six risk factors identified, fifteen of these were prioritised for stillbirth at any gestation, eleven for early stillbirth, and sixteen for late stillbirth, across three domains of maternal characteristics, ultrasound markers and biochemical markers. The prioritised maternal characteristics common to early, late, and stillbirth at any gestation were: maternal age, smoking, drug misuse, history of heritable thrombophilia, hypertension, renal disease, diabetes, previous stillbirth and multiple pregnancy. Maternal BMI, access to healthcare, and socioeconomic status were prioritised for late stillbirth and stillbirth at any gestation. Previous pre-eclampsia and previous small for gestational age baby were prioritised for late stillbirth. Of the ultrasound markers, uterine artery Doppler pulsatility index and congenital fetal anomaly were prioritised for all. One biochemical marker, placental growth factor, was prioritised for stillbirth at any gestation.

CONCLUSIONS

Our prioritised risk factors for stillbirth can inform formal factor-outcome evaluation of early pregnancy risk factors to influence public health strategies on prevention of such risk factors to prevent stillbirth.

Development and validation of a prognostic model to predict birth weight: individual participant data meta-analysis

Objective

To predict birth weight at various potential gestational ages of delivery based on data routinely available at the first antenatal visit.

Design

Individual participant data meta-analysis.

Data sources

Individual participant data of four cohorts (237 228 pregnancies) from the International Prediction of Pregnancy Complications (IPPIC) network dataset.

Eligibility criteria for selecting studies

Studies in the IPPIC network were identified by searching major databases for studies reporting risk factors for adverse pregnancy outcomes, such as pre-eclampsia, fetal growth restriction, and stillbirth, from database inception to August 2019. Data of four IPPIC cohorts (237 228 pregnancies) from the US (National Institute of Child Health and Human Development, 2018; 233 483 pregnancies), UK (Allen et al, 2017; 1045 pregnancies), Norway (STORK Groruddalen research programme, 2010; 823 pregnancies), and Australia (Rumbold et al, 2006; 1877 pregnancies) were included in the development of the model.

Results

The IPPIC birth weight model was developed with random intercept regression models with backward elimination for variable selection. Internal-external cross validation was performed to assess the study specific and pooled performance of the model, reported as calibration slope, calibration-in-the-large, and observed versus expected average birth weight ratio. Meta-analysis showed that the apparent performance of the model had good calibration (calibration slope 0.99, 95% confidence interval (CI) 0.88 to 1.10; calibration-in-the-large 44.5 g, -18.4 to 107.3) with an observed versus expected average birth weight ratio of 1.02 (95% CI 0.97 to 1.07). The proportion of variation in birth weight explained by the model (R2) was 46.9% (range 32.7-56.1% in each cohort). On internal-external cross validation, the model showed good calibration and predictive performance when validated in three cohorts with a calibration slope of 0.90 (Allen cohort), 1.04 (STORK Groruddalen cohort), and 1.07 (Rumbold cohort), calibration-in-the-large of -22.3 g (Allen cohort), -33.42 (Rumbold cohort), and 86.4 g (STORK Groruddalen cohort), and observed versus expected ratio of 0.99 (Rumbold cohort), 1.00 (Allen cohort), and 1.03 (STORK Groruddalen cohort); respective pooled estimates were 1.00 (95% CI 0.78 to 1.23; calibration slope), 9.7 g (-154.3 to 173.8; calibration-in-the-large), and 1.00 (0.94 to 1.07; observed v expected ratio). The model predictions were more accurate (smaller mean square error) in the lower end of predicted birth weight, which is important in informing clinical decision making.

Conclusions

The IPPIC birth weight model allowed birth weight predictions for a range of possible gestational ages. The model explained about 50% of individual variation in birth weights, was well calibrated (especially in babies at high risk of fetal growth restriction and its complications), and showed promising performance in four different populations included in the individual participant data meta-analysis. Further research to examine the generalisability of performance in other countries, settings, and subgroups is required.

Trial registration

PROSPERO CRD42019135045.

Interpretable Predictive Models to Understand Risk Factors for Maternal and Fetal Outcomes

Although most pregnancies result in a good outcome, complications are not uncommon and can be associated with serious implications for mothers and babies. Predictive modeling has the potential to improve outcomes through a better understanding of risk factors, heightened surveillance for high-risk patients, and more timely and appropriate interventions, thereby helping obstetricians deliver better care. We identify and study the most important risk factors for four types of pregnancy complications: (i) severe maternal morbidity, (ii) shoulder dystocia, (iii) preterm preeclampsia, and (iv) antepartum stillbirth. We use an Explainable Boosting Machine (EBM), a high-accuracy glass-box learning method, for the prediction and identification of important risk factors. We undertake external validation and perform an extensive robustness analysis of the EBM models. EBMs match the accuracy of other black-box ML methods, such as deep neural networks and random forests, and outperform logistic regression, while being more interpretable. EBMs prove to be robust. The interpretability of the EBM models reveal surprising insights into the features contributing to risk (e.g., maternal height is the second most important feature for shoulder dystocia) and may have potential for clinical application in the prediction and prevention of serious complications in pregnancy.

ADNEX risk prediction model for diagnosis of ovarian cancer: systematic review and meta-analysis of external validation studies

Objectives

To conduct a systematic review of studies externally validating the ADNEX (Assessment of Different Neoplasias in the adnexa) model for diagnosis of ovarian cancer and to present a meta-analysis of its performance.

Design

Systematic review and meta-analysis of external validation studies.

Data sources

Medline, Embase, Web of Science, Scopus, and Europe PMC, from 15 October 2014 to 15 May 2023.

Eligibility criteria for selecting studies

All external validation studies of the performance of ADNEX, with any study design and any study population of patients with an adnexal mass. Two independent reviewers extracted the data. Disagreements were resolved by discussion. Reporting quality of the studies was scored with the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) reporting guideline, and methodological conduct and risk of bias with PROBAST (Prediction model Risk Of Bias Assessment Tool). Random effects meta-analysis of the area under the receiver operating characteristic curve (AUC), sensitivity and specificity at the 10% risk of malignancy threshold, and net benefit and relative utility at the 10% risk of malignancy threshold were performed.

Results

47 studies (17 007 tumours) were included, with a median study sample size of 261 (range 24-4905). On average, 61% of TRIPOD items were reported. Handling of missing data, justification of sample size, and model calibration were rarely described. 91% of validations were at high risk of bias, mainly because of the unexplained exclusion of incomplete cases, small sample size, or no assessment of calibration. The summary AUC to distinguish benign from malignant tumours in patients who underwent surgery was 0.93 (95% confidence interval 0.92 to 0.94, 95% prediction interval 0.85 to 0.98) for ADNEX with the serum biomarker, cancer antigen 125 (CA125), as a predictor (9202 tumours, 43 centres, 18 countries, and 21 studies) and 0.93 (95% confidence interval 0.91 to 0.94, 95% prediction interval 0.85 to 0.98) for ADNEX without CA125 (6309 tumours, 31 centres, 13 countries, and 12 studies). The estimated probability that the model has use clinically in a new centre was 95% (with CA125) and 91% (without CA125). When restricting analysis to studies with a low risk of bias, summary AUC values were 0.93 (with CA125) and 0.91 (without CA125), and estimated probabilities that the model has use clinically were 89% (with CA125) and 87% (without CA125).

Conclusions

The results of the meta-analysis indicated that ADNEX performed well in distinguishing between benign and malignant tumours in populations from different countries and settings, regardless of whether the serum biomarker, CA125, was used as a predictor. A key limitation was that calibration was rarely assessed.

Systematic review registration

PROSPERO CRD42022373182.

Identifying risk of stillbirth using machine learning

BACKGROUND

Previous predictive models using logistic regression for stillbirth do not leverage the advanced and nuanced techniques involved in sophisticated machine learning methods, such as modeling nonlinear relationships between outcomes.

OBJECTIVE

This study aimed to create and refine machine learning models for predicting stillbirth using data available before viability (22-24 weeks) and throughout pregnancy, as well as demographic, medical, and prenatal visit data, including ultrasound and fetal genetics.

STUDY DESIGN

This is a secondary analysis of the Stillbirth Collaborative Research Network, which included data from pregnancies resulting in stillborn and live-born infants delivered at 59 hospitals in 5 diverse regions across the United States from 2006 to 2009. The primary aim was the creation of a model for predicting stillbirth using data available before viability. Secondary aims included refining models with variables available throughout pregnancy and determining variable importance.

RESULTS

Among 3000 live births and 982 stillbirths, 101 variables of interest were identified. Of the models incorporating data available before viability, the random forests model had 85.1% accuracy (area under the curve) and high sensitivity (88.6%), specificity (85.3%), positive predictive value (85.3%), and negative predictive value (84.8%). A random forests model using data collected throughout pregnancy resulted in accuracy of 85.0%; this model had 92.2% sensitivity, 77.9% specificity, 84.7% positive predictive value, and 88.3% negative predictive value. Important variables in the previability model included previous stillbirth, minority race, gestational age at the earliest prenatal visit and ultrasound, and second-trimester serum screening.

CONCLUSION

Applying advanced machine learning techniques to a comprehensive database of stillbirths and live births with unique and clinically relevant variables resulted in an algorithm that could accurately identify 85% of pregnancies that would result in stillbirth, before they reached viability. Once validated in representative databases reflective of the US birthing population and then prospectively, these models may provide effective risk stratification and clinical decision-making support to better identify and monitor those at risk of stillbirth.

Effects of race and ethnicity on perinatal outcomes in high-income and upper-middle-income countries: an individual participant data meta-analysis of 2 198 655 pregnancies

BACKGROUND

Existing evidence on the effects of race and ethnicity on pregnancy outcomes is restricted to individual studies done within specific countries and health systems. We aimed to assess the impact of race and ethnicity on perinatal outcomes in high-income and upper-middle-income countries, and to ascertain whether the magnitude of disparities, if any, varied across geographical regions.

METHODS

For this individual participant data (IPD) meta-analysis we used data from the International Prediction of Pregnancy Complications (IPPIC) Network of studies on pregnancy complications; the full dataset comprised 94 studies, 53 countries, and 4 539 640 pregnancies. We included studies that reported perinatal outcomes (neonatal death, stillbirth, preterm birth, and small-for-gestational-age babies) in at least two racial or ethnic groups (White, Black, south Asian, Hispanic, or other). For our two-step random-effects IPD meta-analysis, we did multiple imputations for confounder variables (maternal age, BMI, parity, and level of maternal education) selected with a directed acyclic graph. The primary outcomes were neonatal mortality and stillbirth. Secondary outcomes were preterm birth and a small-for-gestational-age baby. We estimated the association of race and ethnicity with perinatal outcomes using a multivariate logistic regression model and reported this association with odds ratios (ORs) and 95% CIs. We also did a subgroup analysis of studies by geographical region.

FINDINGS

51 studies from 20 high-income and upper-middle-income countries, comprising 2 198 655 pregnancies, were eligible for inclusion in this IPD meta-analysis. Neonatal death was twice as likely in babies born to Black women than in babies born to White women (OR 2·00, 95% CI 1·44-2·78), as was stillbirth (2·16, 1·46-3·19), and babies born to Black women were at increased risk of preterm birth (1·65, 1·46-1·88) and being small for gestational age (1·39, 1·13-1·72). Babies of women categorised as Hispanic had a three-times increased risk of neonatal death (OR 3·34, 95% CI 2·77-4·02) than did those born to White women, and those born to south Asian women were at increased risk of preterm birth (OR 1·26, 95% CI 1·07-1·48) and being small for gestational age (1·61, 1·32-1·95). The effects of race and ethnicity on preterm birth and small-for-gestational-age babies did not vary across regions.

INTERPRETATION

Globally, among underserved groups, babies born to Black women had consistently poorer perinatal outcomes than White women after adjusting for maternal characteristics, although the risks varied for other groups. The effects of race and ethnicity on adverse perinatal outcomes did not vary by region.

FUNDING

National Institute for Health and Care Research, Wellbeing of Women.

Development and validation of model for prediction of placental dysfunction-related stillbirth from maternal factors, fetal weight and uterine artery Doppler at mid-gestation

OBJECTIVE

To examine the performance of a model combining maternal risk factors, uterine artery pulsatility index (UtA-PI) and estimated fetal weight (EFW) at 19-24 weeks' gestation, for predicting all antepartum stillbirths and those due to impaired placentation, in a training dataset used for development of the model and in a validation dataset.

METHODS

The data for this study were derived from prospective screening for adverse obstetric outcome in women with singleton pregnancy attending for routine pregnancy care at 19 + 0 to 24 + 6 weeks' gestation. The study population was divided into a training dataset used to develop prediction models for placental dysfunction-related antepartum stillbirth and a validation dataset to which the models were then applied. Multivariable logistic regression analysis was used to develop a model based on a combination of maternal risk factors, EFW Z-score and UtA-PI multiples of the normal median. We examined the predictive performance of the model by, first, the ability of the model to discriminate between the stillbirth and live-birth groups, using the area under the receiver-operating-characteristics curve (AUC) and the detection rate (DR) at a fixed false-positive rate (FPR) of 10%, and, second, calibration by measurements of calibration slope and intercept.

RESULTS

The study population of 131 514 pregnancies included 131 037 live births and 477 (0.36%) stillbirths. There are four main findings of this study. First, 92.5% (441/477) of stillbirths were antepartum and 7.5% (36/477) were intrapartum, and 59.2% (261/441) of antepartum stillbirths were observed in association with placental dysfunction and 40.8% (180/441) were unexplained or due to other causes. Second, placental dysfunction accounted for 80.1% (161/201) of antepartum stillbirths at < 32 weeks' gestation, 54.2% (52/96) at 32 + 0 to 36 + 6 weeks and 33.3% (48/144) at ≥ 37 weeks. Third, the risk of placental dysfunction-related antepartum stillbirth increased with increasing maternal weight and decreasing maternal height, was 3-fold higher in black than in white women, was 5.5-fold higher in parous women with previous stillbirth than in those with previous live birth, and was increased in smokers, in women with chronic hypertension and in parous women with a previous pregnancy complicated by pre-eclampsia and/or birth of a small-for-gestational-age baby. Fourth, in screening for placental dysfunction-related antepartum stillbirth by a combination of maternal risk factors, EFW and UtA-PI in the validation dataset, the DR at a 10% FPR was 62.3% (95% CI, 57.2-67.4%) and the AUC was 0.838 (95% CI, 0.799-0.878); these results were consistent with those in the dataset used for developing the algorithm and demonstrate high discrimination between affected and unaffected pregnancies. Similarly, the calibration slope was 1.029 and the intercept was -0.009, demonstrating good agreement between the predicted risk and observed incidence of placental dysfunction-related antepartum stillbirth. The performance of screening was better for placental dysfunction-related antepartum stillbirth at < 37 weeks' gestation compared to at term (DR at a 10% FPR, 69.8% vs 29.2%).

CONCLUSIONS

Screening at mid-gestation by a combination of maternal risk factors, EFW and UtA-PI can predict a high proportion of placental dysfunction-related stillbirths and, in particular, those that occur preterm. Such screening provides poor prediction of unexplained stillbirth or stillbirth due to other causes. © 2021 International Society of Ultrasound in Obstetrics and Gynecology.

Predictive performance for placental dysfunction related stillbirth of the competing risks model for small for gestational age fetuses

OBJECTIVES

First, to examine the predictive performance for placental dysfunction related stillbirths of the competing risks model for small for gestational age (SGA) fetuses based on a combination of maternal risk factors, estimated fetal weight (EFW) and uterine artery pulsatility index (UtA-PI); and second, to compare the performance of this model to that of stillbirth-specific model utilizing the same biomarkers and to the Royal College of Obstetricians and Gynecologists (RCOG) guideline for the investigation and management of the SGA fetus.

DESIGN

Prospective observational study.

SETTING

Two UK maternity hospitals.

POPULATION

131,514 women with singleton pregnancies attending for routine ultrasound examination at 19-24 weeks' gestation.

METHODS

The predictive performance for stillbirth achieved by three models was compared. Main outcome measure Placental dysfunction related stillbirth.

RESULTS

At 10% false positive rate, the competing risks model predicted 59%, 66% and 71% of placental dysfunction related stillbirths, at any gestation, at <37 weeks and at <32 weeks, respectively, which were similar to the respective figures of 62%, 70% and 73% for the stillbirth-specific model. At a screen positive rate of 21.8 %, as defined by the RCOG guideline, the competing risks model predicted 71%, 76% and 79% of placental dysfunction related stillbirths at any gestation, at <37 weeks and at <32 weeks, respectively, and the respective figures for the RCOG guideline were 40%, 44% and 42%.

CONCLUSION

The predictive performance for placental dysfunction related stillbirths by the competing risks model for SGA was similar to the stillbirth-specific model and superior to the RCOG guideline.