External validation and clinical utility of prognostic prediction models for gestational diabetes mellitus: A prospective cohort study

First-Trimester Prediction Models Based on Maternal Characteristics for Adverse Pregnancy Outcomes: A Systematic Review and Meta-Analysis

BACKGROUND

Early risk stratification can facilitate timely interventions for adverse pregnancy outcomes, including preeclampsia (PE), small-for-gestational-age neonates (SGA), spontaneous preterm birth (sPTB) and gestational diabetes mellitus (GDM).

OBJECTIVES

To perform a systematic review and meta-analysis of first-trimester prediction models for adverse pregnancy outcomes.

SEARCH STRATEGY

The PubMed database was searched until 6 June 2024.

SELECTION CRITERIA

First-trimester prediction models based on maternal characteristics were included. Articles reporting on prediction models that comprised biochemical or ultrasound markers were excluded.

DATA COLLECTION AND ANALYSIS

Two authors identified articles, extracted data and assessed risk of bias and applicability using PROBAST.

MAIN RESULTS

A total of 77 articles were included, comprising 30 developed models for PE, 15 for SGA, 11 for sPTB and 35 for GDM. Discriminatory performance in terms of median area under the curve (AUC) of these models was 0.75 [IQR 0.69-0.78] for PE models, 0.62 [0.60-0.71] for SGA models of nulliparous women, 0.74 [0.72-0.74] for SGA models of multiparous women, 0.65 [0.61-0.67] for sPTB models of nulliparous women, 0.71 [0.68-0.74] for sPTB models of multiparous women and 0.71 [0.67-0.76] for GDM models. Internal validation was performed in 40/91 (43.9%) of the models. Model calibration was reported in 21/91 (23.1%) models. External validation was performed a total of 96 times in 45/91 (49.5%) of the models. High risk of bias was observed in 94.5% of the developed models and in 58.3% of the external validations.

CONCLUSIONS

Multiple first-trimester prediction models are available, but almost all suffer from high risk of bias, and internal and external validations were often not performed. Hence, methodological quality improvement and assessment of the clinical utility are needed.

Using Machine Learning and Artificial Intelligence to Predict Diabetes Mellitus among Women Population

BACKGROUND

Diabetes Mellitus is a chronic health condition (long-lasting) due to inadequate control of blood levels of glucose. This study presents a prediction of Type 2 Diabetes Mellitus among women using various Machine Learning Algorithms deployed to predict the diabetic condition. A University of California Irvine Diabetes Mellitus Dataset posted in Kaggle was used for analysis.

METHODS

The dataset included eight risk factors for Type 2 Diabetes Mellitus prediction, including Age, Systolic Blood Pressure, Glucose, Body Mass Index, Insulin, Skin Thickness, Diabetic Pedigree Function, and Pregnancy. R language was used for the data visualization, while the algorithms considered for the study are Logistic Regression, Support Vector Machines, Decision Trees and Extreme Gradient Boost. The performance analysis of these algorithms on various classification metrics is also presented here, considering the Area Under the Curve and Receiver Operating Characteristics score is the best for Extreme Gradient Boost with 85%, followed by Support Vector Machines and Decision Trees.

RESULTS

The Logistic Regression is showing low performance. But the Decision Trees and Extreme Gradient Boost show promising performance against all the classification metrics. But the Support Vector Machines offers a lower support value; hence it cannot be claimed to be a good classifier. The model showed that the most significant predictors of Type 2 Diabetes Mellitus were strongly correlated with Glucose Levels and mediumly correlated with Body Mass Index, whereas Age, Skin Thickness, Systolic Blood Pressure, Insulin, Pregnancy, and Pedigree Function were less significant. This type of real-time analysis has proved that the symptoms of Type 2 Diabetes Mellitus in women fall entirely different compared to men, which highlights the importance of Glucose Levels and Body Mass Index in women.

CONCLUSION

The prediction of Type 2 Diabetes Mellitus helps public health professionals to help people by suggesting proper food intake and adjusting lifestyle activities with good fitness management in women to make glucose levels and body mass index controlled. Therefore, the healthcare systems should give special attention to diabetic conditions in women to reduce exacerbations of the disease and other associated symptoms. This work attempts to predict the occurrence of Type 2 Diabetes Mellitus among women on their behavioral and biological conditions.

The utility of a machine learning model in identifying people at high risk of type 2 diabetes mellitus

BACKGROUND

According to previous reports, very high percentages of individuals in Saudi Arabia are undiagnosed for type 2 diabetes mellitus (T2DM). Despite conducting several screening and awareness campaigns, these efforts lacked full accessibility and consumed extensive human and material resources. Thus, developing machine learning (ML) models could enhance the population-based screening process. The study aims to compare a newly developed ML model's outcomes with the validated American Diabetes Association's (ADA) risk assessment regarding predicting people with high risk for T2DM.

RESEARCH DESIGN AND METHODS

Patients' age, gender, and risk factors that were obtained from the National Health Information Center's dataset were used to build and train the ML model. To evaluate the developed ML model, an external validation study was conducted in three primary health care centers. A random sample (N = 3400) was selected from the non-diabetic individuals.

RESULTS

The results showed the plotted data of sensitivity/100-specificity represented in the Receiver Operating Characteristic (ROC) curve with an AROC value of 0.803, 95% CI: 0.779-0.826.

CONCLUSIONS

The current study reveals a new ML model proposed for population-level classification that can be an adequate tool for identifying those at high risk of T2DM or who already have T2DM but have not been diagnosed.

Prediction of Gestational Diabetes Mellitus in the First Trimester of Pregnancy Based on Maternal Variables and Pregnancy Biomarkers

Gestational diabetes mellitus (GDM) is a significant health concern with adverse outcomes for both pregnant women and their offspring. Recognizing the need for early intervention, this study aimed to develop an early prediction model for GDM risk assessment during the first trimester. Utilizing a prospective cohort of 4917 pregnant women from the Third Department of Obstetrics and Gynecology, Aristotle University of Thessaloniki, Greece, the study sought to combine maternal characteristics, obstetric and medical history, and early pregnancy-specific biomarker concentrations into a predictive tool. The primary objective was to create a series of predictive models that could accurately identify women at high risk for developing GDM, thereby facilitating early and targeted interventions. To this end, maternal age, body mass index (BMI), obstetric and medical history, and biomarker concentrations were analyzed and incorporated into five distinct prediction models. The study's findings revealed that the models varied in effectiveness, with the most comprehensive model combining maternal characteristics, obstetric and medical history, and biomarkers showing the highest potential for early GDM prediction. The current research provides a foundation for future studies to refine and expand upon the predictive models, aiming for even earlier and more accurate detection methods.

A Prediction Model of Gestational Diabetes Mellitus Based on OGTT in Early Pregnancy: A Prospective Cohort Study

CONTEXT

Gestational diabetes mellitus (GDM) is a common obstetric complication. Although early intervention could prevent the development of GDM, there was no consensus on early identification for women at high risk of GDM.

OBJECTIVE

To develop a reliable prediction model of GDM in early pregnancy.

METHODS

In this prospective cohort study, between May 30, 2021, and August 13, 2022, a total of 721 women were included from Women's Hospital, Zhejiang University School of Medicine. Participants were asked to complete an oral glucose tolerance test (OGTT) during gestational weeks 7 through 14 for early prediction of GDM, and at weeks 24 through 28 for GDM diagnosis. Using OGTT results and baseline characteristics, logistic regression analysis was used to construct the prediction model. Receiver operating characteristic (ROC) curve, Hosmer-Lemeshow test, decision clinical analysis, and a nomogram were used for model performances assessment and visualization. Internal and external validation was performed to testify the stability of this model.

RESULTS

According to the International Association of Diabetes and Pregnancy Study Groups criteria in early OGTT, the mean (SD) age was 30.5 ± 3.7 years in low-risk participants and 31.0 ± 3.9 years in high-risk participants. The area under ROC curve (AUC) of the existing criteria at weeks 7 through 14 varied from 0.705 to 0.724. Based on maternal age, prepregnancy body mass index, and results of early OGTT, the AUC of our prediction model was 0.8720, which was validated by both internal (AUC 0.8541) and external (AUC 0.8241) confirmation.

CONCLUSIONS

The existing diagnostic criteria were unsatisfactory for early prediction of GDM. By combining early OGTT, we provided an effective prediction model of GDM in the first trimester.

Clinical First-Trimester Prediction Models for Gestational Diabetes Mellitus: A Systematic Review and Meta-Analysis

BACKGROUND

Gestational diabetes mellitus (GDM) is a common pregnancy complication that negatively impacts the health of both the mother and child. Early prediction of the risk of GDM may permit prompt and effective interventions. This systematic review and meta-analysis aimed to summarize the study characteristics, methodological quality, and model performance of first-trimester prediction model studies for GDM.

METHODS

Five electronic databases, one clinical trial register, and gray literature were searched from the inception date to March 19, 2022. Studies developing or validating a first-trimester prediction model for GDM were included. Two reviewers independently extracted data according to an established checklist and assessed the risk of bias by the Prediction Model Risk of Bias Assessment Tool (PROBAST). We used a random-effects model to perform a quantitative meta-analysis of the predictive power of models that were externally validated at least three times.

RESULTS

We identified 43 model development studies, six model development and external validation studies, and five external validation-only studies. Body mass index, maternal age, and fasting plasma glucose were the most commonly included predictors across all models. Multiple estimates of performance measures were available for eight of the models. Summary estimates range from 0.68 to 0.78 (I² ranged from 0% to 97%).

CONCLUSION

Most studies were assessed as having a high overall risk of bias. Only eight prediction models for GDM have been externally validated at least three times. Future research needs to focus on updating and externally validating existing models.

Utilizing fog computing and explainable deep learning techniques for gestational diabetes prediction

Gestational diabetes mellitus (GDM) is one of the pregnancy complications that poses a significant risk on mothers and babies as well. GDM usually diagnosed at 22–26 of gestation. However, the early prediction is desirable as it may contribute to decrease the risk. The continuous monitoring for mother’s vital signs helps in predicting any deterioration during pregnancy. The originality of this paper is to provide comprehensive framework for pregnancy women monitoring. The proposed Data Replacement and Prediction Framework consists of three layers which are: (i) IoT Layer, (ii) Fog Layer, and (iii) Cloud Layer. The first layer used IOT sensors to aggregate vital sings from pregnancies using invasive and noninvasive sensors. Then the vital signs transmitted to fog nodes to processed and finally stored in the cloud layer. The main contribution in this paper is located in the fog layer producing GDM module to implement two influential tasks which are: (i) Data Finding Methodology (DFM), and (ii) Explainable Prediction Algorithm (EPM) using DNN. First, the DFM is used to replace the unused data to free the cache space for the new incoming data items. The cache replacement is very important in the case of healthcare system as the incoming vital signs are frequent and must be replaced continuously. Second, the EPM is used to predict the incidence of GDM that may occur in the second trimester of the pregnancy. To evaluate our model, we extract data of 16,354 pregnancy women from medical information mart for intensive care (MIMIC III) benchmark dataset. For each woman, vital signs, demographic data and laboratory tests was aggregated. The results of the prediction model superior the state of the art (ACC = 0.957, AUC = 0.942). Regarding to explainability, we utilized Shapley additive explanation framework to provide local and global explanation for the developed models. Overall, the proposed framework is medically intuitive, allow the early prediction of GDM with cost effective solution.

Prediction of gestational diabetes based on explainable deep learning and fog computing

Gestational diabetes mellitus (GDM) is one of the pregnancy complications that endangers both mothers and babies. GDM is usually diagnosed at 22–26 weeks of gestation. However, early prediction is preferable because it may decrease the risk. The continuous monitoring of the mother’s vital signs helps in predicting any deterioration during pregnancy. The originality of this research is to provide a comprehensive framework for pregnancy women monitoring. The proposed Data Replacement and Prediction Framework consists of three layers, which are: (i) Internet of things (IoT) Layer, (ii) Fog Layer, and (iii) Cloud Layer. The first layer used IoT sensors to aggregate vital signs from pregnancies using invasive and non-invasive sensors. The vital signs are then transmitted to fog nodes to be processed and finally stored in the cloud layer. The main contribution in this research is located in the fog layer producing the GDM module to implement two influential tasks which are as follows: (i) Data Finding Methodology (DFM), and (ii) Explainable Prediction Algorithm (EPM) using DNN. First, the DFM is used to replace the unused data to free up the cache space for new incoming data items. The cache replacement is very important in the case of the healthcare system as the incoming vital signs are frequent and must be replaced continuously. Second, the EPM is used to predict the occurrence of GDM in the second trimester of the pregnancy. To evaluate our model, we extracted data from 16,354 pregnant women from the medical information mart for intensive care (MIMIC III) benchmark dataset. For each woman, vital signs, demographic data, and laboratory tests were aggregated. The results of the prediction model are superior to the state-of-the-art (ACC = 0.957, AUC = 0.942). Regarding explainability, we used Shapley additive explanation (SHAP) framework to provide local and global explanations for the developed models. Overall, the proposed framework is medically intuitive and allows the early prediction of GDM with a cost-effective solution.

Optimising Cardiometabolic Risk Factors in Pregnancy: A Review of Risk Prediction Models Targeting Gestational Diabetes and Hypertensive Disorders

Cardiovascular disease, especially coronary heart disease and cerebrovascular disease, is a leading cause of mortality and morbidity in women globally. The development of cardiometabolic conditions in pregnancy, such as gestational diabetes mellitus and hypertensive disorders of pregnancy, portend an increased risk of future cardiovascular disease in women. Pregnancy therefore represents a unique opportunity to detect and manage risk factors, prior to the development of cardiovascular sequelae. Risk prediction models for gestational diabetes mellitus and hypertensive disorders of pregnancy can help identify at-risk women in early pregnancy, allowing timely intervention to mitigate both short- and long-term adverse outcomes. In this narrative review, we outline the shared pathophysiological pathways for gestational diabetes mellitus and hypertensive disorders of pregnancy, summarise contemporary risk prediction models and candidate predictors for these conditions, and discuss the utility of these models in clinical application.

Performance of early risk assessment tools to predict the later development of gestational diabetes

BACKGROUND

Several prognostic models for gestational diabetes mellitus (GDM) are provided in the literature; however, their clinical significance has not been thoroughly evaluated, especially with regard to application at early gestation and in accordance with the most recent diagnostic criteria. This external validation study aimed to assess the predictive accuracy of published risk estimation models for the later development of GDM at early pregnancy.

METHODS

In this cohort study, we prospectively included 1132 pregnant women. Risk evaluation was performed before 16 + 0 weeks of gestation including a routine laboratory examination. Study participants were followed-up until delivery to assess GDM status according to the IADPSG 2010 diagnostic criteria. Fifteen clinical prediction models were calculated according to the published literature.

RESULTS

Gestational diabetes mellitus was diagnosed in 239 women, that is 21.1% of the study participants. Discrimination was assessed by the area under the ROC curve and ranged between 60.7% and 76.9%, corresponding to an acceptable accuracy. With some exceptions, calibration performance was poor as most models were developed based on older diagnostic criteria with lower prevalence and therefore tended to underestimate the risk of GDM. The highest variable importance scores were observed for history of GDM and routine laboratory parameters.

CONCLUSIONS

Most prediction models showed acceptable accuracy in terms of discrimination but lacked in calibration, which was strongly dependent on study settings. Simple biochemical variables such as fasting glucose, HbA1c and triglycerides can improve risk prediction. One model consisting of clinical and laboratory parameters showed satisfactory accuracy and could be used for further investigations.

Implementation of a first-trimester prognostic model to improve screening for gestational diabetes mellitus

Background

Improvement in the accuracy of identifying women who are at risk to develop gestational diabetes mellitus (GDM) is warranted, since timely diagnosis and treatment improves the outcomes of this common pregnancy disorder. Although prognostic models for GDM are externally validated and outperform current risk factor based selective approaches, there is little known about the impact of such models in day-to-day obstetric care.

Methods

A prognostic model was implemented as a directive clinical prediction rule, classifying women as low- or high-risk for GDM, with subsequent distinctive care pathways including selective midpregnancy testing for GDM in high-risk women in a prospective multicenter birth cohort comprising 1073 pregnant women without pre-existing diabetes and 60 obstetric healthcare professionals included in nine independent midwifery practices and three hospitals in the Netherlands (effectiveness-implementation hybrid type 2 study). Model performance (c-statistic) and implementation outcomes (acceptability, adoption, appropriateness, feasibility, fidelity, penetration, sustainability) were evaluated after 6 months by indicators and implementation instruments (NoMAD; MIDI).

Results

The adherence to the prognostic model (c-statistic 0.85 (95%CI 0.81–0.90)) was 95% (n = 1021). Healthcare professionals scored 3.7 (IQR 3.3–4.0) on implementation instruments on a 5-point Likert scale. Important facilitators were knowledge, willingness and confidence to use the model, client cooperation and opportunities for reconfiguration. Identified barriers mostly related to operational and organizational issues. Regardless of risk-status, pregnant women appreciated first-trimester information on GDM risk-status and lifestyle advice to achieve risk reduction, respectively 89% (n = 556) and 90% (n = 564)).

Conclusions

The prognostic model was successfully implemented and well received by healthcare professionals and pregnant women. Prognostic models should be recommended for adoption in guidelines.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12884-021-03749-x.

Adherence rates to a prediction tool identifying women with an increased gestational diabetes risk: An implementation study

OBJECTIVE

The best screening strategy for gestational diabetes mellitus (GDM) remains a topic of debate. Several organizations made a statement in favor of universal screening, but the volume of oral glucose tolerance tests (OGTT) required may burden healthcare systems. As a result, many countries still rely on selective screening using a checklist of risk factors, but reported diagnostic characteristics vary. Moreover, women's discomfort due to an OGTT is often neglected. Since 2017, obstetric healthcare professionals in a Dutch region assessed women's GDM risk with a prediction model and counseled those with an increased risk regarding an OGTT.

METHODS

From 2017 to 2018, 865 women were recruited in a multicenter prospective cohort.

RESULTS

In total, 385 women (48%) had an increased predicted GDM risk. Of all women, 78% reported that their healthcare professional discussed their GDM risk. Predicted GDM risks were positively correlated with conducting an OGTT.

CONCLUSION

Implementation of a GDM prediction model resulted in moderate rates of OGTTs performed in general, but high rates in high-risk women. As 25% of women experienced discomfort from the OGTT, a selective screening strategy based on a prediction model with a high detection rate may be an interesting alternative to universal screening.

STUDY COHORT REGISTRATION

Netherlands Trial Register: NTR4143; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=4143.

Machine Learning Prediction Models for Gestational Diabetes Mellitus: A meta- analysis (Preprint)

Background

Gestational diabetes mellitus (GDM) is a common endocrine metabolic disease, involving a carbohydrate intolerance of variable severity during pregnancy. The incidence of GDM-related complications and adverse pregnancy outcomes has declined, in part, due to early screening. Machine learning (ML) models are increasingly used to identify risk factors and enable the early prediction of GDM.

Objective

The aim of this study was to perform a meta-analysis and comparison of published prognostic models for predicting the risk of GDM and identify predictors applicable to the models.

Methods

Four reliable electronic databases were searched for studies that developed ML prediction models for GDM in the general population instead of among high-risk groups only. The novel Prediction Model Risk of Bias Assessment Tool (PROBAST) was used to assess the risk of bias of the ML models. The Meta-DiSc software program (version 1.4) was used to perform the meta-analysis and determination of heterogeneity. To limit the influence of heterogeneity, we also performed sensitivity analyses, a meta-regression, and subgroup analysis.

Results

A total of 25 studies that included women older than 18 years without a history of vital disease were analyzed. The pooled area under the receiver operating characteristic curve (AUROC) for ML models predicting GDM was 0.8492; the pooled sensitivity was 0.69 (95% CI 0.68-0.69; P<.001; I²=99.6%) and the pooled specificity was 0.75 (95% CI 0.75-0.75; P<.001; I²=100%). As one of the most commonly employed ML methods, logistic regression achieved an overall pooled AUROC of 0.8151, while non–logistic regression models performed better, with an overall pooled AUROC of 0.8891. Additionally, maternal age, family history of diabetes, BMI, and fasting blood glucose were the four most commonly used features of models established by the various feature selection methods.

Conclusions

Compared to current screening strategies, ML methods are attractive for predicting GDM. To expand their use, the importance of quality assessments and unified diagnostic criteria should be further emphasized.

External validation and clinical utility of prognostic prediction models for gestational diabetes mellitus: A prospective cohort study

Introduction

We performed an independent validation study of all published first trimester prediction models, containing non‐invasive predictors, for the risk of gestational diabetes mellitus. Furthermore, the clinical potential of the best performing models was evaluated.

Material and methods

Systemically selected prediction models from the literature were validated in a Dutch prospective cohort using data from Expect Study I and PRIDE Study. The predictive performance of the models was evaluated by discrimination and calibration. Clinical utility was assessed using decision curve analysis. Screening performance measures were calculated at different risk thresholds for the best model and compared with current selective screening strategies.

Results

The validation cohort included 5260 women. Gestational diabetes mellitus was diagnosed in 127 women (2.4%). The discriminative performance of the 12 included models ranged from 68% to 75%. Nearly all models overestimated the risk. After recalibration, agreement between the observed outcomes and predicted probabilities improved for most models.

Conclusions

The best performing prediction models showed acceptable performance measures and may enable more personalized medicine‐based antenatal care for women at risk of developing gestational diabetes mellitus compared with current applied strategies.