Embryo selection with artificial intelligence: how to evaluate and compare methods?

How should the best human embryo in vitro be? Current and future challenges for embryo selection

In-vitro fertilization (IVF) aims at overcoming the causes of infertility and lead to a healthy live birth. To maximize IVF efficiency, it is critical to identify and transfer the most competent embryo within a cohort produced by a couple during a cycle. Conventional static embryo morphological assessment involves sequential observations under a light microscope at specific timepoints. The introduction of time-lapse technology enhanced morphological evaluation via the continuous monitoring of embryo preimplantation in vitro development, thereby unveiling features otherwise undetectable via multiple static assessments. Although an association exists, blastocyst morphology poorly predicts chromosomal competence. In fact, the only reliable approach currently available to diagnose the embryonic karyotype is trophectoderm biopsy and comprehensive chromosome testing to assess non-mosaic aneuploidies, namely preimplantation genetic testing for aneuploidies (PGT-A). Lately, the focus is shifting towards the fine-tuning of non-invasive technologies, such as "omic" analyses of waste products of IVF (e.g., spent culture media) and/or artificial intelligence-powered morphologic/morphodynamic evaluations. This review summarizes the main tools currently available to assess (or predict) embryo developmental, chromosomal, and reproductive competence, their strengths, the limitations, and the most probable future challenges.

Assessment of artificial intelligence model and manual morphokinetic annotation system as embryo grading methods for successful live birth prediction: a retrospective monocentric study

PURPOSE

The introduction of the time-lapse monitoring system (TMS) and the development of predictive algorithms could contribute to the optimal embryos selection for transfer. Therefore, the present study aims at investigating the efficiency of KIDScore and iDAScore systems for blastocyst stage embryos in predicting live birth events.

METHODS

The present retrospective study was conducted in a private IVF Unit setting throughout a 10-month period from October 2021 to July 2022, and included the analysis of 429 embryos deriving from 91 IVF/ICSI cycles conducted due to infertility of various etiologies. Embryos incubated at the Embryoscope+ timelapse incubator were analyzed through the established scoring systems: KIDScore and iDAScore®. The main outcome measure was the comparison of the two scoring systems in terms of live birth prediction. Embryos with the higher scores at day 5 (KID5 score/iDA5 score) were transferred or cryopreserved for later use.

RESULTS

Embryos with high KID5 and iDA5 scores positively correlated with the probability of successful live birth, with KID5 score yielding a higher efficiency in predicting a successful reproductive outcome compared to a proportionally high iDA5 score. KID5 demonstrated conservative performance in successfully predicting live birth compared to iDA5 score, indicating that an efficient prediction can be either provided by a relatively lower KID5 score or a relatively higher iDA5 score.

CONCLUSION

The developed artificial intelligence tools should be implemented in clinical practice in conjunction with the conventional morphological assessment for the conduction of optimized embryo transfer in terms of a successful live birth.

Criteria for implementing artificial intelligence systems in reproductive medicine

This review article discusses the integration of artificial intelligence (AI) in assisted reproductive technology and provides key concepts to consider when introducing AI systems into reproductive medicine practices. The article highlights the various applications of AI in reproductive medicine and discusses whether to use commercial or in-house AI systems. This review also provides criteria for implementing new AI systems in the laboratory and discusses the factors that should be considered when introducing AI in the laboratory, including the user interface, scalability, training, support, follow-up, cost, ethics, and data quality. The article emphasises the importance of ethical considerations, data quality, and continuous algorithm updates to ensure the accuracy and safety of AI systems.

How great thou ART: biomechanical properties of oocytes and embryos as indicators of quality in assisted reproductive technologies

Assisted Reproductive Technologies (ART) have revolutionized infertility treatment and animal breeding, but their success largely depends on selecting high-quality oocytes for fertilization and embryos for transfer. During preimplantation development, embryos undergo complex morphogenetic processes, such as compaction and cavitation, driven by cellular forces dependent on cytoskeletal dynamics and cell-cell interactions. These processes are pivotal in dictating an embryo's capacity to implant and progress to full-term development. Hence, a comprehensive grasp of the biomechanical attributes characterizing healthy oocytes and embryos is essential for selecting those with higher developmental potential. Various noninvasive techniques have emerged as valuable tools for assessing biomechanical properties without disturbing the oocyte or embryo physiological state, including morphokinetics, analysis of cytoplasmic movement velocity, or quantification of cortical tension and elasticity using microaspiration. By shedding light on the cytoskeletal processes involved in chromosome segregation, cytokinesis, cellular trafficking, and cell adhesion, underlying oogenesis, and embryonic development, this review explores the significance of embryo biomechanics in ART and its potential implications for improving clinical IVF outcomes, offering valuable insights and research directions to enhance oocyte and embryo selection procedures.

Embryo ranking agreement between embryologists and artificial intelligence algorithms

OBJECTIVE

To evaluate the degree of agreement of embryo ranking between embryologists and eight artificial intelligence (AI) algorithms.

DESIGN

Retrospective study.

PATIENT(S)

A total of 100 cycles with at least eight embryos were selected from the Weill Cornell Medicine database. For each embryo, the full-length time-lapse (TL) videos, as well as a single embryo image at 120 hours, were given to five embryologists and eight AI algorithms for ranking.

INTERVENTION(S)

None.

MAIN OUTCOME MEASURE(S)

Kendall rank correlation coefficient (Kendall's τ).

RESULT(S)

Embryologists had a high degree of agreement in the overall ranking of 100 cycles with an average Kendall's tau (K-τ) of 0.70, slightly lower than the interembryologist agreement when using a single image or video (average K-τ = 0.78). Overall agreement between embryologists and the AI algorithms was significantly lower (average K-τ = 0.53) and similar to the observed low inter-AI algorithm agreement (average K-τ = 0.47). Notably, two of the eight algorithms had a very low agreement with other ranking methodologies (average K-τ = 0.05) and between each other (K-τ = 0.01). The average agreement in selecting the best-quality embryo (1/8 in 100 cycles with an expected agreement by random chance of 12.5%; confidence interval [CI]95: 6%-19%) was 59.5% among embryologists and 40.3% for six AI algorithms. The incidence of the agreement for the two algorithms with the low overall agreement was 11.7%. Agreement on selecting the same top two embryos/cycle (expected agreement by random chance corresponds to 25.0%; CI95: 17%-32%) was 73.5% among embryologists and 56.0% among AI methods excluding two discordant algorithms, which had an average agreement of 24.4%, the expected range of agreement by random chance. Intraembryologist ranking agreement (single image vs. video) was 71.7% and 77.8% for single and top two embryos, respectively. Analysis of average raw scores indicated that cycles with low diversity of embryo quality generally resulted in a lower overall agreement between the methods (embryologists and AI models).

CONCLUSION(S)

To our knowledge, this is the first study that evaluates the level of agreement in ranking embryo quality between different AI algorithms and embryologists. The different concordance methods were consistent and indicated that the highest agreement was intraembryologist agreement, followed by interembryologist agreement. In contrast, the agreement between some of the AI algorithms and embryologists was similar to the inter-AI algorithm agreement, which also showed a wide range of pairwise concordance. Specifically, two AI models showed intra- and interagreement at the level expected from random selection.

Pretreatment prediction for IVF outcomes: generalized applicable model or centre-specific model?

STUDY QUESTION

What was the performance of different pretreatment prediction models for IVF, which were developed based on UK/US population (McLernon 2016 model, Luke model, Dhillon model, and McLernon 2022 model), in wider populations?

SUMMARY ANSWER

For a patient in China, the published pretreatment prediction models based on the UK/US population provide similar discriminatory power with reasonable AUCs and underestimated predictions.

WHAT IS KNOWN ALREADY

Several pretreatment prediction models for IVF allow patients and clinicians to estimate the cumulative probability of live birth in a cycle before the treatment, but they are mostly based on the population of Europe or the USA, and their performance and applicability in the countries and regions beyond these regions are largely unknown.

STUDY DESIGN, SIZE, DURATION

A total of 26 382 Chinese patients underwent oocyte pick-up cycles between January 2013 and December 2020.

PARTICIPANTS/MATERIALS, SETTING, METHODS

UK/US model performance was externally validated according to the coefficients and intercepts they provided. Centre-specific models were established with XGboost, Lasso, and generalized linear model algorithms. Discriminatory power and calibration of the models were compared as the forms of the AUC of the Receiver Operator Characteristic and calibration curves.

MAIN RESULTS AND THE ROLE OF CHANCE

The AUCs for McLernon 2016 model, Luke model, Dhillon model, and McLernon 2022 model were 0.69 (95% CI 0.68-0.69), 0.67 (95% CI 0.67-0.68), 0.69 (95% CI 0.68-0.69), and 0.67 (95% CI 0.67-0.68), respectively. The centre-specific yielded an AUC of 0.71 (95% CI 0.71-0.72) with key predictors including age, duration of infertility, and endocrine parameters. All external models suggested underestimation. Among the external models, the rescaled McLernon 2022 model demonstrated the best calibration (Slope 1.12, intercept 0.06).

LIMITATIONS, REASONS FOR CAUTION

The study is limited by its single-centre design and may not be representative elsewhere. Only per-complete cycle validation was carried out to provide a similar framework to compare different models in the sample population. Newer predictors, such as AMH, were not used.

WIDER IMPLICATIONS OF THE FINDINGS

Existing pretreatment prediction models for IVF may be used to provide useful discriminatory power in populations different from those on which they were developed. However, models based on newer more relevant datasets may provide better calibrations.

STUDY FUNDING/COMPETING INTEREST(S)

This work was supported by the National Natural Science Foundation of China [grant number 22176159], the Xiamen Medical Advantage Subspecialty Construction Project [grant number 2018296], and the Special Fund for Clinical and Scientific Research of Chinese Medical Association [grant number 18010360765].

TRIAL REGISTRATION NUMBER

N/A.

Diagnostic or prognostic? Decoding the role of embryo selection on in vitro fertilization treatment outcomes

In this review, we take a fresh look at embryo assessment and selection methods from the perspective of diagnosis and prognosis. On the basis of a systematic search in the literature, we examined the evidence on the prognostic value of different embryo assessment methods, including morphological assessment, blastocyst culture, time-lapse imaging, artificial intelligence, and preimplantation genetic testing for aneuploidy.

Improved pregnancy prediction performance in an updated deep-learning embryo selection model: a retrospective independent validation study

RESEARCH QUESTION

What is the effect of increasing training data on the performance of ongoing pregnancy prediction after single vitrified-warmed blastocyst transfer (SVBT) in a deep-learning model?

DESIGN

A total of 3960 SVBT cycles were retrospectively analysed. Embryos were stratified according to the Society for Assisted Reproductive Technology age groups. Embryos were scored by deep-learning models iDAScore v1.0 (IDA-V1) and iDAScore v2.0 (IDA-V2) (15% more training data than v1.0) and by Gardner grading. The discriminative performance of the pregnancy prediction for each embryo scoring model was compared using the area under the curve (AUC) of the receiver operating characteristic curve for each maternal age group.

RESULTS

The AUC of iDA-V2, iDA-V1 and Gardener grading in all cohort were 0.736, 0.720 and 0.702, respectively. iDA-V2 was significantly higher than iDA-V1 and Gardener grading (P < 0.0001). Group > 35 years (n = 757): the AUC of iDA-V2 was significantly higher than Gardener grading (0.718 versus 0.694, P = 0.015); group aged 35-37 years (n = 821), the AUC of iDA-V2 was significantly higher than iDA-V1 (0.712 versus 0.696, P = 0.035); group aged 41-42 years (n = 715, the AUC of iDA-V2 was significantly higher than Gardener grading (0.745 versus 0.696, P = 0.007); group > 42 years (n = 660) and group aged 38-40 years (n = 1007), no significant differences were found between the groups.

CONCLUSION

The performance of deep learning models for pregnancy prediction will be improved by increasing the size of the training data.

Novel embryo selection strategies-finding the right balance

The use of novel technologies in the selection of embryos during in vitro fertilisation (IVF) has the potential to improve the chances of pregnancy and birth of a healthy child. However, it is important to be aware of the potential risks and unintended consequences that may arise from the premature implementation of these technologies. This article discusses the ethical considerations surrounding the use of novel embryo selection technologies in IVF, including the growing uptake of genetic testing and others, and argues that prioritising embryos for transfer using these technologies is acceptable, but discarding embryos based on unproven advances is not. Several historical examples are provided, which demonstrate possible harms, where the overall chance of pregnancy may have been reduced, and some patients may have missed out on biological parenthood altogether. We emphasise the need for caution and a balanced approach to ensure that the benefits of these technologies outweigh any potential harm. We also highlight the primacy of patients' autonomy in reproductive decision-making, especially when information gained by utilising novel technologies is imprecise.

Interpretable artificial intelligence-assisted embryo selection improved single-blastocyst transfer outcomes: a prospective cohort study

RESEARCH QUESTION

What is the pregnancy and neonatal outcomes of an interpretable artificial intelligence (AI) model for embryo selection in a prospective clinical trial?

DESIGN

This single-centre prospective cohort study was carried out from October 2021 to March 2022. A total of 330 eligible patients were assigned to their preferred groups, with 250 patients undergoing a fresh single-blastocyst transfer cycle after the exclusion criteria had been applied. For the AI-assisted group (AAG), embryologists selected the embryos for transfer based on the ranking recommendations provided by an interpretable AI system, while with the manual group, embryologists used the Gardner grading system to make their decisions.

RESULTS

The implantation rate was significantly higher in the AAG than the manual group (80.87% versus 68.15%, P = 0.022). No significant difference was found in terms of monozygotic twin rate, miscarriage rate, live birth rate and ectopic pregnancy rate between the groups. Furthermore, there was no significant difference in terms of neonatal outcomes, including gestational weeks, premature birth rate, birth height, birthweight, sex ratio at birth and newborn malformation rate. The consensus rate between the AI and retrospective analysis by the embryologists was significantly higher for good-quality embryos (i.e. grade 4BB or higher) versus poor-quality embryos (i.e. less than 4BB) (84.71% versus 25%, P < 0.001).

CONCLUSIONS

These prospective trial results suggest that the proposed AI system could effectively help embryologists to improve the implantation rate with single-blastocyst transfer compared with traditional manual evaluation methods.

Identifying predictors of Day 5 blastocyst utilization rate using an artificial neural network

RESEARCH QUESTION

Can artificial intelligence identify predictors of an increased Day 5 blastocyst utilization rate (D5BUR), which is one of the most informative key performance indicators in an IVF laboratory?

DESIGN

This retrospective, multicentre study evaluated six variables for predicting D5BUR using an artificial neural network (ANN): number of metaphase II (MII) oocytes injected (intracytoplasmic sperm injection); use of autologous/donated gametes; maternal age at oocyte retrieval; sperm concentration; progressive sperm motility rate; and fertilization rate. Cycles were divided into training and testing sets through stratified random sampling. D5BUR on Day 5 was grouped into <60% and ≥60% as per the Vienna consensus benchmark values.

RESULTS

The area under the receiver operating characteristic curve (AUC) to predict the D5BUR groups was 80.2%. From the ANN model, all six independent variables were found to be of significant value for the prediction of D5BUR (P<0.0001), with the most important variable being the number of MII oocytes injected. Investigation of the effect of MII oocytes injected on D5BUR indicated an inverse correlation, with injection of an increasing number of MII oocytes resulting in a decreasing D5BUR (r=-0.344, P<0.001) and injection of up to six oocytes resulting in D5BUR ≥60%.

CONCLUSION

The number of MII oocytes injected is the most important predictor of D5BUR. Exploration of additional variables and further validation of models that can predict D5BUR can guide the way towards personalized treatment and increased safety.

Using blastocyst re-expansion rate for deciding when to warm a new blastocyst for single vitrified-warmed blastocyst transfer

RESEARCH QUESTION

Can predictive post-warm parameters that support the decision to transfer a warmed blastocyst or to warm another blastocyst be identified in women with multiple frozen-vitrified blastocysts?

DESIGN

Retrospective single-centre observational cohort analysis. A total of 1092 single vitrified-warmed blastocyst transfers (SVBT) with known Gardner score, maternal age and live birth were used to develop live birth prediction models based on logistic regression, including post-warm re-expansion parameters. Time-lapse incubation was used for pre-vitrification and post-warm embryo culture. A dataset of 558 SVBT with the same inclusion criteria was used to validate the model, but with known clinical pregnancy outcome instead of live birth outcome.

RESULTS

Three different logistic regression models were developed for predicting live birth based on post-warm blastocyst re-expansion. Different post-warm assessment times indicated that a 2-h post-warm culture period was optimal for live birth prediction (model 1). Adjusting for pre-vitrification Gardner score (model 2) and in combination with maternal age (model 3) further increased predictability (area under the curve [AUC] = 0.623, 0.633, 0.666, respectively). Model validation gave an AUC of 0.617, 0.609 and 0.624, respectively. The false negative rate and true negative rate for model 3 were 2.0 and 10.1 in the development dataset and 3.5 and 8.0 in the validation dataset.

CONCLUSIONS

Clinical application of a simple model based on 2 h of post-warm re-expansion data, pre-vitrification Gardner score and maternal age can support a standardized approach for deciding if warming another blastocyst may increase the likelihood of live birth in SVBT.

Correlations between a deep learning-based algorithm for embryo evaluation with cleavage-stage cell numbers and fragmentation

RESEARCH QUESTION

Do cell numbers and degree of fragmentation in cleavage-stage embryos, assessed manually, correlate with evaluations made by deep learning algorithm model iDAScore v2.0?

DESIGN

Retrospective observational study (n = 5040 embryos; 1786 treatments) conducted at two Swedish assisted reproductive technology centres between 2016 and 2021. Fresh single embryo transfer was carried out on days 2 or 3 after fertilization. Embryo evaluation using iDAScore v2.0 was compared with manual assessment of numbers of cells and grade of fragmentation, analysed by video sequences.

RESULTS

Data from embryos transferred on days 2 and 3 showed that having three or fewer cells compared with four or fewer cells on day 2, and six or fewer cells versus seven to eight cells on day 3, correlated significantly with a difference in iDAScore (medians 2.4 versus 4.0 and 2.6 versus 4.6 respectively; both P < 0.001). The iDAScore for 0-10% fragmentation was significantly higher compared with the groups with higher fragmentation (P < 0.001). When combining cell numbers and fragmentation, iDAScore values decreased as fragmentation increased, regardless of cell number. iDAScore discriminated between embryos that resulted in live birth or no live birth (AUC of 0.627 and 0.607), compared with the morphological model (AUC of 0.618 and 0.585) for day 2 and day 3, respectively.

CONCLUSIONS

The iDAScore v2.0 values correlated significantly with cell numbers and fragmentation scored manually for cleavage-stage embryos on days 2 and 3. iDAScore had some predictive value for live birth, conditional that embryo selection was based on morphology.

Deep learning for embryo evaluation using time-lapse: a systematic review of diagnostic test accuracy

OBJECTIVE

This study aimed to investigate the accuracy of convolutional neural network models in the assessment of embryos using time-lapse monitoring.

DATA SOURCES

A systematic search was conducted in PubMed and Web of Science databases from January 2016 to December 2022. The search strategy was carried out by using key words and MeSH (Medical Subject Headings) terms.

STUDY ELIGIBILITY CRITERIA

Studies were included if they reported the accuracy of convolutional neural network models for embryo evaluation using time-lapse monitoring. The review was registered with PROSPERO (International Prospective Register of Systematic Reviews; identification number CRD42021275916).

METHODS

Two reviewer authors independently screened results using the Covidence systematic review software. The full-text articles were reviewed when studies met the inclusion criteria or in any uncertainty. Nonconsensus was resolved by a third reviewer. Risk of bias and applicability were evaluated using the QUADAS-2 tool and the modified Joanna Briggs Institute or JBI checklist.

RESULTS

Following a systematic search of the literature, 22 studies were identified as eligible for inclusion. All studies were retrospective. A total of 522,516 images of 222,998 embryos were analyzed. Three main outcomes were evaluated: successful in vitro fertilization, blastocyst stage classification, and blastocyst quality. Most studies reported >80% accuracy, and embryologists were outperformed in some. Ten studies had a high risk of bias, mostly because of patient bias.

CONCLUSION

The application of artificial intelligence in time-lapse monitoring has the potential to provide more efficient, accurate, and objective embryo evaluation. Models that examined blastocyst stage classification showed the best predictions. Models that predicted live birth had a low risk of bias, used the largest databases, and had external validation, which heightens their relevance to clinical application. Our systematic review is limited by the high heterogeneity among the included studies. Researchers should share databases and standardize reporting.

Association between a morphokinetic ploidy prediction model risk score and miscarriage and live birth: a multicentre cohort study

OBJECTIVE

To determine whether the aneuploidy risk score from a morphokinetic ploidy prediction model, Predicting Euploidy for Embryos in Reproductive Medicine (PREFER), is associated with miscarriage and live birth outcomes.

DESIGN

Multicentre cohort study.

SETTING

Nine in vitro fertilization clinics in the United Kingdom.

PATIENTS

Data were obtained from the treatment of patients from 2016-2019. A total of 3587 fresh single embryo transfers were included; preimplantation genetic testing for aneuploidy) cycles were excluded.

INTERVENTION

PREFER is a model developed using 8,147 biopsied blastocyst specimens to predict ploidy status using morphokinetic and clinical biodata. A second model using only morphokinetic (MK) predictors was developed, P PREFER-MK. The models will categorize embryos into the following three risk score categories for aneuploidy: "high risk," "medium risk," and "low risk."

MAIN OUTCOME MEASURES

The primary outcomes are miscarriage and live birth. Secondary outcomes include biochemical clinical pregnancy per single embryo transfer.

RESULTS

When applying PREFER, the miscarriage rates were 12%, 14%, and 22% in the "low risk," "moderate risk," and "high risk" categories, respectively. Those embryos deemed "high risk" had a significantly higher egg provider age compared with "low risk," and there was little variation in risk categories in patients of the same age. The trend in miscarriage rate was not seen when using PREFER-MK; however, there was an association with live birth, increasing from 38%-49% and 50% in the "high risk," "moderate risk," and "low risk" groups, respectively. An adjusted logistic regression analysis demonstrated that PREFER-MK was not associated with miscarriage when comparing "high risk" to "moderate risk" embryos (odds ratio [OR], 0.87; 95% confidence interval [CI], 0.63-1.63) or "high risk" to "low risk" embryos (OR, 1.07; 95% CI, 0.79-1.46). An embryo deemed "low risk" by PREFER-MK was significantly more likely to result in a live birth than those embryos graded "high risk" (OR, 1.95; 95% CI, 1.65-2.25).

CONCLUSION

The PREFER model's risk scores were significantly associated with live births and miscarriages. Importantly, this study also found that this model applied too much weight to clinical factors, such that it could no longer rank a patient's embryos effectively. Therefore, a model including only MKs would be preferred; this was similarly associated with live birth but not miscarriage.

Comparing performance between clinics of an embryo evaluation algorithm based on time-lapse images and machine learning

PURPOSE

This article aims to assess how differences in maternal age distributions between IVF clinics affect the performance of an artificial intelligence model for embryo viability prediction and proposes a method to account for such differences.

METHODS

Using retrospectively collected data from 4805 fresh and frozen single blastocyst transfers of embryos incubated for 5 to 6 days, the discriminative performance was assessed based on fetal heartbeat outcomes. The data was collected from 4 clinics, and the discrimination was measured in terms of the area under ROC curves (AUC) for each clinic. To account for the different age distributions between clinics, a method for age-standardizing the AUCs was developed in which the clinic-specific AUCs were standardized using weights for each embryo according to the relative frequency of the maternal age in the relevant clinic compared to the age distribution in a common reference population.

RESULTS

There was substantial variation in the clinic-specific AUCs with estimates ranging from 0.58 to 0.69 before standardization. The age-standardization of the AUCs reduced the between-clinic variance by 16%. Most notably, three of the clinics had quite similar AUCs after standardization, while the last clinic had a markedly lower AUC both with and without standardization.

CONCLUSION

The method of using age-standardization of the AUCs that is proposed in this article mitigates some of the variability between clinics. This enables a comparison of clinic-specific AUCs where the difference in age distributions is accounted for.

External validation of a model for selecting day 3 embryos for transfer based upon deep learning and time-lapse imaging

RESEARCH QUESTION

Could objective embryo assessment using iDAScore Version 2.0 perform as well as conventional morphological assessment?

DESIGN

A retrospective cohort study of fresh day 3 embryo transfer cycles was conducted at a large reproductive medicine centre. In total, 7786 embryos from 4328 cycles with known implantation data were cultured in a time-lapse incubator and included in the study. Fetal heartbeat (FHB) rate was analysed retrospectively using iDAScore Version 2.0 and conventional morphological assessment associated with the transferred embryos. The pregnancy-prediction performance of the two assessment methods was compared using area under the curve (AUC) values for predicting FHB.

RESULTS

AUC values were significantly higher for iDAScore compared with morphological assessment for all cycles (0.62 versus 0.60; P = 0.005), single-embryo transfer cycles (0.63 versus 0.60; P = 0.043) and double-embryo transfer cycles (0.61 versus 0.59; P = 0.012). For the age subgroups, AUC values were significantly higher for iDAScore compared with morphological assessment in the <35 years subgroup (0.62 versus 0.60; P = 0.009); however, no significant difference was found in the ≥35 years subgroup. In terms of the number of blastomeres, AUC values were significantly higher for iDAScore compared with morphological assessment for both the <8c subgroup (0.67 versus 0.56; P < 0.001) and the ≥8c subgroup (0.58 versus 0.55; P = 0.012).

CONCLUSIONS

iDAScore Version 2.0 performed as well as, or better than, conventional morphological assessment in fresh day 3 embryo transfer cycles. iDAScore Version 2.0 may therefore constitute a promising tool for selecting embryos with the highest likelihood of implantation.

How AI can learn from the law: putting humans in the loop only on appeal

While the literature on putting a "human in the loop" in artificial intelligence (AI) and machine learning (ML) has grown significantly, limited attention has been paid to how human expertise ought to be combined with AI/ML judgments. This design question arises because of the ubiquity and quantity of algorithmic decisions being made today in the face of widespread public reluctance to forgo human expert judgment. To resolve this conflict, we propose that human expert judges be included via appeals processes for review of algorithmic decisions. Thus, the human intervenes only in a limited number of cases and only after an initial AI/ML judgment has been made. Based on an analogy with appellate processes in judiciary decision-making, we argue that this is, in many respects, a more efficient way to divide the labor between a human and a machine. Human reviewers can add more nuanced clinical, moral, or legal reasoning, and they can consider case-specific information that is not easily quantified and, as such, not available to the AI/ML at an initial stage. In doing so, the human can serve as a crucial error correction check on the AI/ML, while retaining much of the efficiency of AI/ML's use in the decision-making process. In this paper, we develop these widely applicable arguments while focusing primarily on examples from the use of AI/ML in medicine, including organ allocation, fertility care, and hospital readmission.

Embryo selection through artificial intelligence versus embryologists: a systematic review

STUDY QUESTION

What is the present performance of artificial intelligence (AI) decision support during embryo selection compared to the standard embryo selection by embryologists?

SUMMARY ANSWER

AI consistently outperformed the clinical teams in all the studies focused on embryo morphology and clinical outcome prediction during embryo selection assessment.

WHAT IS KNOWN ALREADY

The ART success rate is ∼30%, with a worrying trend of increasing female age correlating with considerably worse results. As such, there have been ongoing efforts to address this low success rate through the development of new technologies. With the advent of AI, there is potential for machine learning to be applied in such a manner that areas limited by human subjectivity, such as embryo selection, can be enhanced through increased objectivity. Given the potential of AI to improve IVF success rates, it remains crucial to review the performance between AI and embryologists during embryo selection.

STUDY DESIGN SIZE DURATION

The search was done across PubMed, EMBASE, Ovid Medline, and IEEE Xplore from 1 June 2005 up to and including 7 January 2022. Included articles were also restricted to those written in English. Search terms utilized across all databases for the study were: ('Artificial intelligence' OR 'Machine Learning' OR 'Deep learning' OR 'Neural network') AND ('IVF' OR 'in vitro fertili*' OR 'assisted reproductive techn*' OR 'embryo'), where the character '*' refers the search engine to include any auto completion of the search term.

PARTICIPANTS/MATERIALS SETTING METHODS

A literature search was conducted for literature relating to AI applications to IVF. Primary outcomes of interest were accuracy, sensitivity, and specificity of the embryo morphology grade assessments and the likelihood of clinical outcomes, such as clinical pregnancy after IVF treatments. Risk of bias was assessed using the Modified Down and Black Checklist.

MAIN RESULTS AND THE ROLE OF CHANCE

Twenty articles were included in this review. There was no specific embryo assessment day across the studies-Day 1 until Day 5/6 of embryo development was investigated. The types of input for training AI algorithms were images and time-lapse (10/20), clinical information (6/20), and both images and clinical information (4/20). Each AI model demonstrated promise when compared to an embryologist's visual assessment. On average, the models predicted the likelihood of successful clinical pregnancy with greater accuracy than clinical embryologists, signifying greater reliability when compared to human prediction. The AI models performed at a median accuracy of 75.5% (range 59-94%) on predicting embryo morphology grade. The correct prediction (Ground Truth) was defined through the use of embryo images according to post embryologists' assessment following local respective guidelines. Using blind test datasets, the embryologists' accuracy prediction was 65.4% (range 47-75%) with the same ground truth provided by the original local respective assessment. Similarly, AI models had a median accuracy of 77.8% (range 68-90%) in predicting clinical pregnancy through the use of patient clinical treatment information compared to 64% (range 58-76%) when performed by embryologists. When both images/time-lapse and clinical information inputs were combined, the median accuracy by the AI models was higher at 81.5% (range 67-98%), while clinical embryologists had a median accuracy of 51% (range 43-59%).

LIMITATIONS REASONS FOR CAUTION

The findings of this review are based on studies that have not been prospectively evaluated in a clinical setting. Additionally, a fair comparison of all the studies were deemed unfeasible owing to the heterogeneity of the studies, development of the AI models, database employed and the study design and quality.

WIDER IMPLICATIONS OF THE FINDINGS

AI provides considerable promise to the IVF field and embryo selection. However, there needs to be a shift in developers' perception of the clinical outcome from successful implantation towards ongoing pregnancy or live birth. Additionally, existing models focus on locally generated databases and many lack external validation.

STUDY FUNDING/COMPETING INTERESTS

This study was funded by Monash Data Future Institute. All authors have no conflicts of interest to declare.

REGISTRATION NUMBER

CRD42021256333.

In Contemporary Reproductive Medicine Human Beings are Not Yet Dispensable

In the past few years almost every aspect of an IVF cycle has been investigated, including research on sperm, color doppler in follicular studies, prediction of embryo cleavage, prediction of blastocyst formation, scoring blastocyst quality, prediction of euploid blastocysts and live birth from blastocysts, improving the embryo selection process, and for developing deep machine learning (ML) algorithms for optimal IVF stimulation protocols. Also, artificial intelligence (AI)-based methods have been implemented for some clinical aspects of IVF, such as assessing patient reproductive potential and individualizing gonadotropin stimulation protocols. As AI has the inherent capacity to analyze "Big" data, the goal will be to apply AI tools to the analysis of all embryological, clinical, and genetic data to provide patient-tailored individualized treatments. Human skillsets including hand eye coordination to perform an embryo transfer is probably the only step of IVF that is outside the realm of AI & ML today. Embryo transfer success is presently human skill dependent and deep machine learning may one day intrude into this sacred space with the advent of programed humanoid robots. Embryo transfer is arguably the rate limiting step in the sequential events that complete an IVF cycle. Many variables play a role in the success of embryo transfer, including catheter type, atraumatic technique, and the use of sonography guidance before and during the procedure of embryo transfer. In contemporary Reproductive Medicine human beings are not yet dispensable.

[Interest of iDAScore (intelligent Data Analysis Score) for embryo selection in routine IVF laboratory practice: Results of a preliminary study]

INTRODUCTION

Embryo selection is a major challenge in ART, especially since the generalization of single embryo transfer, and its optimization could lead to the improvement of clinical results in IVF. Recently, several Artificial Intelligence (AI) models, based on deep-learning such as iDAScore, have been developed. These models, trained on time-lapse videos of embryos with known implantation data, can predict the probability of pregnancy for a given embryo, allowing automatization and standardization in embryo selection.

MATERIAL AND METHODS

In this study, we have compared the hierarchical categorization of 311 D5 blastocysts of iDAScore v1.0 and the embryologists of our unit. These 311 D5 blastocysts have been classified as top (70.1%), good (Q+: 10.6%) and poor (Q-: 19.3%) quality by embryologists according to Gardner classification. Median iDAScores were [9.9-8.4],]8.4-7.5] and]7.5-2.1] for top, good and poor-quality blastocysts respectively.

RESULTS

We observed a significantly concordant categorization between iDAScore and embryologists for top, good and poor-quality blastocysts (respectively, 89.5, 36.4 and 48.3%, P < 10-4). Moreover, the hierarchical categorization of the three best blastocysts between iDAScore and the embryologists was as follow: 1st rank: 71.9%; 2nd rank: 61.6%; 3rd rank: 56.8% (P=0.07). One hundred and fifty-one blastocysts with known implantation data were analyzed. The iDAScore of blastocysts that implanted was significantly higher than those that did not implant (implantation+: 9.10±0.57; implantation-: 8.70±0.95, P=0.003).

CONCLUSION

This preliminary study shows that iDAScore is able to perform a reproducible, reliable and immediate hierarchical classification of blastocysts. Moreover, this tool can identify the blastocysts with the highest implantation potential. If these results confirmed on a larger scale of embryos and patients, IA could revolutionize IVF laboratories by standardizing embryo hierarchical selection.

ESHRE good practice recommendations on recurrent implantation failure

STUDY QUESTION

How should recurrent implantation failure (RIF) in patients undergoing ART be defined and managed?

SUMMARY ANSWER

This is the first ESHRE good practice recommendations paper providing a definition for RIF together with recommendations on how to investigate causes and contributing factors, and how to improve the chances of a pregnancy.

WHAT IS KNOWN ALREADY

RIF is a challenge in the ART clinic, with a multitude of investigations and interventions offered and applied in clinical practice, often without biological rationale or with unequivocal evidence of benefit.

STUDY DESIGN SIZE DURATION

This document was developed according to a predefined methodology for ESHRE good practice recommendations. Recommendations are supported by data from the literature, if available, and the results of a previously published survey on clinical practice in RIF and the expertise of the working group. A literature search was performed in PubMed and Cochrane focussing on 'recurrent reproductive failure', 'recurrent implantation failure', and 'repeated implantation failure'.

PARTICIPANTS/MATERIALS SETTING METHODS

The ESHRE Working Group on Recurrent Implantation Failure included eight members representing the ESHRE Special Interest Groups for Implantation and Early Pregnancy, Reproductive Endocrinology, and Embryology, with an independent chair and an expert in statistics. The recommendations for clinical practice were formulated based on the expert opinion of the working group, while taking into consideration the published data and results of the survey on uptake in clinical practice. The draft document was then open to ESHRE members for online peer review and was revised in light of the comments received.

MAIN RESULTS AND THE ROLE OF CHANCE

The working group recommends considering RIF as a secondary phenomenon of ART, as it can only be observed in patients undergoing IVF, and that the following description of RIF be adopted: 'RIF describes the scenario in which the transfer of embryos considered to be viable has failed to result in a positive pregnancy test sufficiently often in a specific patient to warrant consideration of further investigations and/or interventions'. It was agreed that the recommended threshold for the cumulative predicted chance of implantation to identify RIF for the purposes of initiating further investigation is 60%. When a couple have not had a successful implantation by a certain number of embryo transfers and the cumulative predicted chance of implantation associated with that number is greater than 60%, then they should be counselled on further investigation and/or treatment options. This term defines clinical RIF for which further actions should be considered. Nineteen recommendations were formulated on investigations when RIF is suspected, and 13 on interventions. Recommendations were colour-coded based on whether the investigations/interventions were recommended (green), to be considered (orange), or not recommended, i.e. not to be offered routinely (red).

LIMITATIONS REASONS FOR CAUTION

While awaiting the results of further studies and trials, the ESHRE Working Group on Recurrent Implantation Failure recommends identifying RIF based on the chance of successful implantation for the individual patient or couple and to restrict investigations and treatments to those supported by a clear rationale and data indicating their likely benefit.

WIDER IMPLICATIONS OF THE FINDINGS

This article provides not only good practice advice but also highlights the investigations and interventions that need further research. This research, when well-conducted, will be key to making progress in the clinical management of RIF.

STUDY FUNDING/COMPETING INTERESTS

The meetings and technical support for this project were funded by ESHRE. N.M. declared consulting fees from ArtPRED (The Netherlands) and Freya Biosciences (Denmark); Honoraria for lectures from Gedeon Richter, Merck, Abbott, and IBSA; being co-founder of Verso Biosense. He is Co-Chief Editor of Reproductive Biomedicine Online (RBMO). D.C. declared being an Associate Editor of Human Reproduction Update, and declared honoraria for lectures from Merck, Organon, IBSA, and Fairtility; support for attending meetings from Cooper Surgical, Fujifilm Irvine Scientific. G.G. declared that he or his institution received financial or non-financial support for research, lectures, workshops, advisory roles, or travelling from Ferring, Merck, Gedeon-Richter, PregLem, Abbott, Vifor, Organon, MSD, Coopersurgical, ObsEVA, and ReprodWissen. He is an Editor of the journals Archives of Obstetrics and Gynecology and Reproductive Biomedicine Online, and Editor in Chief of Journal Gynäkologische Endokrinologie. He is involved in guideline developments and quality control on national and international level. G.L. declared he or his institution received honoraria for lectures from Merck, Ferring, Vianex/Organon, and MSD. He is an Associate Editor of Human Reproduction Update, immediate past Coordinator of Special Interest Group for Reproductive Endocrinology of ESHRE and has been involved in Guideline Development Groups of ESHRE and national fertility authorities. D.J.M. declared being an Associate Editor for Human Reproduction Open and statistical Advisor for Reproductive Biomedicine Online. B.T. declared being shareholder of Reprognostics and she or her institution received financial or non-financial support for research, clinical trials, lectures, workshops, advisory roles or travelling from support for attending meetings from Ferring, MSD, Exeltis, Merck Serono, Bayer, Teva, Theramex and Novartis, Astropharm, Ferring. The other authors had nothing to disclose.

DISCLAIMER

This Good Practice Recommendations (GPR) document represents the views of ESHRE, which are the result of consensus between the relevant ESHRE stakeholders and are based on the scientific evidence available at the time of preparation. ESHRE GPRs should be used for information and educational purposes. They should not be interpreted as setting a standard of care or be deemed inclusive of all proper methods of care, or be exclusive of other methods of care reasonably directed to obtaining the same results. They do not replace the need for application of clinical judgement to each individual presentation, or variations based on locality and facility type. Furthermore, ESHRE GPRs do not constitute or imply the endorsement, or favouring, of any of the included technologies by ESHRE.

Lineage segregation in human pre-implantation embryos is specified by YAP1 and TEAD1

STUDY QUESTION

Which processes and transcription factors specify the first and second lineage segregation events during human preimplantation development?

SUMMARY ANSWER

Differentiation into trophectoderm (TE) cells can be initiated independently of polarity; moreover, TEAD1 and YAP1 co-localize in (precursor) TE and primitive endoderm (PrE) cells, suggesting a role in both the first and the second lineage segregation events.

WHAT IS KNOWN ALREADY

We know that polarity, YAP1/GATA3 signalling and phospholipase C signalling play a key role in TE initiation in compacted human embryos, however, little is known about the TEAD family of transcription factors that become activated by YAP1 and, especially, whether they play a role during epiblast (EPI) and PrE formation. In mouse embryos, polarized outer cells show nuclear TEAD4/YAP1 activity that upregulates Cdx2 and Gata3 expression while inner cells exclude YAP1 which upregulates Sox2 expression. The second lineage segregation event in mouse embryos is orchestrated by FGF4/FGFR2 signalling which could not be confirmed in human embryos; TEAD1/YAP1 signalling also plays a role during the establishment of mouse EPI cells.

STUDY DESIGN, SIZE, DURATION

Based on morphology, we set up a development timeline of 188 human preimplantation embryos between Day 4 and 6 post-fertilization (dpf). The compaction process was divided into three subgroups: embryos at the start (C0), during (C1), and at the end (C2) of, compaction. Inner cells were identified as cells that were entirely separated from the perivitelline space and enclosed by cellular contacts on all sides. The blastulation process was divided into six subgroups, starting with early blastocysts with sickle-cell shaped outer cells (B0) and further on, blastocysts with a cavity (B1). Full blastocysts (B2) showed a visible ICM and outer cells referred to as TE. Further expanded blastocysts (B3) had accumulated fluid and started to expand due to TE cell proliferation and zona pellucida (ZP) thinning. The blastocysts then significantly expanded further (B4) and started to hatch out of the ZP (B5) until they were fully hatched (B6).

PARTICIPANTS/MATERIALS, SETTING, METHODS

After informed consent and the expiration of the 5-year cryopreservation duration, 188 vitrified high quality eight-cell stage human embryos (3 dpf) were warmed and cultured until the required stages were reached. We also cultured 14 embryos that were created for research until the four- and eight-cell stage. The embryos were scored according to their developmental stage (C0-B6) displaying morphological key differences, rather than defining them according to their chronological age. They were fixed and immunostained for different combinations of cytoskeleton (F-actin), polarization (p-ERM), TE (GATA3), EPI (NANOG), PrE (GATA4 and SOX17), and members of the Hippo signalling pathway (YAP1, TEAD1 and TEAD4). We choose these markers based on previous observations in mouse embryos and single cell RNA-sequencing data of human embryos. After confocal imaging (LSM800, Zeiss), we analysed cell numbers within each lineage, different co-localization patterns and nuclear enrichment.

MAIN RESULTS AND THE ROLE OF CHANCE

We found that in human preimplantation embryos compaction is a heterogeneous process that takes place between the eight-cell to the 16-cell stages. Inner and outer cells are established at the end of the compaction process (C2) when the embryos contain up to six inner cells. Full apical p-ERM polarity is present in all outer cells of compacted C2 embryos. Co-localization of p-ERM and F-actin increases steadily from 42.2% to 100% of the outer cells, between C2 and B1 stages, while p-ERM polarizes before F-actin (P < 0.00001). Next, we sought to determine which factors specify the first lineage segregation event. We found that 19.5% of the nuclei stain positive for YAP1 at the start of compaction (C0) which increases to 56.1% during compaction (C1). At the C2 stage, 84.6% of polarized outer cells display high levels of nuclear YAP1 while it is absent in 75% of non-polarized inner cells. In general, throughout the B0-B3 blastocyst stages, polarized outer/TE cells are mainly positive for YAP1 and non-polarized inner/ICM cells are negative for YAP1. From the C1 stage onwards, before polarity is established, the TE marker GATA3 is detectable in YAP1 positive cells (11.6%), indicating that differentiation into TE cells can be initiated independently of polarity. Co-localization of YAP1 and GATA3 increases steadily in outer/TE cells (21.8% in C2 up to 97.3% in B3). Transcription factor TEAD4 is ubiquitously present throughout preimplantation development from the compacted stage onwards (C2-B6). TEAD1 displays a distinct pattern that coincides with YAP1/GATA3 co-localization in the outer cells. Most outer/TE cells throughout the B0-B3 blastocyst stages are positive for TEAD1 and YAP1. However, TEAD1 proteins are also detected in most nuclei of the inner/ICM cells of the blastocysts from cavitation onwards, but at visibly lower levels as compared to that in TE cells. In the ICM of B3 blastocysts, we found one main population of cells with NANOG+/SOX17-/GATA4- nuclei (89.1%), but exceptionally we found NANOG+/SOX17+/GATA4+ cells (0.8%). In seven out of nine B3 blastocysts, nuclear NANOG was found in all the ICM cells, supporting the previously reported hypothesis that PrE cells arise from EPI cells. Finally, to determine which factors specify the second lineage segregation event, we co-stained for TEAD1, YAP1, and GATA4. We identified two main ICM cell populations in B4-6 blastocysts: the EPI (negative for the three markers, 46.5%) and the PrE (positive for the three markers, 28.1%) cells. We conclude that TEAD1 and YAP1 co-localise in (precursor) TE and PrE cells, indicating that TEAD1/YAP1 signalling plays a role in the first and the second lineage segregation events.

LIMITATIONS, REASONS FOR CAUTION

In this descriptive study, we did not perform functional studies to investigate the role of TEAD1/YAP1 signalling during the first and second lineage segregation events.

WIDER IMPLICATIONS OF THE FINDINGS

Our detailed roadmap on polarization, compaction, position and lineage segregation events during human preimplantation development paves the way for further functional studies. Understanding the gene regulatory networks and signalling pathways involved in early embryogenesis could ultimately provide insights into why embryonic development is sometimes impaired and facilitate the establishment of guidelines for good practice in the IVF lab.

STUDY FUNDING/COMPETING INTERESTS

This work was financially supported by Wetenschappelijk Fonds Willy Gepts (WFWG) of the University Hospital UZ Brussel (WFWG142) and the Fonds Wetenschappelijk Onderzoek-Vlaanderen (FWO, G034514N). M.R. is doctoral fellow at the FWO. The authors have no conflicts of interest to declare.

TRIAL REGISTRATION NUMBER

N/A.

Impact of Endometrial Receptivity Analysis on Pregnancy Outcomes In Patients Undergoing Embryo Transfer: A Systematic Review and Meta-Analysis

To analyze the influence of endometrial receptivity analysis (ERA) on embryo transfer (ET) results in patients undergoing in vitro fertilization (IVF) treatment. PubMed, Embase, Cochrane Central Register of Controlled Trials, and BioMed Central databases were searched from inception up to December 2022 for studies comparing pregnancy outcomes in patients undergoing personalized embryo transfer (pET) by ERA versus standard ET. Data were pooled by meta-analysis using a random effects model. We identified twelve studies, including 14,224 patients. No differences were observed between patients undergoing ERA test and those not undergoing ERA test prior to ET in terms of live birth (OR 1.00, 95% CI 0.63-1.58, I2 = 92.7%), clinical pregnancy (OR 1.20, 95% CI 0.90-1.61, I2 = 86.5%), biochemical pregnancy (OR 0.83, 95% CI 0.46-1.49, I2 = 87%), positive pregnancy test (OR 0.99, 95% CI 0.80-1.22, I2 = 0%), miscarriage (OR 0.91, 95% CI 0.62-1.34, I2 = 67.1%), and implantation rate (OR 1.18, 95% CI 0.44-3.14, I2 = 93.2%). pET with ERA is not associated with any significant differences in pregnancy outcomes as compared to standard ET protocols. Therefore, the utility of ERA in patients undergoing IVF should be revisited.

Development and validation of deep learning based embryo selection across multiple days of transfer

This work describes the development and validation of a fully automated deep learning model, iDAScore v2.0, for the evaluation of human embryos incubated for 2, 3, and 5 or more days. We trained and evaluated the model on an extensive and diverse dataset including 181,428 embryos from 22 IVF clinics across the world. To discriminate the transferred embryos with known outcome, we show areas under the receiver operating curve ranging from 0.621 to 0.707 depending on the day of transfer. Predictive performance increased over time and showed a strong correlation with morphokinetic parameters. The model's performance is equivalent to the KIDScore D3 model on day 3 embryos while it significantly surpasses the performance of KIDScore D5 v3 on day 5+ embryos. This model provides an analysis of time-lapse sequences without the need for user input, and provides a reliable method for ranking embryos for their likelihood of implantation, at both cleavage and blastocyst stages. This greatly improves embryo grading consistency and saves time compared to traditional embryo evaluation methods.

Towards Automation in IVF: Pre-Clinical Validation of a Deep Learning-Based Embryo Grading System during PGT-A Cycles

Preimplantation genetic testing for aneuploidies (PGT-A) is arguably the most effective embryo selection strategy. Nevertheless, it requires greater workload, costs, and expertise. Therefore, a quest towards user-friendly, non-invasive strategies is ongoing. Although insufficient to replace PGT-A, embryo morphological evaluation is significantly associated with embryonic competence, but scarcely reproducible. Recently, artificial intelligence-powered analyses have been proposed to objectify and automate image evaluations. iDAScore v1.0 is a deep-learning model based on a 3D convolutional neural network trained on time-lapse videos from implanted and non-implanted blastocysts. It is a decision support system for ranking blastocysts without manual input. This retrospective, pre-clinical, external validation included 3604 blastocysts and 808 euploid transfers from 1232 cycles. All blastocysts were retrospectively assessed through the iDAScore v1.0; therefore, it did not influence embryologists' decision-making process. iDAScore v1.0 was significantly associated with embryo morphology and competence, although AUCs for euploidy and live-birth prediction were 0.60 and 0.66, respectively, which is rather comparable to embryologists' performance. Nevertheless, iDAScore v1.0 is objective and reproducible, while embryologists' evaluations are not. In a retrospective simulation, iDAScore v1.0 would have ranked euploid blastocysts as top quality in 63% of cases with one or more euploid and aneuploid blastocysts, and it would have questioned embryologists' ranking in 48% of cases with two or more euploid blastocysts and one or more live birth. Therefore, iDAScore v1.0 may objectify embryologists' evaluations, but randomized controlled trials are required to assess its clinical value.

A hybrid artificial intelligence model leverages multi-centric clinical data to improve fetal heart rate pregnancy prediction across time-lapse systems

STUDY QUESTION

Can artificial intelligence (AI) algorithms developed to assist embryologists in evaluating embryo morphokinetics be enriched with multi-centric clinical data to better predict clinical pregnancy outcome?

SUMMARY ANSWER

Training algorithms on multi-centric clinical data significantly increased AUC compared to algorithms that only analyzed the time-lapse system (TLS) videos.

WHAT IS KNOWN ALREADY

Several AI-based algorithms have been developed to predict pregnancy, most of them based only on analysis of the time-lapse recording of embryo development. It remains unclear, however, whether considering numerous clinical features can improve the predictive performances of time-lapse based embryo evaluation.

STUDY DESIGN, SIZE, DURATION

A dataset of 9986 embryos (95.60% known clinical pregnancy outcome, 32.47% frozen transfers) from 5226 patients from 14 European fertility centers (in two countries) recorded with three different TLS was used to train and validate the algorithms. A total of 31 clinical factors were collected. A separate test set (447 videos) was used to compare performances between embryologists and the algorithm.

PARTICIPANTS/MATERIALS, SETTING, METHODS

Clinical pregnancy (defined as a pregnancy leading to a fetal heartbeat) outcome was first predicted using a 3D convolutional neural network that analyzed videos of the embryonic development up to 2 or 3 days of development (33% of the database) or up to 5 or 6 days of development (67% of the database). The output video score was then fed as input alongside clinical features to a gradient boosting algorithm that generated a second score corresponding to the hybrid model. AUC was computed across 7-fold of the validation dataset for both models. These predictions were compared to those of 13 senior embryologists made on the test dataset.

MAIN RESULTS AND THE ROLE OF CHANCE

The average AUC of the hybrid model across all 7-fold was significantly higher than that of the video model (0.727 versus 0.684, respectively, P = 0.015; Wilcoxon test). A SHapley Additive exPlanations (SHAP) analysis of the hybrid model showed that the six first most important features to predict pregnancy were morphokinetics of the embryo (video score), oocyte age, total gonadotrophin dose intake, number of embryos generated, number of oocytes retrieved, and endometrium thickness. The hybrid model was shown to be superior to embryologists with respect to different metrics, including the balanced accuracy (P ≤ 0.003; Wilcoxon test). The likelihood of pregnancy was linearly linked to the hybrid score, with increasing odds ratio (maximum P-value = 0.001), demonstrating the ranking capacity of the model. Training individual hybrid models did not improve predictive performance. A clinic hold-out experiment was conducted and resulted in AUCs ranging between 0.63 and 0.73. Performance of the hybrid model did not vary between TLS or between subgroups of embryos transferred at different days of embryonic development. The hybrid model did fare better for patients older than 35 years (P < 0.001; Mann-Whitney test), and for fresh transfers (P < 0.001; Mann-Whitney test).

LIMITATIONS, REASONS FOR CAUTION

Participant centers were located in two countries, thus limiting the generalization of our conclusion to wider subpopulations of patients. Not all clinical features were available for all embryos, thus limiting the performances of the hybrid model in some instances.

WIDER IMPLICATIONS OF THE FINDINGS

Our study suggests that considering clinical data improves pregnancy predictive performances and that there is no need to retrain algorithms at the clinic level unless they follow strikingly different practices. This study characterizes a versatile AI algorithm with similar performance on different time-lapse microscopes and on embryos transferred at different development stages. It can also help with patients of different ages and protocols used but with varying performances, presumably because the task of predicting fetal heartbeat becomes more or less hard depending on the clinical context. This AI model can be made widely available and can help embryologists in a wide range of clinical scenarios to standardize their practices.

STUDY FUNDING/COMPETING INTEREST(S)

Funding for the study was provided by ImVitro with grant funding received in part from BPIFrance (Bourse French Tech Emergence (DOS0106572/00), Paris Innovation Amorçage (DOS0132841/00), and Aide au Développement DeepTech (DOS0152872/00)). A.B.-C. is a co-owner of, and holds stocks in, ImVitro SAS. A.B.-C. and F.D.M. hold a patent for 'Devices and processes for machine learning prediction of in vitro fertilization' (EP20305914.2). A.D., N.D., M.M.F., and F.D.M. are or have been employees of ImVitro and have been granted stock options. X.P.-V. has been paid as a consultant to ImVitro and has been granted stocks options of ImVitro. L.C.-D. and C.G.-S. have undertaken paid consultancy for ImVitro SAS. The remaining authors have no conflicts to declare.

TRIAL REGISTRATION NUMBER

N/A.

The Future Is Coming: Artificial Intelligence in the Treatment of Infertility Could Improve Assisted Reproduction Outcomes-The Value of Regulatory Frameworks

Infertility is a global health issue affecting women and men of reproductive age with increasing incidence worldwide, in part due to greater awareness and better diagnosis. Assisted reproduction technologies (ART) are considered the ultimate step in the treatment of infertility. Recently, artificial intelligence (AI) has been progressively used in the many fields of medicine, integrating knowledge and computer science through machine learning algorithms. AI has the potential to improve infertility diagnosis and ART outcomes estimated as pregnancy and/or live birth rate, especially with recurrent ART failure. A broad-ranging review has been conducted, focusing on clinical AI applications up until September 2022, which could be estimated in terms of possible applications, such as ultrasound monitoring of folliculogenesis, endometrial receptivity, embryo selection based on quality and viability, and prediction of post implantation embryo development, in order to eliminate potential contributing risk factors. Oocyte morphology assessment is highly relevant in terms of successful fertilization rate, as well as during oocyte freezing for fertility preservation, and substantially valuable in oocyte donation cycles. AI has great implications in the assessment of male infertility, with computerised semen analysis systems already in use and a broad spectrum of possible AI-based applications in environmental and lifestyle evaluation to predict semen quality. In addition, considerable progress has been made in terms of harnessing AI in cases of idiopathic infertility, to improve the stratification of infertile/fertile couples based on their biological and clinical signatures. With AI as a very powerful tool of the future, our review is meant to summarise current AI applications and investigations in contemporary reproduction medicine, mainly focusing on the nonsurgical aspects of it; in addition, the authors have briefly explored the frames of reference and guiding principles for the definition and implementation of legal, regulatory, and ethical standards for AI in healthcare.

Morphology of inner cell mass: a better predictive biomarker of blastocyst viability

Background

Transfer of embryos at the blastocyst stage is one of the best approaches for achieving a higher success rate in In vitro fertilization (IVF) treatment as it demonstrates an improved uterine and embryonic synchrony at implantation. Despite novel biochemical and genetic markers proposed for the prediction of embryo viability in recent years, the conventional morphological grading of blastocysts remains the classical way of selection in routine practice. This study aims to investigate the association between the morphological features of blastocysts and pregnancy outcomes.

Methods

This prospective study included women undergoing single or double frozen blastocyst transfers following their autologous cycles in a period between October 2020 and September 2021. The morphological grades (A-good, B-average, and C-poor) of inner cell mass (ICM) and trophectoderm (TE) of blastocysts with known implantation were compared to assess their predictive potential of pregnancy outcome. It was further explored by measuring the relationship between the two variables using logistic regression and receiver operating characteristic (ROC) analysis.

Results

A total of 1,972 women underwent frozen embryo transfer (FET) cycles with a total of 3,786 blastocysts. Known implantation data (KID) from 2,060 blastocysts of 1,153 patients were subjected to statistical analysis, the rest were excluded. Implantation rates (IR) from transfer of ICM/TE grades AA, AB, BA, BB were observed as 48.5%, 39.4%, 23.4% and 25% respectively. There was a significantly higher IR observed in blastocysts with ICM grade A (p < 0.001) than those with B irrespective of their TE scores. The analysis of the interaction between the two characteristics confirmed the superiority of ICM over TE as a predictor of the outcome. The rank biserial correlation value for ICM was also greater compared to that of TE (0.11 vs 0.05).

Conclusion

This study confirms that the morphology of ICM of the blastocyst is a stronger predictor of implantation and clinical pregnancy than that of TE and can be utilized as a biomarker of viability.

Embryologist agreement when assessing blastocyst implantation probability: is data-driven prediction the solution to embryo assessment subjectivity?

STUDY QUESTION

What is the accuracy and agreement of embryologists when assessing the implantation probability of blastocysts using time-lapse imaging (TLI), and can it be improved with a data-driven algorithm?

SUMMARY ANSWER

The overall interobserver agreement of a large panel of embryologists was moderate and prediction accuracy was modest, while the purpose-built artificial intelligence model generally resulted in higher performance metrics.

WHAT IS KNOWN ALREADY

Previous studies have demonstrated significant interobserver variability amongst embryologists when assessing embryo quality. However, data concerning embryologists' ability to predict implantation probability using TLI is still lacking. Emerging technologies based on data-driven tools have shown great promise for improving embryo selection and predicting clinical outcomes.

STUDY DESIGN, SIZE, DURATION

TLI video files of 136 embryos with known implantation data were retrospectively collected from two clinical sites between 2018 and 2019 for the performance assessment of 36 embryologists and comparison with a deep neural network (DNN).

PARTICIPANTS/MATERIALS, SETTING, METHODS

We recruited 39 embryologists from 13 different countries. All participants were blinded to clinical outcomes. A total of 136 TLI videos of embryos that reached the blastocyst stage were used for this experiment. Each embryo's likelihood of successfully implanting was assessed by 36 embryologists, providing implantation probability grades (IPGs) from 1 to 5, where 1 indicates a very low likelihood of implantation and 5 indicates a very high likelihood. Subsequently, three embryologists with over 5 years of experience provided Gardner scores. All 136 blastocysts were categorized into three quality groups based on their Gardner scores. Embryologist predictions were then converted into predictions of implantation (IPG ≥ 3) and no implantation (IPG ≤ 2). Embryologists' performance and agreement were assessed using Fleiss kappa coefficient. A 10-fold cross-validation DNN was developed to provide IPGs for TLI video files. The model's performance was compared to that of the embryologists.

MAIN RESULTS AND THE ROLE OF CHANCE

Logistic regression was employed for the following confounding variables: country of residence, academic level, embryo scoring system, log years of experience and experience using TLI. None were found to have a statistically significant impact on embryologist performance at α = 0.05. The average implantation prediction accuracy for the embryologists was 51.9% for all embryos (N = 136). The average accuracy of the embryologists when assessing top quality and poor quality embryos (according to the Gardner score categorizations) was 57.5% and 57.4%, respectively, and 44.6% for fair quality embryos. Overall interobserver agreement was moderate (κ = 0.56, N = 136). The best agreement was achieved in the poor + top quality group (κ = 0.65, N = 77), while the agreement in the fair quality group was lower (κ = 0.25, N = 59). The DNN showed an overall accuracy rate of 62.5%, with accuracies of 62.2%, 61% and 65.6% for the poor, fair and top quality groups, respectively. The AUC for the DNN was higher than that of the embryologists overall (0.70 DNN vs 0.61 embryologists) as well as in all of the Gardner groups (DNN vs embryologists-Poor: 0.69 vs 0.62; Fair: 0.67 vs 0.53; Top: 0.77 vs 0.54).

LIMITATIONS, REASONS FOR CAUTION

Blastocyst assessment was performed using video files acquired from time-lapse incubators, where each video contained data from a single focal plane. Clinical data regarding the underlying cause of infertility and endometrial thickness before the transfer was not available, yet may explain implantation failure and lower accuracy of IPGs. Implantation was defined as the presence of a gestational sac, whereas the detection of fetal heartbeat is a more robust marker of embryo viability. The raw data were anonymized to the extent that it was not possible to quantify the number of unique patients and cycles included in the study, potentially masking the effect of bias from a limited patient pool. Furthermore, the lack of demographic data makes it difficult to draw conclusions on how representative the dataset was of the wider population. Finally, embryologists were required to assess the implantation potential, not embryo quality. Although this is not the traditional approach to embryo evaluation, morphology/morphokinetics as a means of assessing embryo quality is believed to be strongly correlated with viability and, for some methods, implantation potential.

WIDER IMPLICATIONS OF THE FINDINGS

Embryo selection is a key element in IVF success and continues to be a challenge. Improving the predictive ability could assist in optimizing implantation success rates and other clinical outcomes and could minimize the financial and emotional burden on the patient. This study demonstrates moderate agreement rates between embryologists, likely due to the subjective nature of embryo assessment. In particular, we found that average embryologist accuracy and agreement were significantly lower for fair quality embryos when compared with that for top and poor quality embryos. Using data-driven algorithms as an assistive tool may help IVF professionals increase success rates and promote much needed standardization in the IVF clinic. Our results indicate a need for further research regarding technological advancement in this field.

STUDY FUNDING/COMPETING INTEREST(S)

Embryonics Ltd is an Israel-based company. Funding for the study was partially provided by the Israeli Innovation Authority, grant #74556.

TRIAL REGISTRATION NUMBER

N/A.

Human blastocyst spontaneous collapse is associated with worse morphological quality and higher degeneration and aneuploidy rates: a comprehensive analysis standardized through artificial intelligence

STUDY QUESTION

What are the factors associated with human blastocyst spontaneous collapse and the consequences of this event?

SUMMARY ANSWER

Approximately 50% of blastocysts collapsed, especially when non-viable, morphologically poor and/or aneuploid.

WHAT IS KNOWN ALREADY

Time-lapse microscopy (TLM) is a powerful tool to observe preimplantation development dynamics. Lately, artificial intelligence (AI) has been harnessed to automate and standardize such observations. Here, we adopted AI to comprehensively portray blastocyst spontaneous collapse, namely the phenomenon of reduction in size of the embryo accompanied by efflux of blastocoel fluid and the detachment of the trophectoderm (TE) from the zona pellucida (ZP). Although the underlying causes are unknown, blastocyst spontaneous collapse deserves attention as a possible marker of reduced competence.

STUDY DESIGN, SIZE, DURATION

An observational study was carried out, including 2348 TLM videos recorded during preimplantation genetic testing for aneuploidies (PGT-A, n = 720) cycles performed between January 2013 and December 2020. All embryos in the analysis at least reached the time of starting blastulation (tSB), 1943 of them reached full expansion, and were biopsied and then vitrified.

PARTICIPANTS/MATERIALS, SETTING, METHODS

ICSI, blastocyst culture, TE biopsy without Day 3 ZP drilling, comprehensive chromosome testing and vitrification were performed. The AI software automatically registered tSB and time of expanding blastocyst (tEB), start and end time of each collapse, time between consecutive collapses, embryo proper area, percentage of shrinkage, embryo:ZP ratio at embryo collapse, time of biopsy (t-biopsy) and related area of the fully (re-)expanded blastocyst before biopsy, time between the last collapse and biopsy. Blastocyst morphological quality was defined according to both Gardner's criteria and an AI-generated implantation score. Euploidy rate per biopsied blastocyst and live birth rate (LBR) per euploid single embryo transfer (SET) were the main outcomes. All significant associations were confirmed through regression analyses. All couple, cycle and embryo main features were also investigated for possible associations with blastocyst spontaneous collapse.

MAIN RESULTS AND THE ROLE OF CHANCE

At least one collapsing embryo (either viable or subsequently undergoing degeneration) was recorded in 559 cycles (77.6%) and in 498 cycles (69.2%) if considering only viable blastocysts. The prevalence of blastocyst spontaneous collapse after the tSB, but before the achievement of full expansion, was 50% (N = 1168/2348), irrespective of cycle and/or couple characteristics. Blastocyst degeneration was 13% among non-collapsing embryos, while it was 18%, 20%, 26% and 39% among embryos collapsing once, twice, three times or ≥4 times, respectively. The results showed that 47.3% (N = 918/1943) of the viable blastocysts experienced at least one spontaneous collapse (ranging from 1 up to 9). Although starting from similar tSB, the number of spontaneous collapses was associated with a delay in both tEB and time of biopsy. Of note, the worse the quality of a blastocyst, the more and the longer its spontaneous collapses. Blastocyst spontaneous collapse was significantly associated with lower euploidy rates (47% in non-collapsing and 38%, 32%, 31% and 20% in blastocysts collapsing once, twice, three times or ≥4 times, respectively; multivariate odds ratio 0.78, 95%CI 0.62-0.98, adjusted P = 0.03). The difference in the LBR after euploid vitrified-warmed SET was not significant (46% and 39% in non-collapsing and collapsing blastocysts, respectively).

LIMITATIONS, REASONS FOR CAUTION

An association between chromosomal mosaicism and blastocyst collapse cannot be reliably assessed on a single TE biopsy. Gestational and perinatal outcomes were not evaluated. Other culture strategies and media should be tested for their association with blastocyst spontaneous collapse. Future studies with a larger sample size are needed to investigate putative impacts on clinical outcomes after euploid transfers.

WIDER IMPLICATIONS OF THE FINDINGS

These results demonstrate the synergistic power of TLM and AI to increase the throughput of embryo preimplantation development observation. They also highlight the transition from compaction to full blastocyst as a delicate morphogenetic process. Blastocyst spontaneous collapse is common and associates with inherently lower competence, but additional data are required to deepen our knowledge on its causes and consequences.

STUDY FUNDING/COMPETING INTEREST(S)

There is no external funding to report. I.E., A.B.-M., I.H.-V. and B.K. are Fairtility employees. I.E. and B.K. also have stock or stock options of Fairtility.

TRIAL REGISTRATION NUMBER

N/A.

Correlation between an annotation-free embryo scoring system based on deep learning and live birth/neonatal outcomes after single vitrified-warmed blastocyst transfer: a single-centre, large-cohort retrospective study

Propose

Does an annotation-free embryo scoring system based on deep learning and time-lapse sequence images correlate with live birth (LB) and neonatal outcomes?

Methods

Patients who underwent SVBT cycles (3010 cycles, mean age: 39.3 ± 4.0). Scores were calculated using the iDAScore software module in the Vitrolife Technology Hub (Vitrolife, Gothenburg, Sweden). The correlation between iDAScore, LB rates, and total miscarriage (TM), including 1st- and 2nd-trimester miscarriage, was analysed using a trend test and multivariable logistic regression analysis. Furthermore, the correlation between the iDAScore and neonatal outcomes was analysed.

Results

LB rates decreased as iDAScore decreased (P < 0.05), and a similar inverse trend was observed for the TM rates. Additionally, multivariate logistic regression analysis showed that iDAScore significantly correlated with increased LB (adjusted odds ratio: 1.811, 95% CI: 1.666–1.976, P < 0.05) and decreased TM (adjusted odds ratio: 0.799, 95% CI: 0.706–0.905, P < 0.05). There was no significant correlation between iDAScore and neonatal outcomes, including congenital malformations, sex, gestational age, and birth weight. Multivariate logistic regression analysis, which included maternal and paternal age, maternal body mass index, parity, smoking, and presence or absence of caesarean section as confounding factors, revealed no significant difference in any neonatal characteristics.

Conclusion

Automatic embryo scoring using iDAScore correlates with decreased miscarriage and increased LB and has no correlation with neonatal outcomes.

Supplementary information

The online version contains supplementary material available at 10.1007/s10815-022-02562-5.

Non-Invasive Staging of In Vitro Mice Embryos by Means of Volatolomics

Current methods for embryo selection are limited. This study assessed a novel method for the prediction of embryo developmental potential based on the analysis of volatile organic compounds (VOCs) emitted by embryo samples. The study included mice embryos monitored during the pre-implantation period. Four developmental stages of the embryos were tested, covering the period from 1 to 4 days after fecundation. In each stage, the VOCs released by the embryos were collected and examined by employing two different volatolomic techniques, gas-chromatography coupled to mass-spectrometry (GC-MS) and a nanoarray of chemical gas sensors. The GC-MS study revealed that the VOC patterns emanating from embryo samples had statistically different values at different stages of embryo development. The sensor nanoarray was capable of classifying the developmental stages of the embryos. The proposed volatolomics analysis approach for embryos presents a promising potential for predicting their developmental stage. In combination with conventional morphokinetic parameters, it could be effective as a predictive model for detecting metabolic shifts that affect embryo quality before preimplantation.

Robust and generalizable embryo selection based on artificial intelligence and time-lapse image sequences

Assessing and selecting the most viable embryos for transfer is an essential part of in vitro fertilization (IVF). In recent years, several approaches have been made to improve and automate the procedure using artificial intelligence (AI) and deep learning. Based on images of embryos with known implantation data (KID), AI models have been trained to automatically score embryos related to their chance of achieving a successful implantation. However, as of now, only limited research has been conducted to evaluate how embryo selection models generalize to new clinics and how they perform in subgroup analyses across various conditions. In this paper, we investigate how a deep learning-based embryo selection model using only time-lapse image sequences performs across different patient ages and clinical conditions, and how it correlates with traditional morphokinetic parameters. The model was trained and evaluated based on a large dataset from 18 IVF centers consisting of 115,832 embryos, of which 14,644 embryos were transferred KID embryos. In an independent test set, the AI model sorted KID embryos with an area under the curve (AUC) of a receiver operating characteristic curve of 0.67 and all embryos with an AUC of 0.95. A clinic hold-out test showed that the model generalized to new clinics with an AUC range of 0.60–0.75 for KID embryos. Across different subgroups of age, insemination method, incubation time, and transfer protocol, the AUC ranged between 0.63 and 0.69. Furthermore, model predictions correlated positively with blastocyst grading and negatively with direct cleavages. The fully automated iDAScore v1.0 model was shown to perform at least as good as a state-of-the-art manual embryo selection model. Moreover, full automatization of embryo scoring implies fewer manual evaluations and eliminates biases due to inter- and intraobserver variation.