Simple adaptations to the Templeton model for IVF outcome prediction make it current and clinically useful

External validation of models for predicting cumulative live birth over multiple complete cycles of IVF treatment

STUDY QUESTION

Can two prediction models developed using data from 1999 to 2009 accurately predict the cumulative probability of live birth per woman over multiple complete cycles of IVF in an updated UK cohort?

SUMMARY ANSWER

After being updated, the models were able to estimate individualized chances of cumulative live birth over multiple complete cycles of IVF with greater accuracy.

WHAT IS KNOWN ALREADY

The McLernon models were the first to predict cumulative live birth over multiple complete cycles of IVF. They were converted into an online calculator called OPIS (Outcome Prediction In Subfertility) which has 3000 users per month on average. A previous study externally validated the McLernon models using a Dutch prospective cohort containing data from 2011 to 2014. With changes in IVF practice over time, it is important that the McLernon models are externally validated on a more recent cohort of patients to ensure that predictions remain accurate.

STUDY DESIGN, SIZE, DURATION

A population-based cohort of 91 035 women undergoing IVF in the UK between January 2010 and December 2016 was used for external validation. Data on frozen embryo transfers associated with these complete IVF cycles conducted from 1 January 2017 to 31 December 2017 were also collected.

PARTICIPANTS/MATERIALS, SETTING, METHODS

Data on IVF treatments were obtained from the Human Fertilisation and Embryology Authority (HFEA). The predictive performances of the McLernon models were evaluated in terms of discrimination and calibration. Discrimination was assessed using the c-statistic and calibration was assessed using calibration-in-the-large, calibration slope, and calibration plots. Where any model demonstrated poor calibration in the validation cohort, the models were updated using intercept recalibration, logistic recalibration, or model revision to improve model performance.

MAIN RESULTS AND THE ROLE OF CHANCE

Following exclusions, 91 035 women who underwent 144 734 complete cycles were included. The validation cohort had a similar distribution age profile to women in the development cohort. Live birth rates over all complete cycles of IVF per woman were higher in the validation cohort. After calibration assessment, both models required updating. The coefficients of the pre-treatment model were revised, and the updated model showed reasonable discrimination (c-statistic: 0.67, 95% CI: 0.66 to 0.68). After logistic recalibration, the post-treatment model showed good discrimination (c-statistic: 0.75, 95% CI: 0.74 to 0.76). As an example, in the updated pre-treatment model, a 32-year-old woman with 2 years of primary infertility has a 42% chance of having a live birth in the first complete ICSI cycle and a 77% chance over three complete cycles. In a couple with 2 years of primary male factor infertility where a 30-year-old woman has 15 oocytes collected in the first cycle, a single fresh blastocyst embryo transferred in the first cycle and spare embryos cryopreserved, the estimated chance of live birth provided by the post-treatment model is 46% in the first complete ICSI cycle and 81% over three complete cycles.

LIMITATIONS, REASONS FOR CAUTION

Two predictors from the original models, duration of infertility and previous pregnancy, which were not available in the recent HFEA dataset, were imputed using data from the older cohort used to develop the models. The HFEA dataset does not contain some other potentially important predictors, e.g. BMI, ethnicity, race, smoking and alcohol intake in women, as well as measures of ovarian reserve such as antral follicle count.

WIDER IMPLICATIONS OF THE FINDINGS

Both updated models show improved predictive ability and provide estimates which are more reflective of current practice and patient case mix. The updated OPIS tool can be used by clinicians to help shape couples' expectations by informing them of their individualized chances of live birth over a sequence of multiple complete cycles of IVF.

STUDY FUNDING/COMPETING INTEREST(S)

This study was supported by an Elphinstone scholarship scheme at the University of Aberdeen and Aberdeen Fertility Centre, University of Aberdeen. S.B. has a commitment of research funding from Merck. D.J.M. and M.B.R. declare support for the present manuscript from Elphinstone scholarship scheme at the University of Aberdeen and Assisted Reproduction Unit at Aberdeen Fertility Centre, University of Aberdeen. D.J.M. declares grants received by University of Aberdeen from NHS Grampian, The Meikle Foundation, and Chief Scientist Office in the past 3 years. D.J.M. declares receiving an honorarium for lectures from Merck. D.J.M. is Associate Editor of Human Reproduction Open and Statistical Advisor for Reproductive BioMed Online. S.B. declares royalties from Cambridge University Press for a book. S.B. declares receiving an honorarium for lectures from Merck, Organon, Ferring, Obstetric and Gynaecological Society of Singapore, and Taiwanese Society for Reproductive Medicine. S.B. has received support from Merck, ESHRE, and Ferring for attending meetings as speaker and is on the METAFOR and CAPRE Trials Data Monitoring Committee.

TRIAL REGISTRATION NUMBER

N/A.

Adaptive data-driven models to best predict the likelihood of live birth as the IVF cycle moves on and for each embryo transfer

PURPOSE

To dynamically assess the evolution of live birth predictive factors' impact throughout the in vitro fertilization (IVF) process, for each fresh and subsequent frozen embryo transfers.

METHODS

In this multicentric study, data from 13,574 fresh IVF cycles and 6,770 subsequent frozen embryo transfers were retrospectively analyzed. Fifty-seven descriptive parameters were included and split into four categories: (1) demographic (couple's baseline characteristics), (2) ovarian stimulation, (3) laboratory data, and (4) embryo transfer (fresh and frozen). All these parameters were used to develop four successive predictive models with the outcome being a live birth event.

RESULTS

Eight parameters were predictive of live birth in the first step after the first consultation, 9 in the second step after the stimulation, 11 in the third step with laboratory data, and 13 in the 4th step at the transfer stage. The predictive performance of the models increased at each step. Certain parameters remained predictive in all 4 models while others were predictive only in the first models and no longer in the subsequent ones when including new parameters. Moreover, some parameters were predictive in fresh transfers but not in frozen transfers.

CONCLUSION

This work evaluates the chances of live birth for each embryo transfer individually and not the cumulative outcome after multiple IVF attempts. The different predictive models allow to determine which parameters should be taken into account or not at each step of an IVF cycle, and especially at the time of each embryo transfer, fresh or frozen.

A large observational data study supporting the PROsPeR score classification in poor ovarian responders according to live birth outcome

STUDY QUESTION

Can the Poor Responder Outcome Prediction (PROsPeR) score identify live birth outcomes in subpopulations of patients with poor ovarian response (POR) defined according to the ESHRE Bologna criteria (female age, anti-Müllerian hormone (AMH), number of oocytes retrieved during the previous cycle (PNO) after treatment with originator recombinant human follitropin alfa?

SUMMARY ANSWER

The PROsPeR score discriminated the probability of live birth in patients with POR using observational data with fair discrimination (AUC ≅ 70%) and calibration, and the AUC losing less than 5% precision compared with a model developed using the observational data.

WHAT IS KNOWN ALREADY

Although scoring systems for the likelihood of live birth after ART have been developed, their accuracy may be insufficient, as they have generally been developed in the general population with infertility and were not validated for patients with POR. The PROsPeR score was developed using data from the follitropin alfa (GONAL-f; Merck KGaA, Darmstadt, Germany) arm of the Efficacy and Safety of Pergoveris in Assisted Reproductive Technology (ESPART) randomized controlled trial (RCT) and classifies women with POR as mild, moderate or severe, based upon three variables: female age, serum AMH level and number of oocytes retrieved during the previous cycle (PNO).

STUDY DESIGN, SIZE, DURATION

The external validation of the PROsPeR score was completed using data derived from eight different centres in France. In addition, the follitropin alfa data from the ESPART RCT, originally used to develop the PROsPeR score, were used as reference cohort. The external validation of the PROsPeR score l was assessed using AUC. A predetermined non-inferiority limit of 0.10 compared with a reference sample and calibration (Hosmer-Lemeshow test) were the two conditions required for evaluation.

PARTICIPANTS/MATERIALS, SETTING, METHODS

The observational cohort included data from 8085 ART treatment cycles performed with follitropin alfa in patients with POR defined according to the ESHRE Bologna criteria (17.6% of the initial data set). The ESPART cohort included 477 ART treatment cycles with ovarian stimulation performed with follitropin alfa in patients with POR.

MAIN RESULTS AND THE ROLE OF CHANCE

The external validation of the PROsPeR score to identify subpopulations of women with POR with different live birth outcomes was shown in the observational cohort (AUC = 0.688; 95% CI: 0.662, 0.714) compared with the ESPART cohort (AUC = 0.695; 95% CI: 0.623, 0.767). The AUC difference was -0.0074 (95% CI: -0.083, 0.0689). This provided evidence, with 97.5% one-sided confidence, that there was a maximum estimated loss of 8.4% in discrimination between the observational cohort and the ESPART cohort, which was below the predetermined margin of 10%. The Hosmer-Lemeshow test did not reject the calibration when comparing observed and predicted data (Hosmer-Lemeshow test = 1.266688; P = 0.260).

LIMITATIONS, REASONS FOR CAUTION

The study was based on secondary use of data that had not been collected specifically for the analysis reported here and the number of characteristics used to classify women with POR was limited to the available data. The data were from a limited number of ART centres in a single country, which may present a bias risk; however, baseline patient data were similar to other POR studies.

WIDER IMPLICATIONS OF THE FINDINGS

This evaluation of the PROsPeR score using observational data supports the notion that the likelihood of live birth may be calculated with reasonable precision using three readily available pieces of data (female age, serum AMH and PNO). The PROsPeR score has potential to be used to discriminate expected probability of live birth according to the degree of POR (mild, moderate, severe) after treatment with follitropin alfa, enabling comparison of performance at one centre over time and the comparison between centres.

STUDY FUNDING/COMPETING INTEREST(S)

This analysis was funded by Merck KGaA, Darmstadt, Germany. P.L. received grants from Merck KGaA, outside of the submitted work. N.M. reports grants, personal fees and non-financial support from Merck KGaA outside the submitted work. T.D.H. is Vice President and Head of Global Medical Affairs Fertility, Research and Development at Merck KGaA, Darmstadt, Germany. P.A. has received personal fees from Merck KGaA, Darmstadt, Germany, outside the submitted work. C.R. has received grants and personal fees from Gedeon Richter and Merck Serono S.A.S., France, an affiliate of Merck KGaA, Darmstadt, Germany, outside the submitted work. P.S. reports congress support from Merck Serono S.A.S., France (an affiliate of Merck KGaA, Darmstadt, Germany), Gedeon Richter, TEVA and MDS outside the submitted work. C.A., J.P., G.P. and R.W. declare no conflict of interest.

TRIAL REGISTRATION NUMBER

N/A.

A systematic review of the quality of clinical prediction models in in vitro fertilisation

STUDY QUESTION

What are the best-quality clinical prediction models in IVF (including ICSI) treatment to inform clinicians and their patients of their chance of success?

SUMMARY ANSWER

The review recommends the McLernon post-treatment model for predicting the cumulative chance of live birth over and up to six complete cycles of IVF.

WHAT IS KNOWN ALREADY

Prediction models in IVF have not found widespread use in routine clinical practice. This could be due to their limited predictive accuracy and clinical utility. A previous systematic review of IVF prediction models, published a decade ago and which has never been updated, did not assess the methodological quality of existing models nor provided recommendations for the best-quality models for use in clinical practice.

STUDY DESIGN, SIZE, DURATION

The electronic databases OVID MEDLINE, OVID EMBASE and Cochrane library were searched systematically for primary articles published from 1978 to January 2019 using search terms on the development and/or validation (internal and external) of models in predicting pregnancy or live birth. No language or any other restrictions were applied.

PARTICIPANTS/MATERIALS, SETTING, METHODS

The PRISMA flowchart was used for the inclusion of studies after screening. All studies reporting on the development and/or validation of IVF prediction models were included. Articles reporting on women who had any treatment elements involving donor eggs or sperm and surrogacy were excluded. The CHARMS checklist was used to extract and critically appraise the methodological quality of the included articles. We evaluated models' performance by assessing their c-statistics and plots of calibration in studies and assessed correct reporting by calculating the percentage of the TRIPOD 22 checklist items met in each study.

MAIN RESULTS AND THE ROLE OF CHANCE

We identified 33 publications reporting on 35 prediction models. Seventeen articles had been published since the last systematic review. The quality of models has improved over time with regard to clinical relevance, methodological rigour and utility. The percentage of TRIPOD score for all included studies ranged from 29 to 95%, and the c-statistics of all externally validated studies ranged between 0.55 and 0.77. Most of the models predicted the chance of pregnancy/live birth for a single fresh cycle. Six models aimed to predict the chance of pregnancy/live birth per individual treatment cycle, and three predicted more clinically relevant outcomes such as cumulative pregnancy/live birth. The McLernon (pre- and post-treatment) models predict the cumulative chance of live birth over multiple complete cycles of IVF per woman where a complete cycle includes all fresh and frozen embryo transfers from the same episode of ovarian stimulation. McLernon models were developed using national UK data and had the highest TRIPOD score, and the post-treatment model performed best on external validation.

LIMITATIONS, REASONS FOR CAUTION

To assess the reporting quality of all included studies, we used the TRIPOD checklist, but many of the earlier IVF prediction models were developed and validated before the formal TRIPOD reporting was published in 2015. It should also be noted that two of the authors of this systematic review are authors of the McLernon model article. However, we feel we have conducted our review and made our recommendations using a fair and transparent systematic approach.

WIDER IMPLICATIONS OF THE FINDINGS

This study provides a comprehensive picture of the evolving quality of IVF prediction models. Clinicians should use the most appropriate model to suit their patients' needs. We recommend the McLernon post-treatment model as a counselling tool to inform couples of their predicted chance of success over and up to six complete cycles. However, it requires further external validation to assess applicability in countries with different IVF practices and policies.

STUDY FUNDING/COMPETING INTEREST(S)

The study was funded by the Elphinstone Scholarship Scheme and the Assisted Reproduction Unit, University of Aberdeen. Both D.J.M. and S.B. are authors of the McLernon model article and S.B. is Editor in Chief of Human Reproduction Open. They have completed and submitted the ICMJE forms for Disclosure of potential Conflicts of Interest. The other co-authors have no conflicts of interest to declare.

REGISTRATION NUMBER

N/A.

Fertility technologies and how to optimize laboratory performance to support the shortening of time to birth of a healthy singleton: a Delphi consensus

PURPOSE

To explore how the assisted reproductive technology (ART) laboratories can be optimized and standardized to enhance embryo culture and selection, to bridge the gap between standard practice and the new concept of shortening time to healthy singleton birth.

METHODS

A Delphi consensus was conducted (January to July 2018) to assess how the ART laboratory could be optimized, in conjunction with existing guidelines, to reduce the time to a healthy singleton birth. Eight experts plus the coordinator discussed and refined statements proposed by the coordinator. The statements were distributed via an online survey to 29 participants (including the eight experts from step 1), who voted on their agreement/disagreement with each statement. Consensus was reached if ≥ 66% of participants agreed/disagreed with a statement. If consensus was not achieved for any statement, that statement was revised and the process repeated until consensus was achieved. Details of statements achieving consensus were communicated to the participants.

RESULTS

Consensus was achieved for all 13 statements, which underlined the need for professional guidelines and standardization of lab processes to increase laboratory competency and quality. The most important points identified were the improvement of embryo culture and embryo assessment to shorten time to live birth through the availability of more high-quality embryos, priority selection of the most viable embryos and improved cryosurvival.

CONCLUSION

The efficiency of the ART laboratory can be improved through professional guidelines on standardized practices and optimized embryo culture environment, assessment, selection and cryopreservation methodologies, thereby reducing the time to a healthy singleton delivery.

Predicting the chance on live birth per cycle at each step of the IVF journey: external validation and update of the van Loendersloot multivariable prognostic model

OBJECTIVE

To study the performance of the 'van Loendersloot' prognostic model for our clinic's in vitro fertilisation (IVF) in its original version, the refitted version and in an adapted version replacing previous by current cycle IVF laboratory variables.

METHODS

This retrospective cohort study in our academic tertiary fertility clinic analysed 1281 IVF cycles of 591 couples, who completed at least one 2nd-6th IVF cycle with own fresh gametes after a previous IVF cycle with the same partner in our clinic between 2010 and 2018. The outcome of interest was the chance on a live birth after one complete IVF cycle (including all fresh and frozen embryo transfers from the same episode of ovarian stimulation). Model performance was expressed in terms of discrimination (c-statistics) and calibration (calibration model, comparison of prognosis to observed ratios of five disjoint groups formed by the quintiles of the IVF prognoses and a calibration plot).

RESULTS

A total of 344 live births were obtained (26.9%). External validation of the original van Loendersloot model showed a poor c-statistic of 0.64 (95% CI: 0.61 to 0.68) and an underestimation of IVF success. The refitted and the adapted models showed c-statistics of respectively 0.68 (95% CI: 0.65 to 0.71) and 0.74 (95% CI: 0.70 to 0.77). Similar c-statistics were found with cross-validation. Both models showed a good calibration model; refitted model: intercept=0.00 (95% CI: -0.23 to 0.23) and slope=1.00 (95% CI: 0.79 to 1.21); adapted model: intercept=0.00 (95% CI: -0.18 to 0.18) and slope=1.00 (95% CI: 0.83 to 1.17). Prognoses and observed success rates of the disjoint groups matched well for the refitted model and even better for the adapted model.

CONCLUSION

External validation of the original van Loendersloot model indicated that model updating was recommended. The good performance of the refitted and adapted models allows informing couples about their IVF prognosis prior to an IVF cycle and at the time of embryo transfer. Whether this has an impact on couple's expected success rates, distress and IVF discontinuation can now be studied.

Can Ratios Between Prognostic Factors Predict the Clinical Pregnancy Rate in an IVF/ICSI Program with a GnRH Agonist-FSH/hMG Protocol? An Assessment of 2421 Embryo Transfers, and a Review of the Literature

None of the models developed in in vitro fertilization/intracytoplasmic sperm injection (IVF/ICSI) is sufficiently good predictors of pregnancy. The aim of this study was to determine whether ratios between prognostic factors could predict the clinical pregnancy rate in IVF/ICSI. We analyzed IVF/ICSI cycles (based on long GnRH agonist-FSH protocols) at two ART centers (the second to validate externally the data). The ratios studied were (i) the total FSH dose divided by the serum estradiol level on the hCG trigger day, (ii) the total FSH dose divided by the number of mature oocytes, (iii) the serum estradiol level on the trigger day divided by the number of mature oocytes, (iv) the serum estradiol level on the trigger day divided by the endometrial thickness on the trigger day, (v) the serum estradiol level on the trigger day divided by the number of mature oocytes and then by the number of grade 1 or 2 embryos obtained, and (vi) the serum estradiol level on the trigger day divided by the endometrial thickness on the trigger day and then by the number of grade 1 or 2 embryos obtained. The analysis covered 2421 IVF/ICSI cycles with an embryo transfer, leading to 753 clinical pregnancies (31.1% per transfer). Four ratios were significantly predictive in both centers; their discriminant power remained moderate (area under the receiver operating characteristic curve between 0.574 and 0.610). In contrast, the models' calibration was excellent (coefficients: 0.943-0.978; p < 0.001). Our ratios were no better than existing models in IVF/ICSI programs. In fact, a strongly discriminant predictive model will be probably never be obtained, given the many factors that influence the occurrence of a pregnancy.

Methodologic considerations in randomized clinical trials in reproductive medicine

Randomized controlled trials (RCTs) are the cornerstone of evidence-based medicine. In this series in Fertility and Sterility, several aspects of RCTs are discussed, with contributions on multicenter RCTs, different international settings, and integrity of RCTs. The present contribution deals with methodologic issues. We discuss different types of RCTs based on null hypothesis (superiority vs. noninferiority vs. equivalence) as well as frequentist versus Bayesian interpretation. We also discuss the use of RCTs in the era of personalized medicine and RCTs to address diagnostic and prognostic questions. Finally, we address the use of big data compared with the use of RCTs.

Decision points for individualized hormonal stimulation with recombinant gonadotropins for treatment of women with infertility

It is essential that fertility treatment is individualized based on a thorough diagnostic work-up, with treatment tailored to the patients' requirements. This individualization should be kept in mind during the main decision points that occur before and during treatment. Treatment customization must include consideration of both the woman and her partner involved in the process together, including their collective treatment goals. Once treatment goals have been agreed and diagnostic evaluations performed, personalization based on patient characteristics, together with an understanding of treatment goals and patient preferences, enables the selection of appropriate treatments, protocols, products and their dosing. Following treatment initiation, monitoring and adaptation of product and dose can then ensure optimal outcomes. Currently, it is not possible to base treatment decisions on every characteristic of the patient and personalization is based on biomarkers that have been identified as the most relevant. However, in the future, the use of artificial intelligence coupled with continuous monitoring should enable greater individualization and improve outcomes. This review considers the current state-of-the-art related to decision points during individualized treatment of female infertility, before looking at future developments that might further assist in making individualized treatment decisions, including the use of computer-assisted decision making.

Both high and low HCG day progesterone concentrations negatively affect live birth rates in IVF/ICSI cycles

RESEARCH QUESTION

Can previous reports of a decreased probability of success in stimulated IVF cycles with premature rise of progesterone, as determined by progesterone concentration on HCG day (PHCG), be confirmed?

DESIGN

Retrospective, observational, single-centre cohort study conducted on 5447 IVF and intracytoplasmic (ICSI) cycles carried out among 2192 patients between 2009 and 2015, with conventional ovarian stimulation. This large database was used to develop a non-linear mixed prognosis model of live birth rate (LBR) incorporating PHCG as a predictor.

RESULTS

In addition to known predictors (age, body mass index, anti-Müllerian hormone, type of infertility), PHCG was associated with a linear effect (OR 0.78 per Log[PHCG]ng/ml, 95% CI 0.611 to 0.997, P = 0.047) combined with a strong quadratic effect (OR 0.585 per Log²(PHCG)ng/ml, 95% CI 0.444 to 0.775, P < 0.001) resulting into a parabolic reverse-U curve. A significant interaction (P = 0.038) was found between PHCG and number of oocytes if three or less, but the effect of PHCG remains modest. For higher oocyte numbers, LBR rapidly increases with number of retrieved oocytes; however, LBR becomes more sensitive to PHCG as the number of oocytes increases. Higher live birth prognoses occur for optimal PHCG but are sharply reduced for lower or higher PHCG.

CONCLUSIONS

Evidence is provided of an important negative effect of PHCG at lower and higher values, independent of oocyte number, thus defining appropriate ranges for fresh embryo transfer or freeze-all strategy. In poor responders, premature progesterone rise may be ignored, thus avoiding unnecessary cancellations or embryo freezing. Conversely, in higher responders, the negative effect of progesterone elevation is more pronounced, suggesting that freeze-all policy should be applied more widely.

Predicting the cumulative chance of live birth over multiple complete cycles of in vitro fertilization: an external validation study

STUDY QUESTION

Are the published pre-treatment and post-treatment McLernon models, predicting cumulative live birth rates (LBR) over multiple complete IVF cycles, valid in a different context?

SUMMARY ANSWER

With minor recalibration of the pre-treatment model, both McLernon models accurately predict cumulative LBR in a different geographical context and a more recent time period.

WHAT IS KNOWN ALREADY

Previous IVF prediction models have estimated the chance of a live birth after a single fresh embryo transfer, thereby excluding the important contribution of embryo cryopreservation and subsequent IVF cycles to cumulative LBR. In contrast, the recently developed McLernon models predict the cumulative chance of a live birth over multiple complete IVF cycles at two certain time points: (i) before initiating treatment using baseline characteristics (pre-treatment model) and (ii) after the first IVF cycle adding treatment related information to update predictions (post-treatment model). Before implementation of these models in clinical practice, their predictive performance needs to be validated in an independent cohort.

STUDY DESIGN, SIZE, DURATION

External validation study in an independent prospective cohort of 1515 Dutch women who participated in the OPTIMIST study (NTR2657) and underwent their first IVF treatment between 2011 and 2014. Participants underwent a total of 2881 complete treatment cycles, with a complete cycle defined as all fresh and frozen thawed embryo transfers resulting from one episode of ovarian stimulation. The follow up duration was 18 months after inclusion, and the primary outcome was ongoing pregnancy leading to live birth.

PARTICIPANTS/MATERIALS, SETTING, METHODS

Model performance was externally validated up to three complete treatment cycles, using the linear predictor as described by McLernon et al. to calculate the probability of a live birth. Discrimination was expressed by the c-statistic and calibration was depicted graphically in a calibration plot. In contrast to the original model development cohort, anti-Müllerian hormone (AMH), antral follicle count (AFC) and body weight were available in the OPTIMIST cohort, and evaluated as potential additional predictors for model improvement.

MAIN RESULTS AND THE ROLE OF CHANCE

Applying the McLernon models to the OPTIMIST cohort, the c-statistic of the pre-treatment model was 0.62 (95% CI: 0.59-0.64) and of the post-treatment model 0.71 (95% CI: 0.69-0.74). The calibration plot of the pre-treatment model indicated a slight overestimation of the cumulative LBR. To improve calibration, the pre-treatment model was recalibrated by subtracting 0.35 from the intercept. The post-treatment model calibration plot revealed accurate cumulative LBR predictions. After addition of AMH, AFC and body weight to the McLernon models, the c-statistic of the updated pre-treatment model improved slightly to 0.66 (95% CI: 0.64-0.68), and of the updated post-treatment model remained at the previous level of 0.71 (95% CI: 0.69-0.73). Using the recalibrated pre-treatment model, a woman aged 30 years with 2 years of primary infertility who starts ICSI treatment for male factor infertility has a chance of 40% of a live birth from the first complete cycle, increasing to 72% over three complete cycles. If this woman weighs 70 kg, has an AMH of 1.5 ng/mL and an AFC of 10 measured at the beginning of her treatment, the updated pre-treatment model revises the estimated chance of a live birth to 30% in the first complete cycle and 59% over three complete cycles. If this woman then has five retrieved oocytes, no embryos cryopreserved and a single fresh cleavage stage embryo transfer in her first ICSI cycle, the post-treatment model estimates the chances of a live birth at 28 and 58%, respectively.

LIMITATIONS, REASONS FOR CAUTION

Two randomized controlled trials (RCT) evaluating the effectiveness of gonadotropin dose individualization on basis of the AFC were nested within the OPTIMIST study. The strict dosing regimens, the RCT in- and exclusion criteria and the limited follow up time of 18 months might have influenced model performance in this independent cohort. Also, consistent with the original model development study, external validation was performed using the optimistic assumption that the cumulative LBR in couples who discontinue treatment without a live birth would have been equal to that of those who continue treatment.

WIDER IMPLICATIONS OF THE FINDINGS

After national recalibration to account for geographical differences in IVF treatment, the McLernon prediction models can be introduced as new counselling tools in clinical practice to inform patients and to complement clinical reasoning. These models are the first to offer an objective and personalized estimate of the cumulative probability of a live birth over multiple complete IVF cycles.

STUDY FUNDING/COMPETING INTEREST(S)

No external funds were obtained for this study. M.J.C.E., D.J.M. and S.B. have nothing to disclose. J.A.L, S.C.O, T.C.v.T. and H.LT. received an unrestricted personal grant from Merck BV. B.W.M. is supported by a NHMRC Practitioner Fellowship (GNT1082548) and reports consultancy for ObsEva, Merck and Guerbet. F.J.M.B. receives monetary compensation as a member of the external advisory board for Merck BV (the Netherlands) and Ferring pharmaceutics BV (the Netherlands), for consultancy work for Gedeon Richter (Belgium) and Roche Diagnostics on automated AMH assay development, and for a research cooperation with Ansh Labs (USA).

TRIAL REGISTRATION NUMBER

Not applicable.

An Observational Retrospective Cohort Trial on 4,828 IVF Cycles Evaluating Different Low Prognosis Patients Following the POSEIDON Criteria

Objective: To study the actual controlled ovarian stimulation (COS) management in women with suboptimal response, comparing clinical outcomes to the gonadotropins consume, considering potential role of luteinizing hormone (LH) addition to follicle-stimulating hormone (FSH). Design: Monocentric, observational, retrospective, real-world, clinical trial on fresh intra-cytoplasmic sperm injection (ICSI) cycles retrieving from 1 to 9 oocytes, performed at Humanitas Fertility Center from January 1st, 2012 to December 31st, 2015. Methods: COS protocols provided gonadotropin releasing-hormone (GnRH) agonist long, flare-up, short and antagonist. Both recombinant and urinary FSH were used for COS and LH was added according to the clinical practice. ICSI outcomes considered were: gonadotropins dosages; total, mature, injected and frozen oocytes; cumulative, transferred and frozen embryos; implantation rate; pregnancy, delivery and miscarriage rates. Outcomes were compared according to the gonadotropin regimen used during COS. Results: Our cohort showed 20.8% of low responders, defined as 1-3 oocytes retrieved and 79.2% of "suboptimal" responders, defined as 4-9 oocytes retrieved. According to recent POSEIDON stratification, cycles were divided in group 1 (6.9%), 2 (19.8%), 3 (11.7%), and 4 (61.5%). The cohort was divided in 3 groups, according to the gonadotropin's regimen. Women treated with FSH plus LH showed worst prognostic factors, in terms of age, basal FSH, AMH, and AFC. This difference was evident in suboptimal responders, whereas only AMH and AFC were different among treatment groups in low responders. Although a different result, in terms of oocytes and embryos detected, major ICSI outcomes (i.e., pregnancy and delivery rates) were similar among groups of COS treatment. Outcomes were significantly different among Poseidon groups. Implantation, pregnancy and delivery rates were significantly higher in Poseidon group 1 and progressively declined in other POSEIDON groups, reaching the worst percentage in group 4. Conclusions: In clinical practice, women with worst prognosis factors are generally treated with a combination of LH and FSH. Despite low prognosis women showed a reduced number of oocytes retrieved, the final ICSI outcome, in terms of pregnancy, is similarly among treatment group. This result suggests that the LH addition to FSH during COS could improve the quality of oocytes retrieved, balancing those differences that are evident at baseline. Clinical Trial Registration: www.ClinicalTrials.gov, identifier: NCT03290911.

How time to healthy singleton delivery could affect decision-making during infertility treatment: a Delphi consensus

RESEARCH QUESTION

How might time to healthy singleton delivery affect decision-making during infertility treatment?

DESIGN

This was a Delphi consensus investigating expert opinion that comprised three steps. In Step 1, 12 experts developed statements. In Step 2, 27 experts (including 12 from Step 1) voted (online survey) on their agreement/disagreement with each statement (providing reasons). Consensus was reached if ≥66% of participants agreed/disagreed. Statements not reaching consensus were revised and the process repeated until consensus was achieved. In Step 3 details of the final agreed statements were communicated.

RESULTS

Twelve statements were developed, and consensus (agreement) was reached on all after one round of voting.

CONCLUSIONS

Time to healthy singleton delivery should be taken into consideration when making decisions related to infertility treatment, and it is important that fertility treatment is provided in a timely manner, avoiding over- or under-treatment. In all subfertile women <40 years old, IVF outcomes could be optimized by performing up to six single-embryo transfers and certain procedures might reduce time to healthy singleton delivery. These procedures include preimplantation genetic testing for aneuploidies, frozen replacement cycles immediately after failed fresh cycles and use of gonadotrophin-releasing hormone antagonists. Finally, the number of oocytes retrieved should be maximized to increase cumulative live birth rate.

Making IVF more effective through the evolution of prediction models: is prognosis the missing piece of the puzzle?

Assisted reproductive technology has evolved tremendously since the emergence of in vitro fertilization (IVF). In the course of the recent decade, there have been significant efforts in order to minimize multiple gestations, while improving percentages of singleton pregnancies and offering individualized services in IVF, in line with the trend of personalized medicine. Patients as well as clinicians and the entire IVF team benefit majorly from 'knowing what to expect' from an IVF cycle. Hereby, the question that has emerged is to what extent prognosis could facilitate toward the achievement of the above goal. In the current review, we present prediction models based on patients' characteristics and IVF data, as well as models based on embryo morphology and biomarkers during culture shaping a complication free and cost-effective personalized treatment. The starting point for the implementation of prediction models was initiated by the aspiration of moving toward optimal practice. Thus, prediction models could serve as useful tools that could safely set the expectations involved during this journey guiding and making IVF treatment more effective. The aim and scope of this review is to thoroughly present the evolution and contribution of prediction models toward an efficient IVF treatment.

ABBREVIATIONS

IVF: In vitro fertilization; ART: assisted reproduction techniques; BMI: body mass index; OHSS: ovarian hyperstimulation syndrome; eSET: elective single embryo transfer; ESHRE: European Society of Human Reproduction and Embryology; mtDNA: mitochondrial DNA; nDNA: nuclear DNA; ICSI: intracytoplasmic sperm injection; MBR: multiple birth rates; LBR: live birth rates; SART: Society for Assisted Reproductive Technology Clinic Outcome Reporting System; AFC: antral follicle count; GnRH: gonadotrophin releasing hormone; FSH: follicle stimulating hormone; LH: luteinizing hormone; AMH: anti-Müllerian hormone; DHEA: dehydroepiandrosterone; PCOS: polycystic ovarian syndrome; NPCOS: non-polycystic ovarian syndrome; CE: cost-effectiveness; CC: clomiphene citrate; ORT: ovarian reserve test; EU: embryo-uterus; DET: double embryo transfer; CES: Cumulative Embryo Score; GES: Graduated Embryo Score; CSS: Combined Scoring System; MSEQ: Mean Score of Embryo Quality; IMC: integrated morphology cleavage; EFNB2: ephrin-B2; CAMK1D: calcium/calmodulin-dependent protein kinase 1D; GSTA4: glutathione S-transferase alpha 4; GSR: glutathione reductase; PGR: progesterone receptor; AMHR2: anti-Müllerian hormone receptor 2; LIF: leukemia inhibitory factor; sHLA-G: soluble human leukocyte antigen G.

Establishment and validation of a score to predict ovarian response to stimulation in IVF

This study aimed to integrate clinical and biological parameters in a score able to predict ovarian response to stimulation for IVF in gonadotrophin-releasing hormone (GnRH) antagonist protocols. A progressive discriminant analysis to establish a score including the main clinical and biological parameters predicting ovarian response was performed by retrospectively analysing data from the first ovarian stimulation cycle of 494 patients. The score was validated in a prospectively enrolled, independent set of 257 patients undergoing their first ovarian stimulation cycle. All ovarian stimulations were performed using a combination of GnRH antagonist and recombinant FSH. Ovarian response was assessed through ovarian sensitivity index (OSI). Parameters from the patients' database were classified according to correlation with OSI: the progressive discriminant analysis resulted in the following calculation: score = 0.192 - (0.004 × FSH (IU/l)) + (0.012 × LH:FSH ratio) + (0.002 × AMH (ng/ml)) - (0.002 × BMI (kg/m²)) + (0.001 × AFC) - (0.002 × age (years)). This score was significantly correlated with OSI in the retrospective (r = 0.599; P < 0.0001) and prospective (r = 0.584; P < 0.0001) studies. In conclusion, the score including clinical and biological parameters could explain 60% of the variance in ovarian response to stimulation.

Towards better meta-analyses in assisted reproductive technology: Fixed, random or multivariate models?

AIM: To study the validity of the fixed, random, and multivariate meta-analytical models applied in meta-analyses in artificial reproduction technique.

METHODS: Based on common characteristics of in vitro fertilization (IVF) meta-analyses, we simulated a large number of data to compare results issued from the fixed model (FM) with the random model (RM). For multiple endpoints meta-analysis (MA), we compared the univariate RM with the multivariate model (MM). Finally, we illustrate our findings in re-analyzing a recent MA.

RESULTS: In our review, although a homogeneous effect was excluded in 89% of the MAs (11%), FM was utilized in 41 studies (82%). From simulations, a concordance of 59% ± 6% was found between the two tests, with up to 65% of falsely significant results with FM. The Q-test on studies characterized by substantial heterogeneity falsely accepted homogeneity in 46% of studies. Comparing separate univariate RM and MM on multiple endpoints studies, MM reduces the between endpoint discrepancy (BED) of 68%, and increases the power of 57% ± 8%. In the example dealing with the controversial effect of luteneizing hormone supplementation to follicle stimulating hormone during ovarian stimulation in IVF cycles, MM reduced BED by 66%, and consistent effects were found for all the endpoints, irrespective of partial reporting.

CONCLUSION: The FM generally may produce falsely significant differences. The RM should always be used. For multiple endpoints, the MM constitutes the best option.

Antimüllerian hormone levels and antral follicle count as prognostic indicators in a personalized prediction model of live birth

OBJECTIVE

To compare antimüllerian hormone (AMH) and antral follicle count (AFC) separately and in combination with clinical characteristics for the prediction of live birth after controlled ovarian stimulation.

DESIGN

Retrospective development and temporal external validation of prediction model.

SETTING

Outpatient IVF clinic.

PATIENT(S)

We applied the boosted tree method to develop three prediction models incorporating clinical characteristics plus AMH or AFC or the combination on 2,124 linked IVF cycles from 2006 to 2010 and temporally externally validated predicted live-birth probabilities with an independent data set comprising 1,121 cycles from 2011 to 2012.

INTERVENTION(S)

None.

MAIN OUTCOME MEASURE(S)

Predictive power (posterior log of odds ratio compared to age, or PLORA), reclassification, receiver operator characteristic analysis, calibration, dynamic range.

RESULT(S)

Predictive power, was highest for the AMH model (PLORA = 29.1), followed by the AMH-AFC model (PLORA = 28.3) and AFC model (PLORA = 22.5). The prediction errors were 1% to <5% in each prognostic tier for all three models, except for the predicted live-birth probabilities of <10% in the AFC model, where the prediction error was 8%. The improvement in predictive power was highest for the AMH model: 76.2% improvement over age alone relative to 59% improvement for AFC and 73.3% for the combined model. Receiver operating characteristic analysis demonstrated that the AMH and the combined model had comparable discrimination (area under the curve = 0.716) and similar prediction error for high and low strata of live-birth prediction, with an improvement of 6.3% over age alone.

CONCLUSION(S)

The validated prediction model confirmed that AMH when combined with clinical characteristics can accurately identify the likelihood of live birth with a low prediction error. AFC provided no added predictive value beyond AMH.

External validation and calibration of IVFpredict: a national prospective cohort study of 130,960 in vitro fertilisation cycles

BACKGROUND

Accurately predicting the probability of a live birth after in vitro fertilisation (IVF) is important for patients, healthcare providers and policy makers. Two prediction models (Templeton and IVFpredict) have been previously developed from UK data and are widely used internationally. The more recent of these, IVFpredict, was shown to have greater predictive power in the development dataset. The aim of this study was external validation of the two models and comparison of their predictive ability.

METHODS AND FINDINGS

130,960 IVF cycles undertaken in the UK in 2008-2010 were used to validate and compare the Templeton and IVFpredict models. Discriminatory power was calculated using the area under the receiver-operator curve and calibration assessed using a calibration plot and Hosmer-Lemeshow statistic. The scaled modified Brier score, with measures of reliability and resolution, were calculated to assess overall accuracy. Both models were compared after updating for current live birth rates to ensure that the average observed and predicted live birth rates were equal. The discriminative power of both methods was comparable: the area under the receiver-operator curve was 0.628 (95% confidence interval (CI): 0.625-0.631) for IVFpredict and 0.616 (95% CI: 0.613-0.620) for the Templeton model. IVFpredict had markedly better calibration and higher diagnostic accuracy, with calibration plot intercept of 0.040 (95% CI: 0.017-0.063) and slope of 0.932 (95% CI: 0.839-1.025) compared with 0.080 (95% CI: 0.044-0.117) and 1.419 (95% CI: 1.149-1.690) for the Templeton model. Both models underestimated the live birth rate, but this was particularly marked in the Templeton model. Updating the models to reflect improvements in live birth rates since the models were developed enhanced their performance, but IVFpredict remained superior.

CONCLUSION

External validation in a large population cohort confirms IVFpredict has superior discrimination and calibration for informing patients, clinicians and healthcare policy makers of the probability of live birth following IVF.

To what extent does Anti-Mullerian Hormone contribute to a better prediction of live birth after IVF?

OBJECTIVE

We assessed the predictive value added by Anti-Mullerian Hormone (AMH) to currently validated live birth (LB) prediction models.

METHODS

Based on recent data from our center, we compared the external validity of the Templeton Model (TM) and its recent improvement (TMA) to select our model of reference. The added predictive value of AMH was assessed in testing the likelihood ratio significance and the Net Reclassification Index (NRI). The surrogate utility of AMH was tested by conducting an exploratory stepwise logistic regression.

RESULTS

Based on 715 cycles, the original TM had poor performances (auROC C = 0.61 [0.58, 0.66], improving by fitting TM to our data (C = 0.71[0.66, 0.75]. TMA fitting proved better (C = 0.76; 95 %CI: 0.71, 0.80) and was selected as model of reference. Adding AMH to TMA or TM had no effect on discrimination (C = 0.76; 95 %CI: 0.72, 0.80), the likelihood ratio test was significant (p = 0.023), but the NRI was not (6.7 %; p = 0.055). A stepwise exploratory logistic regression identified the effects of age, previous IVF resulting in LB, time trend and AMH, leading to a prediction model reduced to four predictors (C = 0.75 [0.70, 0.81]).

CONCLUSION

The added predictive value of AMH is limited. A possible surrogate/simplifying effect of AMH was found in eliminating 9/13 predictors from the model of reference. We conclude that whereas AMH does not add significant predictive value to the existing model, it contributes to simplifying the equation to reliable, easy to collect, and available in all databases predictors: age, AMH, time trend and female previous fertility history.

Comparison of two models predicting IVF success; the effect of time trends on model performance

STUDY QUESTION

How well does the recently developed UK model predicting the success rate of IVF treatment (the 2011 Nelson model) perform in comparison with a UK model developed in the early 1990s (the Templeton model)?

SUMMARY ANSWER

Both models showed similar performance, after correction for the increasing success rate over time of IVF.

WHAT IS KNOWN ALREADY

For counselling couples undergoing IVF treatment it is of paramount importance to be able to predict success. Several prediction models for the chance of success after IVF treatment have been developed. So far, the Templeton model has been recommended as the best approach after having been validated in several independent patient data sets. The Nelson model, developed in 2011 and characterized by the largest development sample containing the most recently treated couples, may well perform better.

STUDY DESIGN, SIZE, DURATION

We tested both models in couples that were included in a national cohort study carried out in the Netherlands between the beginning of January 2002 and the end of December 2004.

PARTICIPANTS/MATERIALS, SETTING, METHODS

We analysed the IVF cycles of Dutch couples with primary infertility (n = 5176). The chance of success was calculated using the two UK models that had been developed using the information collected in the Human Fertilisation and Embryology Authority database. Women were treated in 1991-1994 (Templeton) or 2003-2007 (Nelson). The outcome of success for both UK models is the occurrence of a live birth after IVF but the outcome in the Dutch data is an ongoing pregnancy. In order to make the outcomes compatible, we used a factor to convert the chance of live birth to ongoing pregnancy and use the overall terms 'success or no success after IVF'. The discriminative ability and the calibration of both models were assessed, the latter before and after adjustment for time trends in IVF success rates.

MAIN RESULTS AND THE ROLE OF CHANCE

The two models showed a similarly limited degree of discriminative ability on the tested data (area under the receiver operating characteristic curve 0.597 for the Templeton model and 0.590 for the Nelson model). The Templeton model underestimated the success rate (observed 21% versus predicted 14%); the Nelson model overestimated the success rate (observed 21% versus predicted 29%). When the models were adjusted for the changing success rates over time, the calibration of both models considerably improved (Templeton observed 21% versus predicted 20%; Nelson observed 21% versus predicted 24%).

LIMITATIONS, REASONS FOR CAUTION

We could only test the models in couples with primary infertility because detailed information on secondary infertile couples was lacking in the Dutch data. This shortcoming may have negatively influenced the performance of the Nelson model.

WIDER IMPLICATIONS OF THE FINDINGS

The changes in success rates over time should be taken into account when assessing prediction models for estimating the success rate of IVF treatment. In patients with primary infertility, the choice to use the Templeton or Nelson model is arbitrary.

Predicting live birth chances for women with multiple consecutive failing IVF cycles: a simple and accurate prediction for routine medical practice

BACKGROUND

Women having experienced several consecutive failing IVF cycles constitute a critical and particular subset of patients, for which growing perception of irremediable failure, increasing costs and IVF treatment related risks necessitate appropriate decision making when starting or not a new cycle. Predicting chances of LB might constitute a useful tool for discussion between the patient and the clinician. Our essential objective was to dispose of a simple and accurate prediction model for use in routine medical practice. The currently available predictive models applicable to general populations cannot be considered as accurate enough for this purpose.

METHODS

Patients with at least four consecutive Failing cycles (CFCs) were selected. We constructed a predictive model of LB occurrence during the last cycle, by using a stepwise logistic regression, using all the baseline patient characteristics and intermediate stage variables during the four first cycles.

RESULTS

On as set of 151 patients, we identified five determinant predictors: the number of previous cycles with at least one gestational sac (NGS), the mean number of good-quality embryos, age, male infertility (MI) aetiology and basal FSH. Our model was characterized by a much higher discrimination as the existing models (C-statistics=0.76), and an excellent calibration.

CONCLUSIONS

Couples having experienced multiple IVF failures need precise and appropriate information to decide to resume or interrupt their fertility project. Our essential objective was to dispose of a simple and accurate prediction model to allow a routine practice use. Our model is adapted to this purpose: It is very simple, combines five easily collected variables in a short calculation; it is more accurate than existing models, with a fair discrimination and a well calibrated prediction.