P–016 Using artificial intelligence to predict semen upgrading after microsurgical varicocele repair

Study question

Can we use artificial intelligence models to predict semen upgrading after microsurgical varicocele repair?

Summary answer

A machine learning model performed well in predicting clinically meaningful post-varicocelectomy semen upgrade using pre-operative hormonal, clinical, and semen analysis data.

What is known already

Varicocele repair is recommended in the presence of a clinical varicocele together with at least one abnormal semen parameter, and male infertility. Unfortunately, up to 50% of men who meet criteria for repair will not see meaningful benefit in outcomes despite successful surgery. Nomograms exist to help predict success, but these are based out of single-center databases, do not incorporate hormonal data, and are rarely designed to predict pre-defined, clinically meaningful improvements in semen parameters.

Study design, size, duration

Data were collected from an international, multi-center retrospective cohort. A total of 240 men were identified. Data from 160 men from Miami, USA and 80 men from Toronto, Canada were included. Data was collected from 2006 to 2020.

Participants/materials, setting, methods

We collected pre and postoperative clinical data following varicocele surgery. Clinical upgrading was defined as an increase in sperm concentration that would allow a couple to access new reproductive technologies/techniques. The tiers used for upgrading were 0–1million/cc (Intracytoplasmic Sperm Injection), 1–5 million (In Vitro Fertilization), 5–15 million (Intrauterine Insemination), and >15 million (Natural conception). Artificial intelligence models were trained and tested using R to predict which patients upgraded after surgery.

Main results and the role of chance

51% of men underwent bilateral varicocele repair. The majority of men had grade 2 varicocele on the left, and (when present) a grade 1 varicocele on the right. Overall, 47% of men experienced an upgrade following varicocele surgery, 47% did not change, and 6% downgraded. The data from Miami were used to create a random forest model for predicting clinically significant upgrade in sperm concentration. The most informative model parameters were preoperative FSH, sperm concentration, and surgical laterality. The model identified three clinical categories: men with unfavorable, intermediate, and favorable features to predict varicocele upgrade. On external validation using data from Toronto, the model accurately predicted upgrade in 87% of men with favorable features, and in 49% and 36% of men with intermediate and unfavorable features, respectively. Overall, the model performed well on external validation with an AUC of 0.72 and good calibration.

Calibration plots, using cross-validation, define how well the predicted probabilities match the actual probability of sperm concentration upgrade. The random forest model was run twelve times. All model characteristics are the mean of ten model runs with the highest and lowest performing runs removed.

The model was translated to an online calculator that can be used by clinicians.

Limitations, reasons for caution

One limitation to our study is that we were not able to predict total motile sperm count (TMSC), which has been shown to perform slightly better than concentration at predicting assisted reproduction outcomes. By focusing on clinically significant upgrading, this difference should be minimized.

Wider implications of the findings: Predicting the chances of clinically significant semen upgrading after varicocele repair is essential for patients and clinicians to understand. Several men undergo surgery with no subsequent benefit, which may lead to a delay in definitive treatment with IVF/IUI. Understanding their chances will help couples make better informed decisions moving forward.

Trial registration number

Not applicable