The Prospective External Validation of International Ovarian Tumor Analysis (IOTA) Simple Rules in the Hands of Level I and II Examiners

Predictive accuracy and reproducibility of the O-RADS US scoring system among sonologists with different training levels

PURPOSE

To investigate the predictive performance and reproducibility of Ovarian-Adnexal Reporting and Data System (O-RADS) ultrasound (US) system in evaluating adnexal masses between sonologists with varying levels of expertise.

METHODS

This was a single-center retrospective study conducted between May 2019 and May 2020, which included 147 adnexal mases with pathological results. Four sonologists with varying experiences independently assigned an O-RADS US category to each adnexal mass twice. The intra- and inter-observer agreement was assessed using weighted kappa values. The area under the curve (AUC), sensitivity, specificity, positive and negative predictive value (PPV and NPV) were assessed for each sonologist.

RESULTS

Of the 147 adnexal mases, 115 (78.2%) lesions were benign and 32 (21.8%) lesions were malignant. Considering O-RADS > 3 as a predictor for adnexal malignancy, the predictive accuracies of the four sonologists were excellent, with AUCs ranging from 0.831 to 0.926. The predictive accuracies of O-RADS US by experienced sonologists were significantly higher compared to inexperienced sonologists (all P values < 0.005). The O-RADS US presented high sensitivity and NPV value for each sonologist. With regard to the reproducibility of O-RADS, the intra- and inter-observer agreement among experienced sonologists performed better than inexperienced sonologists.

CONCLUSION

O-RADS showed difference in the predictive accuracy and reproducibility in the evaluation of adnexal masses among sonologists with different levels of expertise. Training is required for inexperienced sonologists before the generalization of O-RADS classification system in clinical practice.

Diagnostic performance of IOTA SR and O-RADS combined with CA125, HE4, and risk of malignancy algorithm to distinguish benign and malignant adnexal masses

PURPOSE

To compare the diagnostic performance of International Ovarian Tumour Analysis Simple Rules (IOTA SR) and Ovarian-Adnexal Reporting and Data System (O-RADS), and to analyse whether combining IOTA SR and O-RADS with the biomarkers cancer antigen 125 (CA125), human epididymis protein 4 (HE4), and risk of malignancy algorithm (ROMA) further improves diagnostic performance in women with different menopause status.

METHODS

This study retrospectively included patients with ovarian adnexal masses confirmed by surgical pathology between September 2021 and February 2022. The area under the curve (AUC), sensitivity, and specificity were calculated to evaluate the diagnostic efficacy of IOTA SR, O-RADS, and their combination with CA125, HE4, and ROMA.

RESULTS

This study included 1,179 ovarian adnexal masses. In all women, the AUC of IOTA SR was comparable to O-RADS (0.879 vs. 0.889, P = 0.361), and O-RADS had a significantly higher sensitivity than IOTA SR (95.77 % vs. 87.32 %, P < 0.001). In premenopausal women, O-RADS had a significantly higher AUC than other diagnostic strategies (all P < 0.05), and the sensitivity, specificity, and accuracy were 93.33 %, 84.74 %, and 85.59 %, respectively. In postmenopausal women, IOTA SR + ROMA had a significantly higher AUC than other diagnostic strategies (all P < 0.05), and the sensitivity, specificity, and accuracy were 85.37 %, 93.88 %, and 90.00 %, respectively.

CONCLUSIONS

Our study supports the high diagnostic value of IOTA SR or O-RADS alone in all women, and O-RADS was more sensitive than IOTA SR. In premenopausal women, O-RADS had the highest diagnostic value. In postmenopausal women, IOTA SR outperformed O-RADS, and IOTA SR + ROMA had the highest diagnostic value.

Comparison of HE4, CA125, ROMA and CPH-I for Preoperative Assessment of Adnexal Tumors

(1) OBJECTIVE: To assess the performance of CA125, HE4, ROMA index and CPH-I index to preoperatively identify epithelial ovarian cancer (EOC) or metastatic cancer in the ovary (MCO). (2) METHODS: single center retrospective study, including women with a diagnosis of adnexal mass. We obtained the AUC, sensitivity, specificity and predictive values were of HE4, CA125, ROMA and CPH-I for the diagnosis of EOC and MCO. Subgroup analysis for women harboring adnexal masses with inconclusive diagnosis of malignancy by ultrasound features and Stage I EOC was performed. (3) RESULTS: 1071 patients were included, 852 (79.6%) presented benign/borderline tumors and 219 (20.4%) presented EOC/MCO. AUC for HE4 was higher than for CA125 (0.91 vs. 0.87). No differences were seen between AUC of ROMA and CPH-I, but they were both higher than HE4 AUC. None of the tumor markers alone achieved a sensitivity of 90%; HE4 was highly specific (93.5%). ROMA showed a sensitivity and specificity of 91.1% and 84.6% respectively, while CPH-I showed a sensitivity of 91.1% with 79.2% specificity. For patients with inconclusive diagnosis of malignancy by ultrasound features and with Stage I EOC, ROMA showed the best diagnostic performance (4) CONCLUSIONS: ROMA and CPH-I perform better than tumor markers alone to identify patients harboring EOC or MCO. They can be helpful to assess the risk of malignancy of adnexal masses, especially in cases where ultrasonographic diagnosis is challenging (stage I EOC, inconclusive diagnosis of malignancy by ultrasound features).

Ultrasound Assessment of Adnexal Pathology: Standardized Methods and Different Levels of Experience

Background and objectives: An expert’s subjective assessment is still the most reliable evaluation of adnexal pathology, thus raising the need for methods less dependent on the examiner’s experience. The aim of this study was to evaluate the performance of standardized methods when applied by examiners with different levels of experience and to suggest the most suitable method for less-experienced gynecologists. Materials and methods: This single-center retrospective study included 50 cases of histologically proven first-time benign or malignant adnexal pathology. Three examiners evaluated the same transvaginal ultrasound images: an expert (level III), a 4th year resident in gynecology (level I), and a final year medical student after basic training (labeled as level 0). The assessment methods included subjective evaluation, Simple Rules (SR) with and without algorithm, ADNEX and Gynecologic Imaging Reporting and Data System (GI-RADS) models. Sensitivity, specificity, accuracy, positive and negative predictive values with 95% confidence interval were calculated. Results: Out of 50 cases, 33 (66%) were benign and 17 (34%) were malignant adnexal masses. Using only SR, level III could classify 48 (96%), level I—41 (82%) and level 0—40 (80%) adnexal lesions. Using SR and algorithm, the performance improved the most for all levels and yielded sensitivity and specificity of 100% for level III, 100% and 97% for level I, 94.4% and 100% for level 0, respectively. Compared to subjective assessment, ADNEX lowered the accuracy of level III evaluation from 97.9% to 88% and GI-RADS had no impact. ADNEX and GI-RADS improved the sensitivity up to 100% for the less experienced; however, the specificity and accuracy were notably decreased. Conclusions: SR and SR+ algorithm have the most potential to improve not only sensitivity, but also specificity and accuracy, irrespective of the experience level. ADNEX and GI-RADS can yield sensitivity of 100%; however, the accuracy is decreased.

ESGO/ISUOG/IOTA/ESGE Consensus Statement on preoperative diagnosis of ovarian tumours

The European Society of Gynaecological Oncology (ESGO), the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG), the International Ovarian Tumour Analysis (IOTA) group and the European Society for Gynaecological Endoscopy (ESGE) jointly developed clinically relevant and evidence-based statements on the preoperative diagnosis of ovarian tumours, including imaging techniques, biomarkers and prediction models.

ESGO/ISUOG/IOTA/ESGE nominated a multidisciplinary international group, including expert practising clinicians and researchers who have demonstrated leadership and expertise in the preoperative diagnosis of ovarian tumours and management of patients with ovarian cancer (19 experts across Europe). A patient representative was also included in the group. To ensure that the statements were evidence-based, the current literature was reviewed and critically appraised.

Preliminary statements were drafted based on the review of the relevant literature. During a conference call, the whole group discussed each preliminary statement and a first round of voting was carried out. Statements were removed when a consensus among group members was not obtained. The voters had the opportunity to provide comments/suggestions with their votes. The statements were then revised accordingly. Another round of voting was carried out according to the same rules to allow the whole group to evaluate the revised version of the statements. The group achieved consensus on 18 statements.

This Consensus Statement presents these ESGO/ISUOG/IOTA/ESGE statements on the preoperative diagnosis of ovarian tumours and the assessment of carcinomatosis, together with a summary of the evidence supporting each statement.

ESGO/ISUOG/IOTA/ESGE Consensus Statement on pre-operative diagnosis of ovarian tumors

The European Society of Gynaecological Oncology (ESGO), the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG), the International Ovarian Tumour Analysis (IOTA) group, and the European Society for Gynaecological Endoscopy (ESGE) jointly developed clinically relevant and evidence-based statements on the pre-operative diagnosis of ovarian tumors, including imaging techniques, biomarkers, and prediction models. ESGO/ISUOG/IOTA/ESGE nominated a multidisciplinary international group, including expert practising clinicians and researchers who have demonstrated leadership and expertise in the pre-operative diagnosis of ovarian tumors and management of patients with ovarian cancer (19 experts across Europe). A patient representative was also included in the group. To ensure that the statements were evidence-based, the current literature was reviewed and critically appraised. Preliminary statements were drafted based on the review of the relevant literature. During a conference call, the whole group discussed each preliminary statement and a first round of voting was carried out. Statements were removed when a consensus among group members was not obtained. The voters had the opportunity to provide comments/suggestions with their votes. The statements were then revised accordingly. Another round of voting was carried out according to the same rules to allow the whole group to evaluate the revised version of the statements. The group achieved consensus on 18 statements. This Consensus Statement presents these ESGO/ISUOG/IOTA/ESGE statements on the pre-operative diagnosis of ovarian tumors and the assessment of carcinomatosis, together with a summary of the evidence supporting each statement.

Sonographic Assessment of Complex Ultrasound Morphology Adnexal Tumors in Pregnant Women with the Use of IOTA Simple Rules Risk and ADNEX Scoring Systems

Background: To evaluate the accuracy of subjective assessment (SA), the International Ovarian Tumor Analysis (IOTA) group Simple Rules Risk (SRR) and the Assessment of Different NEoplasias in the adneXa (ADNEX) model for the preoperative differentiation of adnexal masses in pregnant women. Methods: The study population comprised 36 pregnant women (median age: 28.5 years old, range: 20–42 years old) with a mean gestation age of 13.5 (range: 8–31) weeks at diagnosis. Tumors were prospectively classified by local sonographers as probably benign or probably malignant using SA. Final tumor histological diagnosis was used as the reference standard in all cases. Logistic regression SRR and ADNEX models were used to obtain a risk score for every case. Serum CA125 and human epidydimis protein 4 (HE4) concentrations were also retrieved and the Risk of Ovarian Malignancy Algorithm (ROMA) value was calculated. The calculated predictive values included positive and negative likelihood ratios of ultrasound and biochemical tests. Results: Final histology confirmed 27 benign and 9 malignant (including 2 borderline) masses. The highest sensitivity (89%) and specificity (70%) were found for the subjective tumor assessment. Although no malignancy was classified as benign using the SRR criteria (sensitivity = 100%), the specificity of this scoring system was only 37%. At the cut-off risk level of >20%, the ADNEX model had a sensitivity of 78% and a specificity of 70%. Serum levels of CA125, HE4 and the ROMA risk model correctly identified adnexal malignant tumors with a sensitivity of 67%, 25% and 25%, respectively. Corresponding specificities were 72%, 100% and 100%, respectively. The highest positive and negative likelihood ratios were found for SA (LR+ = 3.0 and LR− = 0.16, respectively). Overall diagnostic accuracy of all predictive methods used in this study were similar (range: 70–75%) except for SRR (53%). Conclusion: Subjective assessment remains the best predictive method in complex adnexal masses found at prenatal ultrasound in pregnant women. For less experienced sonographers, both the SRR and ADNEX scoring systems may be also used for the characterization of such tumors, while serum tumor markers CA125 and HE4, along with the ROMA algorithm appear to be less accurate.

Risk scores to guide referral decisions for people with suspected ovarian cancer in secondary care: a systematic review and cost-effectiveness analysis

BACKGROUND

Ovarian cancer is the sixth most common cancer in UK women and can be difficult to diagnose, particularly in the early stages. Risk-scoring can help to guide referral to specialist centres.

OBJECTIVES

To assess the clinical and cost-effectiveness of risk scores to guide referral decisions for women with suspected ovarian cancer in secondary care.

METHODS

Twenty-one databases, including MEDLINE and EMBASE, were searched from inception to November 2016. Review methods followed published guidelines. The meta-analysis using weighted averages and random-effects modelling was used to estimate summary sensitivity and specificity with 95% confidence intervals (CIs). The cost-effectiveness analysis considered the long-term costs and quality-adjusted life-years (QALYs) associated with different risk-scoring methods, and subsequent care pathways. Modelling comprised a decision tree and a Markov model. The decision tree was used to model short-term outcomes and the Markov model was used to estimate the long-term costs and QALYs associated with treatment and progression.

RESULTS

Fifty-one diagnostic cohort studies were included in the systematic review. The Risk of Ovarian Malignancy Algorithm (ROMA) score did not offer any advantage over the Risk of Malignancy Index 1 (RMI 1). Patients with borderline tumours or non-ovarian primaries appeared to account for disproportionately high numbers of false-negative, low-risk ROMA scores. (Confidential information has been removed.) To achieve similar levels of sensitivity to the Assessment of Different NEoplasias in the adneXa (ADNEX) model and the International Ovarian Tumour Analysis (IOTA) group's simple ultrasound rules, a very low RMI 1 decision threshold (25) would be needed; the summary sensitivity and specificity estimates for the RMI 1 at this threshold were 94.9% (95% CI 91.5% to 97.2%) and 51.1% (95% CI 47.0% to 55.2%), respectively. In the base-case analysis, RMI 1 (threshold of 250) was the least effective [16.926 life-years (LYs), 13.820 QALYs] and the second cheapest (£5669). The IOTA group's simple ultrasound rules (inconclusive, assumed to be malignant) were the cheapest (£5667) and the second most effective [16.954 LYs, 13.841 QALYs], dominating RMI 1. The ADNEX model (threshold of 10%), costing £5699, was the most effective (16.957 LYs, 13.843 QALYs), and compared with the IOTA group's simple ultrasound rules, resulted in an incremental cost-effectiveness ratio of £15,304 per QALY gained. At thresholds of up to £15,304 per QALY gained, the IOTA group's simple ultrasound rules are cost-effective; the ADNEX model (threshold of 10%) is cost-effective for higher thresholds.

LIMITATIONS

Information on the downstream clinical consequences of risk-scoring was limited.

CONCLUSIONS

Both the ADNEX model and the IOTA group's simple ultrasound rules may offer increased sensitivity relative to current practice (RMI 1); that is, more women with malignant tumours would be referred to a specialist multidisciplinary team, although more women with benign tumours would also be referred. The cost-effectiveness model supports prioritisation of sensitivity over specificity. Further research is needed on the clinical consequences of risk-scoring.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42016053326.

FUNDING DETAILS

The National Institute for Health Research Health Technology Assessment programme.

Validation of the Performance of International Ovarian Tumor Analysis (IOTA) Methods in the Diagnosis of Early Stage Ovarian Cancer in a Non-Screening Population

Background: The aim of this study was to assess and compare the performance of different ultrasound-based International Ovarian Tumor Analysis (IOTA) strategies and subjective assessment for the diagnosis of early stage ovarian malignancy. Methods: This is a secondary analysis of a prospective multicenter cross-sectional diagnostic accuracy study that included 1653 patients recruited at 18 centers from 2009 to 2012. All patients underwent standardized transvaginal ultrasonography by experienced ultrasound investigators. We assessed test performance of the IOTA Simple Rules (SRs), Simple Rules Risk (SRR), the Assessment of Different NEoplasias in the adneXa (ADNEX) model and subjective assessment to discriminate between stage I-II ovarian cancer and benign disease. Reference standard was histology after surgery. Results: 230 (13.9%) patients proved to have stage I–II primary invasive ovarian malignancy, and 1423 (86.1%) had benign disease. Sensitivity and specificity with respect to malignancy (95% confidence intervals) of the original SRs (classifying all inconclusive cases as malignant) were 94.3% (90.6% to 96.7%) and 73.4% (71.0% to 75.6%). Subjective assessment had a sensitivity and specificity of 90.0% (85.4% to 93.2%) and 86.7% (84.9% to 88.4%), respectively. The areas under the receiver operator characteristic curves of SRR and ADNEX were 0.917 (0.902 to 0.933) and 0.905 (0.920 to 0.934), respectively. At a 1% risk cut-off, sensitivity and specificity for SRR were 100% (98.4% to 100%) and 38.0% (35.5% to 40.6%), and for ADNEX were 100% (98.4% to 100%) and 19.4% (17.4% to 21.5%). At a 30% risk cut-off, sensitivity and specificity for SRR were 88.3% (83.5% to 91.8%) and 81.1% (79% to 83%), and for ADNEX were 84.5% (80.5% to 89.6%) and 84.5% (82.6% to 86.3%). Conclusion: This study shows that all three IOTA strategies have good ability to discriminate between stage I-II ovarian malignancy and benign disease.