Clinical Utility of Risk Models to Refer Patients with Adnexal Masses to Specialized Oncology Care: Multicenter External Validation Using Decision Curve Analysis

Comparison of the ADNEX and ROMA risk prediction models for the diagnosis of ovarian cancer: a multicentre external validation in patients who underwent surgery

BACKGROUND

Several diagnostic prediction models to help clinicians discriminate between benign and malignant adnexal masses are available. This study is a head-to-head comparison of the performance of the Assessment of Different NEoplasias in the adneXa (ADNEX) model with that of the Risk of Ovarian Malignancy Algorithm (ROMA).

METHODS

This is a retrospective study based on prospectively included consecutive women with an adnexal tumour scheduled for surgery at five oncology centres and one non-oncology centre in four countries between 2015 and 2019. The reference standard was histology. Model performance for ADNEX and ROMA was evaluated regarding discrimination, calibration, and clinical utility.

RESULTS

The primary analysis included 894 patients, of whom 434 (49%) had a malignant tumour. The area under the receiver operating characteristic curve (AUC) was 0.92 (95% CI 0.88-0.95) for ADNEX with CA125, 0.90 (0.84-0.94) for ADNEX without CA125, and 0.85 (0.80-0.89) for ROMA. ROMA, and to a lesser extent ADNEX, underestimated the risk of malignancy. Clinical utility was highest for ADNEX. ROMA had no clinical utility at decision thresholds <27%.

CONCLUSIONS

ADNEX had better ability to discriminate between benign and malignant adnexal tumours and higher clinical utility than ROMA.

CLINICAL TRIAL REGISTRATION

clinicaltrials.gov NCT01698632 and NCT02847832.

ADNEX risk prediction model for diagnosis of ovarian cancer: systematic review and meta-analysis of external validation studies

Objectives

To conduct a systematic review of studies externally validating the ADNEX (Assessment of Different Neoplasias in the adnexa) model for diagnosis of ovarian cancer and to present a meta-analysis of its performance.

Design

Systematic review and meta-analysis of external validation studies.

Data sources

Medline, Embase, Web of Science, Scopus, and Europe PMC, from 15 October 2014 to 15 May 2023.

Eligibility criteria for selecting studies

All external validation studies of the performance of ADNEX, with any study design and any study population of patients with an adnexal mass. Two independent reviewers extracted the data. Disagreements were resolved by discussion. Reporting quality of the studies was scored with the TRIPOD (Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis) reporting guideline, and methodological conduct and risk of bias with PROBAST (Prediction model Risk Of Bias Assessment Tool). Random effects meta-analysis of the area under the receiver operating characteristic curve (AUC), sensitivity and specificity at the 10% risk of malignancy threshold, and net benefit and relative utility at the 10% risk of malignancy threshold were performed.

Results

47 studies (17 007 tumours) were included, with a median study sample size of 261 (range 24-4905). On average, 61% of TRIPOD items were reported. Handling of missing data, justification of sample size, and model calibration were rarely described. 91% of validations were at high risk of bias, mainly because of the unexplained exclusion of incomplete cases, small sample size, or no assessment of calibration. The summary AUC to distinguish benign from malignant tumours in patients who underwent surgery was 0.93 (95% confidence interval 0.92 to 0.94, 95% prediction interval 0.85 to 0.98) for ADNEX with the serum biomarker, cancer antigen 125 (CA125), as a predictor (9202 tumours, 43 centres, 18 countries, and 21 studies) and 0.93 (95% confidence interval 0.91 to 0.94, 95% prediction interval 0.85 to 0.98) for ADNEX without CA125 (6309 tumours, 31 centres, 13 countries, and 12 studies). The estimated probability that the model has use clinically in a new centre was 95% (with CA125) and 91% (without CA125). When restricting analysis to studies with a low risk of bias, summary AUC values were 0.93 (with CA125) and 0.91 (without CA125), and estimated probabilities that the model has use clinically were 89% (with CA125) and 87% (without CA125).

Conclusions

The results of the meta-analysis indicated that ADNEX performed well in distinguishing between benign and malignant tumours in populations from different countries and settings, regardless of whether the serum biomarker, CA125, was used as a predictor. A key limitation was that calibration was rarely assessed.

Systematic review registration

PROSPERO CRD42022373182.

Comparative diagnostic accuracy of the IOTA SRR and LR2 scoring systems for discriminating between malignant and Benign Adnexal masses by junior physicians in Chinese patients: a retrospective observational study

BACKGROUND

The accuracy of ultrasound in distinguishing benign from malignant adnexal masses is highly correlated with the experience of ultrasound physicians. In China, most of ultrasound differentiation is done by junior physicians.

PURPOSE

To compare the diagnostic performance of the International Ovarian Tumour Analysis (IOTA) Simple Rules Risk (SRR) and IOTA Logistic Regression Model 2 (LR2) scoring systems in Chinese patients with adnexal masses.

METHODS

Retrospective analysis of ovarian cancer tumor patients who underwent surgery at a hospital in China from January 2016 to December 2021. Screening patients with at least one adnexal mass on inclusion and exclusion criteria. Two trained junior physicians evaluated each mass using the two scoring systems. A receiver operating characteristic curve was used to test the diagnostic performance of each system.

RESULTS

A total of 144 adnexal masses were retrospectively collected. Forty masses were histologically diagnosed as malignant. Compared with premenopausal women, postmenopausal women had a much higher rate of malignant masses. The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) of the SRR was 97.5% (95% CI: 86.8 -99.9%), 82.7% (95% CI: 74.0 -89.4%), 68.4% (95% CI: 58.7 -76.8%) and 98.9% (95% CI: 92.5 -99.8%). The sensitivity, specificity, PPV, NPV of the LR2 were 90.0% (95% CI: 76.5 -97.2%), 89.4% (95% CI: 81.9 -94.6%), 76.6% (95% CI: 65.0 -85.2%), and 95.9% (95% CI: 90.2 -98.3%). There was good agreement between two scoring systems, with 84.03% total agreement and a kappa value of 0.783 (95% CI: 0.70-0.864). The areas under the curve for predicting malignant tumours using SRR and LR2 were similar for all patients (P > 0.05 ).

CONCLUSION

The two scoring systems can effectively distinguish benign from malignant adnexal masses. Both scoring systems have high diagnostic efficacy, and diagnostic efficacy is stable, which can provide an important reference for clinical decision making.

Subjective assessment and IOTA ADNEX model in evaluation of adnexal masses in patients with history of breast cancer

OBJECTIVE

To evaluate the performance of subjective assessment and the Assessment of Different NEoplasias in the adneXa (ADNEX) model in discriminating between benign and malignant adnexal tumors and between metastatic and primary adnexal tumors in patients with a personal history of breast cancer.

METHODS

This was a retrospective single-center study including patients with a history of breast cancer who underwent surgery for an adnexal mass between 2013 and 2020. All patients had been examined with transvaginal or transrectal ultrasound using a standardized examination technique and all ultrasound reports had been stored and were retrieved for the purposes of this study. The specific diagnosis suggested by the original ultrasound examiner in the retrieved report was analyzed. For each mass, the ADNEX model risks were calculated prospectively and the highest relative risk was used to categorize each into one of five categories (benign, borderline, primary Stage I, primary Stages II-IV or metastatic ovarian cancer) for analysis of the ADNEX model in predicting the specific tumor type. The performance of subjective assessment and the ADNEX model in discriminating between benign and malignant adnexal tumors and between primary and metastatic adnexal tumors was evaluated, using final histology as the reference standard.

RESULTS

Included in the study were 202 women with a history of breast cancer who underwent surgery for an adnexal mass. At histology, 93/202 (46.0%) masses were benign, 76/202 (37.6%) were primary malignancies (four borderline and 72 invasive tumors) and 33/202 (16.3%) were metastases. The original ultrasound examiner classified correctly 79/93 (84.9%) benign adnexal masses, 72/76 (94.7%) primary adnexal malignancies and 30/33 (90.9%) metastatic tumors. Subjective ultrasound evaluation had a sensitivity of 93.6%, specificity of 84.9% and accuracy of 89.6%, while the ADNEX model had higher sensitivity (98.2%) but lower specificity (78.5%), with similar accuracy (89.1%), in discriminating between benign and malignant ovarian masses. Subjective evaluation had a sensitivity of 51.5%, specificity of 88.8% and accuracy of 82.7% in distinguishing metastatic and primary tumors (including benign, borderline and invasive tumors), and the ADNEX model had a sensitivity of 63.6%, specificity of 84.6% and similar accuracy (81.2%).

CONCLUSIONS

The performance of subjective assessment and the ADNEX model in discriminating between benign and malignant adnexal masses in this series of patients with history of breast cancer was relatively similar. Both subjective assessment and the ADNEX model demonstrated good accuracy and specificity in discriminating between metastatic and primary tumors, but the sensitivity was low. © 2023 International Society of Ultrasound in Obstetrics and Gynecology.

Diagnostic performance of IOTA SR and O-RADS combined with CA125, HE4, and risk of malignancy algorithm to distinguish benign and malignant adnexal masses

PURPOSE

To compare the diagnostic performance of International Ovarian Tumour Analysis Simple Rules (IOTA SR) and Ovarian-Adnexal Reporting and Data System (O-RADS), and to analyse whether combining IOTA SR and O-RADS with the biomarkers cancer antigen 125 (CA125), human epididymis protein 4 (HE4), and risk of malignancy algorithm (ROMA) further improves diagnostic performance in women with different menopause status.

METHODS

This study retrospectively included patients with ovarian adnexal masses confirmed by surgical pathology between September 2021 and February 2022. The area under the curve (AUC), sensitivity, and specificity were calculated to evaluate the diagnostic efficacy of IOTA SR, O-RADS, and their combination with CA125, HE4, and ROMA.

RESULTS

This study included 1,179 ovarian adnexal masses. In all women, the AUC of IOTA SR was comparable to O-RADS (0.879 vs. 0.889, P = 0.361), and O-RADS had a significantly higher sensitivity than IOTA SR (95.77 % vs. 87.32 %, P < 0.001). In premenopausal women, O-RADS had a significantly higher AUC than other diagnostic strategies (all P < 0.05), and the sensitivity, specificity, and accuracy were 93.33 %, 84.74 %, and 85.59 %, respectively. In postmenopausal women, IOTA SR + ROMA had a significantly higher AUC than other diagnostic strategies (all P < 0.05), and the sensitivity, specificity, and accuracy were 85.37 %, 93.88 %, and 90.00 %, respectively.

CONCLUSIONS

Our study supports the high diagnostic value of IOTA SR or O-RADS alone in all women, and O-RADS was more sensitive than IOTA SR. In premenopausal women, O-RADS had the highest diagnostic value. In postmenopausal women, IOTA SR outperformed O-RADS, and IOTA SR + ROMA had the highest diagnostic value.

Profiling the metabolome of uterine fluid for early detection of ovarian cancer

Ovarian cancer (OC) causes high mortality in women because of ineffective biomarkers for early diagnosis. Here, we perform metabolomics analysis on an initial training set of uterine fluid from 96 gynecological patients. A seven-metabolite-marker panel consisting of vanillylmandelic acid, norepinephrine, phenylalanine, beta-alanine, tyrosine, 12-S-hydroxy-5,8,10-heptadecatrienoic acid, and crithmumdiol is established for detecting early-stage OC. The panel is further validated in an independent sample set from 123 patients, discriminating early OC from controls with an area under the curve (AUC) of 0.957 (95% confidence interval [CI], 0.894-1). Interestingly, we find elevated norepinephrine and decreased vanillylmandelic acid in most OC cells, resulting from excess 4-hydroxyestradiol that antagonizes the catabolism of norepinephrine by catechol-O-methyltransferase. Moreover, exposure to 4-hydroxyestradiol induces cellular DNA damage and genomic instability that could lead to tumorigenesis. Thus, this study not only reveals metabolic features in uterine fluid of gynecological patients but also establishes a noninvasive approach for the early diagnosis of OC.

Value of Assessment of Different Neoplasias in the Adnexa in the Differential Diagnosis of Malignant Ovarian Tumor and Benign Ovarian Tumor: A Meta-analysis

To evaluate the accuracy of the assessment of different neoplasias in the adnexa (ADNEX) model in the differential diagnosis of malignant and benign ovarian tumors, the optimal cutoff value and the accuracy in diagnosing ovarian tumors at different stages, PubMed, Web of Science and Cochrane Library databases were retrieved to search literature with per-patient analysis until publication of the last study in November 2021. STATA 14.1, Meta-Disc 1.4 and Revman software 5.3 were used in the performance of meta-analysis. To explore sources of heterogeneity, a subgroup analysis was conducted for the ADNEX model. The pooled sensitivity, specificity, diagnostic odds ratio, positive likelihood, negative likelihood ratio and area under the summary receiver operating characteristic curve were 0.91 (95% confidence interval [CI]: 0.89-0.93), 0.84 (95% CI: 0.80-0.88), 55.55 (95% CI: 40.47-76.26), 5.71 (95% CI: 4.49-7.26), 0.10 (95% CI: 0.08-0.13) and 0.94 (95% CI: 0.92-0.96) in differentiating benign and malignant ovarian tumors, respectively. The area under the curve in identifying benign, borderline, stage I and stages II-IV were 0.93, 0.73, 0.27 and 0.92. The ADNEX model had high diagnostic performance was influential in the diagnosis of benign and stage II-IV ovarian tumors.

Evaluation of Prognostic and Predictive Models in the Oncology Clinic

Predictive and prognostic models hold great potential to support clinical decision making in oncology and could ultimately facilitate a paradigm shift to a more personalised form of treatment. While a large number of models relevant to the field of oncology have been developed, few have been translated into clinical use and assessment of clinical utility is not currently considered a routine part of model development. In this narrative review of the clinical evaluation of prediction models in oncology, we propose a high-level process diagram for the life cycle of a clinical model, encompassing model commissioning, clinical implementation and ongoing quality assurance, which aims to bridge the gap between model development and clinical implementation.

Two-Step Strategy for Optimizing the Preoperative Classification of Adnexal Masses in a University Hospital, Using International Ovarian Tumor Analysis Models: Simple Rules and Assessment of Different NEoplasias in the adneXa Model

OBJECTIVES

To evaluate the performance of a two-step strategy compared with the International Ovarian Tumor Analysis (IOTA) - Assessment of Different NEoplasias in the adneXa (ADNEX) model for preoperative classification of adnexal masses.

METHODS

An ambispective diagnostic accuracy study based on ultrasound data collected at one university hospital between 2012 and 2018. Two ultrasonographers classified the adnexal masses using IOTA Simple Rules (first step). Not classifiable masses were evaluated using the IOTA ADNEX model (second step). Also, all masses were classified using the IOTA ADNEX model. The sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV), positive likelihood ratio (LR+) and negative likelihood ratio (LR-), and receiver operating characteristic (ROC) curve were estimated. A P value of <.05 was used to determine statistical significance.

RESULTS

The study included 548 patients and 606 masses. Patients' median age was 41 years with an interquartile range between 32 and 51 years. In the first step, 89 (14%) masses were not classifiable. In the second step, 55 (61.8%) masses were classified as malignant. Furthermore, for the totality of 606 masses, the IOTA ADNEX model estimated the probability that 126 (20.8%) masses were malignant. The two-step strategy had a sensitivity, specificity, PPV, NPV, LR+, LR-, and ROC curve of 86.8%, 91.01%, 51.9%, 98.4%, 9.7, 0.1, and 0.889, respectively; compared to IOTA ADNEX model that had values of 91.8%, 87.16%, 44.4%, 99%, 7.1, 0.09, and 0.895, respectively.

CONCLUSION

The two-step strategy shows a similar diagnostic performance when compared to the IOTA ADNEX model. The IOTA ADNEX model involves only one step and can be more practical, and thus would be recommended to use.

Evaluation of the Diagnostic Value of the Ultrasound ADNEX Model for Benign and Malignant Ovarian Tumors

Objective

To investigate the diagnostic performance of the ADNEX model in the International Ovarian Tumor Analysis diagnostic models for ovarian tumors and further explore its application value in the staging of ovarian tumors.

Methods

A total of 224 patients who underwent ultrasound for evaluation of adnexal masses and were treated surgically owing to adnexal masses from January 2018 to June 2020 in our hospital were selected for research on the diagnostic accuracy of the ADNEX model. The clinical information and ultrasonographic findings of the patients were collected, and the pathological diagnosis was taken as the gold standard. According to the ADNEX model, the ovarian tumors were divided into five subtypes: benign and borderline, stage I, stage II–IV, and metastatic cancer. The sensitivity, specificity, positive predictive value, negative predictive value, diagnostic odds ratio, and area under the receiver operating characteristics curve (AUC) of the ADNEX model were calculated.

Results

Of the 224 patients, 119 (53.1%) developed benign tumors and 105 (46.9%) had malignant tumors. When the cut-off value for malignancy risk was 10%, the ADNEX model including CA 125 achieved a sensitivity of 94.3% (95% CI: 88.0–97.9%), specificity of 74.0% (95% CI: 65.1–81.6%), positive predictive value of 76.2% (95% CI: 70.2–81.3%), negative predictive value of 93.6% (95% CI: 87.0–97.0%), diagnostic odds ratio of 45.25, and an AUC of 0.94 (95% CI: 0.90–0.97) for differentiating between benign and malignant ovarian tumors. The AUC in the model excluding CA 125 was 0.93 (95% CI: 0.89–0.96), but the difference was not statistically significant (P=0.20). The accuracy of the ADNEX model for the diagnosis of ovarian tumors of all subtypes exceeds 80% when CA 125 measurements were included in the application, but the sensitivity for diagnosing borderline, stage I, and metastatic ovarian tumors was only 60.0% (95% CI:36.1–80.9%), 28.6% (95% CI:8.4–58.1%) and 45.5% (95% CI:16.7–76.6%).

Conclusion

The ADNEX model shows good diagnostic performance in differentiating between benign and malignant ovarian tumors. The model has a certain clinical value in the diagnosis of all subtypes of ovarian tumors, but the sensitivity is unsatisfactory for the diagnosis of borderline, stage I, and metastatic ovarian tumors and needs to be verified.

ESGO/ISUOG/IOTA/ESGE Consensus Statement on preoperative diagnosis of ovarian tumors

The European Society of Gynaecological Oncology (ESGO), the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG), the International Ovarian Tumour Analysis (IOTA) group and the European Society for Gynaecological Endoscopy (ESGE) jointly developed clinically relevant and evidence-based statements on the preoperative diagnosis of ovarian tumors, including imaging techniques, biomarkers and prediction models. ESGO/ISUOG/IOTA/ESGE nominated a multidisciplinary international group, including expert practising clinicians and researchers who have demonstrated leadership and expertise in the preoperative diagnosis of ovarian tumors and management of patients with ovarian cancer (19 experts across Europe). A patient representative was also included in the group. To ensure that the statements were evidence-based, the current literature was reviewed and critically appraised. Preliminary statements were drafted based on the review of the relevant literature. During a conference call, the whole group discussed each preliminary statement and a first round of voting was carried out. Statements were removed when consensus among group members was not obtained. The voters had the opportunity to provide comments/suggestions with their votes. The statements were then revised accordingly. Another round of voting was carried out according to the same rules to allow the whole group to evaluate the revised version of the statements. The group achieved consensus on 18 statements. This Consensus Statement presents these ESGO/ISUOG/IOTA/ESGE statements on the preoperative diagnosis of ovarian tumors and the assessment of carcinomatosis, together with a summary of the evidence supporting each statement.

Diagnostic Accuracy of the ADNEX Model for Ovarian Cancer at the 15% Cut-Off Value: A Systematic Review and Meta-Analysis

Objectives

To evaluate the diagnostic accuracy of the ADNEX model for ovarian cancer at the 15% cut-off value.

Methods

Studies on the identified diagnosis of the ADNEX model for ovarian cancer published in PubMed, Embase, the Cochrane Library and Web of Science databases from January 1st, 2014 to February 20th, 2021 were searched. Two researchers independently screened the retrieved studies and extracted the basic features and parameter data. The quality of the eligible studies was evaluated by Quality Assessment of Diagnostic Accuracy Studies-2, and the result was summarized by Review Manager 5.3. Meta-Disc 1.4 and STATA 16.0 were used in statistical analysis. Heterogeneity of this meta-analysis was calculated. Meta-regression was performed to investigate the potential sources of heterogeneity. Sensitivity analysis and Deek's funnel plot analysis were conducted to evaluate the stability and publication bias, respectively.

Results

280 studies were initially retrieved through the search strategy, and 10 eligible studies were ultimately included. The random-effects model was selected for data synthesis. The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio and the area under the summary receiver operating characteristic curve were 0.92 (95% CI: 0.89-0.94), 0.82 (95% CI: 0.78-0.86), 5.2 (95% CI: 4.1-6.4), 0.10 (95% CI: 0.07-0.13), 54.0 (95% CI: 37.0-77.0) and 0.95 (95% CI: 0.91-0.95). Meta-regression based on study design, country, enrollment and blind method was not statistically significant. This meta-analysis was stable with no obvious publication bias.

Conclusions

The ADNEX model at the 15% cut-off had high diagnostic accuracy in identifying ovarian cancer.

ESGO/ISUOG/IOTA/ESGE Consensus Statement on preoperative diagnosis of ovarian tumours

The European Society of Gynaecological Oncology (ESGO), the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG), the International Ovarian Tumour Analysis (IOTA) group and the European Society for Gynaecological Endoscopy (ESGE) jointly developed clinically relevant and evidence-based statements on the preoperative diagnosis of ovarian tumours, including imaging techniques, biomarkers and prediction models.

ESGO/ISUOG/IOTA/ESGE nominated a multidisciplinary international group, including expert practising clinicians and researchers who have demonstrated leadership and expertise in the preoperative diagnosis of ovarian tumours and management of patients with ovarian cancer (19 experts across Europe). A patient representative was also included in the group. To ensure that the statements were evidence-based, the current literature was reviewed and critically appraised.

Preliminary statements were drafted based on the review of the relevant literature. During a conference call, the whole group discussed each preliminary statement and a first round of voting was carried out. Statements were removed when a consensus among group members was not obtained. The voters had the opportunity to provide comments/suggestions with their votes. The statements were then revised accordingly. Another round of voting was carried out according to the same rules to allow the whole group to evaluate the revised version of the statements. The group achieved consensus on 18 statements.

This Consensus Statement presents these ESGO/ISUOG/IOTA/ESGE statements on the preoperative diagnosis of ovarian tumours and the assessment of carcinomatosis, together with a summary of the evidence supporting each statement.

ESGO/ISUOG/IOTA/ESGE Consensus Statement on pre-operative diagnosis of ovarian tumors

The European Society of Gynaecological Oncology (ESGO), the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG), the International Ovarian Tumour Analysis (IOTA) group, and the European Society for Gynaecological Endoscopy (ESGE) jointly developed clinically relevant and evidence-based statements on the pre-operative diagnosis of ovarian tumors, including imaging techniques, biomarkers, and prediction models. ESGO/ISUOG/IOTA/ESGE nominated a multidisciplinary international group, including expert practising clinicians and researchers who have demonstrated leadership and expertise in the pre-operative diagnosis of ovarian tumors and management of patients with ovarian cancer (19 experts across Europe). A patient representative was also included in the group. To ensure that the statements were evidence-based, the current literature was reviewed and critically appraised. Preliminary statements were drafted based on the review of the relevant literature. During a conference call, the whole group discussed each preliminary statement and a first round of voting was carried out. Statements were removed when a consensus among group members was not obtained. The voters had the opportunity to provide comments/suggestions with their votes. The statements were then revised accordingly. Another round of voting was carried out according to the same rules to allow the whole group to evaluate the revised version of the statements. The group achieved consensus on 18 statements. This Consensus Statement presents these ESGO/ISUOG/IOTA/ESGE statements on the pre-operative diagnosis of ovarian tumors and the assessment of carcinomatosis, together with a summary of the evidence supporting each statement.

Validation of American College of Radiology Ovarian-Adnexal Reporting and Data System Ultrasound (O-RADS US): Analysis on 1054 adnexal masses

OBJECTIVE

To assess the diagnostic performance and inter-observer agreement of the American College of Radiology (ACR) Ovarian-Adnexal Reporting and Data System Ultrasound (O-RADS US).

METHODS

From January 2016 to December 2018 a total of 1054 adnexal lesions in 1035 patients with pathologic results from two hospitals were retrospectively included. Each lesion was assigned to an O-RADS US category according to the criteria. Kappa (κ) statistics were applied to assess inter-observer agreement between a less experienced and an expert radiologist.

RESULTS

Of the 1054 adnexal lesions, 750 were benign and 304 were malignant. The malignancy rates of O-RADS 5, O-RADS 4, O-RADS 3, and O-RADS 2 lesions were 89.57%, 34.46%, 1.10%, and 0.45% respectively. Area under the receiver operating characteristic curve was 0.960 (95% CI, 0.947-0.971). The optimal cutoff value for predicting malignancy was >O-RADS 3 with a sensitivity and specificity of 98.7% (95% CI, 0.964-0.996) and 83.2% (95% CI, 0.802-0.858) respectively. When sub-classifying multilocular cysts and smooth solid lesions in O-RADS 4 lesions as O-RADS 4a lesions and the rest cystic lesions with solid components as O-RADS 4b lesions, the malignancy rate were 17.02% and 42.57% respectively, which showed better risk stratification (P < 0.001). The inter-observer agreement between a less-experienced and an expert radiologist of O-RADS categorization was good (κ = 0.714).

CONCLUSIONS

The ACR O-RADS US provides effective malignancy risk stratification for adnexal lesions with high reliability for radiologists with different experience. Sub-grouping of O-RADS 4 lesions into two groups facilitated better stratification of the intermediate risk.

External validation of Risk of Malignancy Index compared to IOTA Simple Rules

BACKGROUND

Mathematical predictive models for ovarian tumors have an advantage over subjective assessment due to their relative simplicity, and therefore usefulness for less experienced sonographers. It is currently unclear which predictive model is best at predicting the nature of an ovarian tumor.

PURPOSE

To compare the diagnostic predictive accuracy of the International Ovarian Tumour Analysis Simple Rules (IOTA SR) with Risk of Malignancy Index (RMI), to differentiate between benign and malignant ovarian tumors.

MATERIAL AND METHODS

A total of 202 women diagnosed with ovarian tumor(s) were included. Preoperatively, patients were examined through transvaginal ultrasonography and CA-125 (U/mL) levels were measured. RMI and IOTA SR were determined, and where possible compared to definitive histopathological diagnosis.

RESULTS

Of the 202 women with ovarian tumors, 168 women were included in this cohort study. Of these tumors, 118 (70.2%) were benign, 17 (10.1%) were borderline, and 33 (19.7%) were malignant. The sensitivity, specificity, and area under the curve for the RMI were 72.0%, 90.7%, and 0.896, respectively. For the IOTA SR, these were 90.0%, 68.6%, and 0.793, respectively.

CONCLUSION

This cohort study shows that the RMI is a relatively useful diagnostic model in characterizing ovarian tumors, compared to the IOTA SR. However, due to the relatively low sensitivity of the RMI and high rate of inconclusive results of the IOTA SR, both diagnostic tests do not seem discriminative enough. Therefore, alternative diagnostic models are necessary.

Transferability of the early-stage ovarian malignancy (EOM) score: an external validation study that includes advanced-stage and metastatic ovarian cancer

PURPOSE

To validate the diagnostic performance of the Early-stage Ovarian Malignancy (EOM) score in an external dataset that includes advanced-stage and metastatic ovarian cancer.

METHODS

The data from two cross-sectional cohorts were used in the statistical analysis. The development dataset of the EOM score was collected in Phrapokklao Hospital between September 2013 and December 2017. The validation dataset was collected in Maharaj Nakorn Chiang Mai Hospital between April 2010 and March 2018. The internal and external performance of the EOM score was evaluated in terms of discrimination via area under the receiver-operating characteristic curve (AuROC) and calibration.

RESULTS

There were 270 and 479 patients included in the development and validation datasets, respectively. The prevalence of ovarian malignancy was 20.0% (54/270) in the development set and 30.3% (145/479) in the validation set. The EOM score had excellent discriminative ability in both the development and validation sets (AuROC 88.0 (95% CI 82.6, 93.9) and 88.0 (95% CI 84.3, 91.4), respectively). The EOM score also showed good calibration in both datasets.

CONCLUSIONS

The EOM score had consistent diagnostic performance in the external validation data. It is recommended for use as a triage tool in patient referrals instead of the RMI in settings where experienced sonographers are not available.

Early-Stage Ovarian Malignancy Score versus Risk of Malignancy Indices: Accuracy and Clinical Utility for Preoperative Diagnosis of Women with Adnexal Masses

Background and objectives: To compare the diagnostic accuracy and clinical utility of the Early-stage Ovarian Malignancy (EOM) score with the Risk of Malignancy Index (RMI) in the presurgical assessment of women presenting with adnexal masses. Materials and Methods: A secondary analysis was carried out in a retrospective cohort of women who presented with an adnexal mass and were scheduled for surgery at Phrapokklao Hospital between September 2013 and December 2017. The clinical characteristics, ultrasonographic features of the masses, and preoperative CA-125 levels were recorded. The EOM and the RMI score were calculated and compared in terms of accuracy and clinical utility. Decision curve analysis (DCA), which examined the net benefit (NB) of applying the EOM and the RMI in practice at a range of threshold probabilities, was presented. Results: In this study, data from 270 patients were analyzed. Fifty-four (20.0%) women in the sample had early-stage ovarian cancer. All four RMI versions demonstrated a lower sensitivity for the detection of patients with early-stage ovarian cancer compared to an EOM score ≥ 15. An EOM ≥ 15 resulted in a higher proportion of net true positive or NB than all versions of the RMIs from a threshold probability of 5% to 30%. Conclusions: It also showed a higher capability to reduce the number of inappropriate referrals than the RMIs at a threshold probability between 5% and 30%. The EOM score showed higher diagnostic sensitivity and has the potential to be clinically more useful than the RMIs to triage women who present with adnexal masses for referral to oncologic gynecologists. Further external validation is required to support our findings.

Comparison of O-RADS, GI-RADS, and IOTA simple rules regarding malignancy rate, validity, and reliability for diagnosis of adnexal masses

OBJECTIVE

The American College of Radiology (ACR) recently published the ovarian-adnexal reporting and data system (O-RADS) to provide guidelines to physicians who interpret ultrasound (US) examinations of adnexal masses (AM). This study aimed to compare the O-RADS with two other well-established US classification systems for diagnosis of AM.

METHODS

This retrospective multicenter study between May 2016 and December 2019 assessed consecutive women with AM detected by the US. Five experienced consultant radiologists independently categorized each AM according to O-RADS, gynecologic imaging reporting and data system (GI-RADS), and international ovarian tumor analysis (IOTA) simple rules. Pathology and adequate follow-up were used as reference standards for calculating the validity of three US classification systems for diagnosis of AM. Kappa statistics were used to assess the inter-reviewer agreement (IRA).

RESULTS

A total of 609 women (mean age, 48 ± 13.7 years; range, 18-72 years) with 647 AM were included. Of the 647 AM, 178 were malignant and 469 were benign. Malignancy rates were comparable to recommended rates by previous literature in O-RADS and IOTA, but higher in GI-RADS. O-RADS had significantly higher sensitivity for malignancy than GI-RAD and IOTA (p = 0.003 and 0.0007, respectively), but non-significant slightly lower specificity (p > 0.05). O-RADS, GI-RADS, and IOTA showed similar overall IRA (κ = 0.77, 0.69, and 0.63, respectively) with a tendency toward higher IRA with O-RADS than with GI-RADS and IOTA.

CONCLUSIONS

O-RADS compares favorably with GI-RADS and IOTA. O-RADS had higher sensitivity than GI-RADS and IOTA simple rules with relatively similar specificity and reliability.

KEY POINTS

• The malignancy rates were comparable to recommended rates by previous literature in O-RADS and IOTA, but higher in GI-RADS. • The O-RADS had significantly higher sensitivity for malignancy than GI-RADS and IOTA (96.8% vs 92.7% and 92.1%; p = 0.003 and 0.0007, respectively), but non-significant slightly lower specificity (92.8% vs 93.6% and 93.2%, respectively; p > 0.05). • The O-RADS, GI-RADS, and IOTA showed similar overall inter-reviewer agreement (IRA) (κ = 0.77, 0.69, and 0.63, respectively), with a tendency toward higher IRA with O-RADS than with GI-RADS and IOTA.

Validation of models to diagnose ovarian cancer in patients managed surgically or conservatively: multicentre cohort study

OBJECTIVE

To evaluate the performance of diagnostic prediction models for ovarian malignancy in all patients with an ovarian mass managed surgically or conservatively.

DESIGN

Multicentre cohort study.

SETTING

36 oncology referral centres (tertiary centres with a specific gynaecological oncology unit) or other types of centre.

PARTICIPANTS

Consecutive adult patients presenting with an adnexal mass between January 2012 and March 2015 and managed by surgery or follow-up.

MAIN OUTCOME MEASURES

Overall and centre specific discrimination, calibration, and clinical utility of six prediction models for ovarian malignancy (risk of malignancy index (RMI), logistic regression model 2 (LR2), simple rules, simple rules risk model (SRRisk), assessment of different neoplasias in the adnexa (ADNEX) with or without CA125). ADNEX allows the risk of malignancy to be subdivided into risks of a borderline, stage I primary, stage II-IV primary, or secondary metastatic malignancy. The outcome was based on histology if patients underwent surgery, or on results of clinical and ultrasound follow-up at 12 (±2) months. Multiple imputation was used when outcome based on follow-up was uncertain.

RESULTS

The primary analysis included 17 centres that met strict quality criteria for surgical and follow-up data (5717 of all 8519 patients). 812 patients (14%) had a mass that was already in follow-up at study recruitment, therefore 4905 patients were included in the statistical analysis. The outcome was benign in 3441 (70%) patients and malignant in 978 (20%). Uncertain outcomes (486, 10%) were most often explained by limited follow-up information. The overall area under the receiver operating characteristic curve was highest for ADNEX with CA125 (0.94, 95% confidence interval 0.92 to 0.96), ADNEX without CA125 (0.94, 0.91 to 0.95) and SRRisk (0.94, 0.91 to 0.95), and lowest for RMI (0.89, 0.85 to 0.92). Calibration varied among centres for all models, however the ADNEX models and SRRisk were the best calibrated. Calibration of the estimated risks for the tumour subtypes was good for ADNEX irrespective of whether or not CA125 was included as a predictor. Overall clinical utility (net benefit) was highest for the ADNEX models and SRRisk, and lowest for RMI. For patients who received at least one follow-up scan (n=1958), overall area under the receiver operating characteristic curve ranged from 0.76 (95% confidence interval 0.66 to 0.84) for RMI to 0.89 (0.81 to 0.94) for ADNEX with CA125.

CONCLUSIONS

Our study found the ADNEX models and SRRisk are the best models to distinguish between benign and malignant masses in all patients presenting with an adnexal mass, including those managed conservatively.

TRIAL REGISTRATION

ClinicalTrials.gov NCT01698632.

Validation of IOTA-ADNEX Model in Discriminating Characteristics of Adnexal Masses: A Comparison with Subjective Assessment

(1) Background: The aim of this study is to compare the IOTA-ADNEX (international ovarian tumor analysis–assessment of different neoplasias in the adnexa) model with gynecologic experts in differentiating ovarian diseases. (2) Methods: All participants in this prospective study underwent ultrasonography (US) equipped with the IOTA-ADNEX^TM model and subjective assessment by a sonographic expert. Receiver operating characteristic (ROC) curves were also generated to compare overall accuracies. The optimal cut-off value of the ADNEX model for excluding benign diseases was calculated. (3) Results: Fifty-nine participants were eligible: 54 and 5 underwent surgery and follow-up computed tomography (CT), respectively. Benign and malignant diseases were confirmed in 49 (83.1%) and 10 (16.9%) participants, respectively. The specificity of the ADNEX model was 0.816 (95% confidence interval (CI): 0.680–0.912) in all participants and 0.795 (95% CI, 0.647–0.902) in the surgical group. The area under the ROC curve of the ADNEX model (0.924) was not significantly different from that of subjective assessment (0.953 in all participants, 0.951 in the surgical group; p = 0.391 in all participants, p = 0.407 in the surgical group). The optimal cut-off point using the ADNEX model was 47.3%, with a specificity of 0.977 (95% CI: 0.880–0.999). (4) Conclusions: The IOTA-ADNEX model is equal to gynecologic US experts in excluding benign ovarian tumors. Subsequently, being familiar with this US software may help gynecologic beginners to reduce unnecessary surgery.

Performance of Selected Models for Predicting Malignancy in Ovarian Tumors in Relation to the Degree of Diagnostic Uncertainty by Subjective Assessment With Ultrasound

OBJECTIVES

The study's main aim was to evaluate the relationship between the performance of predictive models for differential diagnoses of ovarian tumors and levels of diagnostic confidence in subjective assessment (SA) with ultrasound. The second aim was to identify the parameters that differentiate between malignant and benign tumors among tumors initially diagnosed as uncertain by SA.

METHODS

The study included 250 (55%) benign ovarian masses and 201 (45%) malignant tumors. According to ultrasound findings, the tumors were divided into 6 groups: certainly benign, probably benign, uncertain but benign, uncertain but malignant, probably malignant, and certainly malignant. The performance of the risk of malignancy index, International Ovarian Tumor Analysis assessment of different neoplasias in the adnexa model, and International Ovarian Tumor Analysis logistic regression model 2 was analyzed in subgroups as follows: SA-certain tumors (including certainly benign and certainly malignant) versus SA-probable tumors (probably benign and probably malignant) versus SA-uncertain tumors (uncertain but benign and uncertain but malignant).

RESULTS

We found a progressive decrease in the performance of all models in association with the increased uncertainty in SA. The areas under the receiver operating characteristic curve for the risk of malignancy index, logistic regression model 2, and assessment of different neoplasias in the adnexa model decreased between the SA-certain and SA-uncertain groups by 20%, 28%, and 20%, respectively. The presence of solid parts and a high color score were the discriminatory features between uncertain but benign and uncertain but malignant tumors.

CONCLUSIONS

Studies are needed that focus on the subgroup of ovarian tumors that are difficult to classify by SA. In cases of uncertain tumors by SA, the presence of solid components or a high color score should prompt a gynecologic oncology clinic referral.

A SEER population analysis of stage IB resected gastric cancer: who can benefit from adjuvant therapy?

Objective: The benefit of adjuvant therapy (AT) remains controversial in stage IB gastric cancer (GC). This study aimed to offer a reference for the rational indications of AT.Methods: We retrospectively included 1216 stage IB GC who experienced curative surgery from the SEER database between 2004 and 2015. These patients were allocated into two groups: Group AT and Group surgery alone (Group SA). We established a nomogram to predict OS and then divided whole cohort into low-risk and high-risk groups based on the OS predicted by the nomogram.Results: Six variables, which were significantly related with OS of entire patients after matched, were incorporated in the nomogram. These variables were age, examined lymph nodes, tumor site, marital, family income and stage IB. The C-index of the model was 0.637 and the calibration curve showed that the anticipated values were in accordance with the actual values. The decision curve demonstrated that the optimal clinical impact was achieved when the threshold possibility was 0-56%. Then, the entire cohort was separated into low-risk (≤159 points) as well as high-risk (>159 points) groups based on the projected 5-year OS of recursive partitioning analysis. Group SA revealed a significantly poorer OS than Group AT for high-risk patients (p < .001); on the other hand, there was a comparable OS for low-risk patients (p = .361).Conclusions: We have developed an effective, intuitional and applied prognostic tool to clinical decision-making. For stage IB GC after surgical resection, AT was only recommended for high-risk patients. However, AT may be dispensable for low-risk patients.

Increased Diagnostic Accuracy of Adnexal Tumors with A Combination of Established Algorithms and Biomarkers

Ovarian cancer is the most lethal gynecologic cancer. Pre-diagnostic testing lacks sensitivity and specificity, and surgery is often the only way to secure the diagnosis. Exploring new biomarkers is of great importance, but the rationale of combining validated well-established biomarkers and algorithms could be a more effective way forward. We hypothesized that we can improve differential diagnostics and reduce false positives by combining (a) risk of malignancy index (RMI) with serum HE4, (b) risk of ovarian malignancy algorithm (ROMA) with a transvaginal ultrasound score or (c) adding HE4 to CA125 in a simple algorithm. With logistic regression modeling, new algorithms were explored and validated using leave-one-out cross validation. The analyses were performed in an existing cohort prospectively collected prior to surgery, 2013-2016. A total of 445 benign tumors and 135 ovarian cancers were included. All presented models improved specificity at cut-off compared to the original algorithm, and goodness of fit was significant (p < 0.001). Our findings confirm that HE4 is a marker that improves specificity without hampering sensitivity or diagnostic accuracy in adnexal tumors. We provide in this study "easy-to-use" algorithms that could aid in the triage of women to the most appropriate level of care when presenting with an unknown ovarian cyst or suspicious ovarian cancer.

Indications for adjuvant chemotherapy in patients with AJCC stage IIa T3N0M0 and T1N2M0 gastric cancer-an east and west multicenter study

PURPOSE

To determine the indications for adjuvant chemotherapy (AC) in patients with stage IIa gastric cancer (T3N0M0 and T1N2M0) according to the 7th American Joint Committee on Cancer (AJCC).

METHODS

A total of 1593 patients with T3N0M0 or T1N2M0 stage gastric cancer were identified from the Surveillance, Epidemiology, and End Results (SEER) database for the period 1988.1-2012.12. Cox multiple regression, nomogram and decision curve analyses were performed. External validation was performed using databases of the Fujian Medical University Union Hospital (FJUUH) (n = 241) and Italy IMIGASTRIC center (n = 45).

RESULTS

Cox multiple regression analysis showed that the risk factors that affected OS in patients receiving AC were age > 65 years old, T1N2M0, LN dissection number ≤ 15, tumor size > 20 mm, and nonadenocarcinoma. A nomogram was constructed to predict 5-year OS, and the patients were divided into those predicted to receive a high benefit (points ≤ 188) or a low benefit from AC (points > 188) according to a recursive partitioning analysis. OS was significantly higher for the high-benefit patients in the SEER database and the FJUUH dataset than in the non-AC patients (Log-rank < 0.05), and there was no significant difference in OS between the low-benefit patients and non-AC patients in any of the three centers (Log-rank = 0.154, 0.470, and 0.434, respectively). The decision curve indicated that the best clinical effect can be obtained when the threshold probability is 0-92%.

CONCLUSION

Regarding the controversy over whether T3N0M0 and T1N2M0 gastric cancer patients should be treated with AC, this study presents a predictive model that provides concise and accurate indications. These data show that high-benefit patients should receive AC.

Untapped potential of multicenter studies: a review of cardiovascular risk prediction models revealed inappropriate analyses and wide variation in reporting

BACKGROUND

Clinical prediction models are often constructed using multicenter databases. Such a data structure poses additional challenges for statistical analysis (clustered data) but offers opportunities for model generalizability to a broad range of centers. The purpose of this study was to describe properties, analysis, and reporting of multicenter studies in the Tufts PACE Clinical Prediction Model Registry and to illustrate consequences of common design and analyses choices.

METHODS

Fifty randomly selected studies that are included in the Tufts registry as multicenter and published after 2000 underwent full-text screening. Simulated examples illustrate some key concepts relevant to multicenter prediction research.

RESULTS

Multicenter studies differed widely in the number of participating centers (range 2 to 5473). Thirty-nine of 50 studies ignored the multicenter nature of data in the statistical analysis. In the others, clustering was resolved by developing the model on only one center, using mixed effects or stratified regression, or by using center-level characteristics as predictors. Twenty-three of 50 studies did not describe the clinical settings or type of centers from which data was obtained. Four of 50 studies discussed neither generalizability nor external validity of the developed model.

CONCLUSIONS

Regression methods and validation strategies tailored to multicenter studies are underutilized. Reporting on generalizability and potential external validity of the model lacks transparency. Hence, multicenter prediction research has untapped potential.

REGISTRATION

This review was not registered.

Are the indications for postoperative radiotherapy in the NCCN guidelines for patients with gastric adenocarcinoma too broad? A study based on the SEER database

Background

The types of patients with gastric adenocarcinoma (GA) for whom postoperative radiotherapy can improve the disease-specific survival rate (DSS) remain controversial. This study aims to explore the ideal indications.

Methods

Patients in the Surveillance, Epidemiology, and End Results (SEER) database with T3–4Nx or TxN+ GA from January 1988 to December 2012 were included and divided into a postoperative chemoradiotherapy group (Group R) and a postoperative chemotherapy group (Group C). We established a nomogram to predict DSS and then divided entire patient cohort into low-risk and high-risk groups based on the DSS predicted by the nomogram.

Results

The Cox multiple regression analysis demonstrated that various risk factors affected DSS for Group R. Based on these risk factors, a nomogram for predicting DSS was established. The decision curve indicated that the best clinical effect could be obtained when the threshold probability was 0–58%. The patients were then divided into low-risk (< 69 points) and high-risk (≥ 69 points) groups according to the five-year DSS predicted. DSS was significantly better for Group R than for Group C for high-risk patients (P < 0.001) but was similar for low-risk patients (P = 0.732).

Conclusion

At present, the National Comprehensive Cancer Network (NCCN) guidelines may include an overly broad range of indications for postoperative radiotherapy for patients with GA. For intestinal GA patients with a postoperative pathologic stage of T1 N1 who are younger than 65 years, have had more than 15 lymph nodes dissected, and have received postoperative chemotherapy, postoperative radiotherapy should not be recommended.

Electronic supplementary material

The online version of this article (10.1186/s12885-018-4957-6) contains supplementary material, which is available to authorized users.

A preliminary study: The sequential use of the risk malignancy index and contrast-enhanced ultrasonography in differential diagnosis of adnexal masses

The aim of this study was to explore the sequential use of risk malignancy index (RMI) combined with contrast-enhanced ultrasonography (CEUS) in identification diagnosis of adnexal masses.This study contained 2 steps: first, 151 patients were analyzed retrospectively with RMI 1, RMI 2, and RMI 3 indices; receiver operating characteristic (ROC) curves were plotted to analyze area under the curves (AUC), and then RMI cut-off value was obtained according to maximum Youden index (YI, Sensitivity + Specificity - 1) and calculating diagnostic sensitivity, specificity, positive/negative predictive value, and accuracy. Second, 151 cases were divided into 2 groups randomly (105 in study group and 46 in test group); in the study group, the lower cut-off value (LC), upper cut-off value (UC), CEUS cut-off value according to maximum YI, and then these cut-offs were validated in test group.There was no statistical significance in 3 RMI models (P = .35), and RMI1 model was established randomly for following study. When the RMI1 cut-off value was 149, the YI was maximal (0.53), and the sensitivity, specificity, positive/negative predictive value, and accuracy were 71.0%, 81.7%, 77.1%, 75.6%, and 76.2%, respectively. The LC was 15 (sensitivity was 98.0%), the UC was 3000 (specificity was 98.0%), and the CEUS cut-off value was 7 (maximal YI was 0.81). In the test group (46 cases), combining RMI1 LC (15) and UC (3000) with CEUS cut-off value (7), the sensitivity, specificity, positive/negative predictive value, and accuracy were up to 85.7%, 92.0%, 90.0%, 88.5%, and 89.1%, respectively.CEUS can help RMI to make a more effective differential diagnosis of the adnexal mass. Further validation by additional multicenter prospective trials is required.

Does the Risk of Ovarian Malignancy Algorithm Provide Better Diagnostic Performance Than HE4 and CA125 in the Presurgical Differentiation of Adnexal Tumors in Polish Women?

Aim

This study compared the diagnostic performance of the Risk of Ovarian Malignancy Algorithm (ROMA) and HE4 and CA125 for the presurgical differentiation of adnexal tumors.

Material and Methods

This prospective study included 302 patients admitted for surgical treatment due to adnexal tumors. The ROMA was calculated depending on CA125, HE4, and menopausal status.

Results

Fifty patients were diagnosed with malignant disease. In the differentiation of malignant from nonmalignant adnexal tumors, the area under curve (AUC) was higher for ROMA and HE4 than that for CA125 in both the premenopausal and postmenopausal subgroups. In the differentiation of stage I FIGO malignancies and epithelial ovarian cancer from nonmalignant pathologies, the AUC of HE4 and ROMA was higher than that of CA125. The ROMA performed significantly better than CA125 in the differentiation of all malignancies and differentiation of stage I FIGO malignancies from nonmalignant pathologies (p = 0.043 and p = 0.025, resp.). There were no significant differences between the ROMA and the tumor markers for any other variants.

Conclusions

The ROMA is more useful than CA125 for the differentiation of malignant (including stage I FIGO) from nonmalignant adnexal tumors. It is also as useful as HE4 and CA125 for the differentiation of epithelial ovarian cancer from nonmalignant adnexal tumors.

New nomogram predicts the recurrence of hepatocellular carcinoma in patients with negative preoperative serum AFP subjected to curative resection

BACKGROUND

There is currently no established model for predicting the recurrence of hepatocellular carcinoma (HCC) in patients with negative alpha-fetoprotein (AFP) after curative resection. Therefore, the objective of this study was to establish a nomogram to identify the risk of recurrence in AFP-negative (<or = 20 ng/mL) patients with HCC.

METHODS

A retrospective study was conducted to establish the recurrence-free survival (RFS) nomogram in a training cohort of 326 AFP-negative HCC patients. The results were validated on a well-matched validation cohort in the literature.

RESULTS

Macrovascular tumour invasion (P = 0.018, HR = 1.642), macronodular cirrhosis (P < 0.001, HR = 2.128), tumor size (P = 0.004, HR = 1.691), and γ-glutamyl transferase (P = 0.039, HR = 1.496) were found to be independent risk factors for RFS in the training cohort, and all these factors were included in the nomogram. The C-index for RFS in the nomogram was 0.661, which was higher than that of the BCLC system (0.551), the CLIP score (0.537), and the prediction model of Ju (0.618). The high consistency between the nomogram prediction and actual observation was further demonstrated by the calibration curve. In the subsequent study, the better net benefit and higher threshold probability of the nomogram were determined by decision curve analysis, and these advantages were confirmed in the validation cohort.

CONCLUSIONS

The present RFS nomogram for AFP-negative HCC patients after curative resection provides an accurate and reliable prognostic model to facilitate recurrence surveillance. Once AFP-negative patients are predicted to have a high recurrence score, additional high-end imaging examinations, such as MRI or CT exams, should be considered, and the interval time of regular folow-up should be reduced.

Poor performance of clinical prediction models: the harm of commonly applied methods

OBJECTIVE

To evaluate limitations of common statistical modeling approaches in deriving clinical prediction models and explore alternative strategies.

STUDY DESIGN AND SETTING

A previously published model predicted the likelihood of having a mutation in germline DNA mismatch repair genes at the time of diagnosis of colorectal cancer. This model was based on a cohort where 38 mutations were found among 870 participants, with validation in an independent cohort with 35 mutations. The modeling strategy included stepwise selection of predictors from a pool of over 37 candidate predictors and dichotomization of continuous predictors. We simulated this strategy in small subsets of a large contemporary cohort (2,051 mutations among 19,866 participants) and made comparisons to other modeling approaches. All models were evaluated according to bias and discriminative ability (concordance index, c) in independent data.

RESULTS

We found over 50% bias for five of six originally selected predictors, unstable model specification, and poor performance at validation (median c = 0.74). A small validation sample hampered stable assessment of performance. Model prespecification based on external knowledge and using continuous predictors led to better performance (c = 0.836 and c = 0.852 with 38 and 2,051 events respectively).

CONCLUSION

Prediction models perform poorly if based on small numbers of events and developed with common but suboptimal statistical approaches. Alternative modeling strategies to best exploit available predictive information need wider implementation, with collaborative research to increase sample sizes.