Preoperative diagnosis of ovarian tumors using Bayesian kernel-based methods

Role of artificial intelligence applied to ultrasound in gynecology oncology: A systematic review

The aim of this paper was to explore the role of artificial intelligence (AI) applied to ultrasound imaging in gynecology oncology. Web of Science, PubMed, and Scopus databases were searched. All studies were imported to RAYYAN QCRI software. The overall quality of the included studies was assessed using QUADAS-AI tool. Fifty studies were included, of these 37/50 (74.0%) on ovarian masses or ovarian cancer, 5/50 (10.0%) on endometrial cancer, 5/50 (10.0%) on cervical cancer, and 3/50 (6.0%) on other malignancies. Most studies were at high risk of bias for subject selection (i.e., sample size, source, or scanner model were not specified; data were not derived from open-source datasets; imaging preprocessing was not performed) and index test (AI models was not externally validated) and at low risk of bias for reference standard (i.e., the reference standard correctly classified the target condition) and workflow (i.e., the time between index test and reference standard was reasonable). Most studies presented machine learning models (33/50, 66.0%) for the diagnosis and histopathological correlation of ovarian masses, while others focused on automatic segmentation, reproducibility of radiomics features, improvement of image quality, prediction of therapy resistance, progression-free survival, and genetic mutation. The current evidence supports the role of AI as a complementary clinical and research tool in diagnosis, patient stratification, and prediction of histopathological correlation in gynecological malignancies. For example, the high performance of AI models to discriminate between benign and malignant ovarian masses or to predict their specific histology can improve the diagnostic accuracy of imaging methods.

Multiclass risk models for ovarian malignancy: an illustration of prediction uncertainty due to the choice of algorithm

BACKGROUND

Assessing malignancy risk is important to choose appropriate management of ovarian tumors. We compared six algorithms to estimate the probabilities that an ovarian tumor is benign, borderline malignant, stage I primary invasive, stage II-IV primary invasive, or secondary metastatic.

METHODS

This retrospective cohort study used 5909 patients recruited from 1999 to 2012 for model development, and 3199 patients recruited from 2012 to 2015 for model validation. Patients were recruited at oncology referral or general centers and underwent an ultrasound examination and surgery ≤ 120 days later. We developed models using standard multinomial logistic regression (MLR), Ridge MLR, random forest (RF), XGBoost, neural networks (NN), and support vector machines (SVM). We used nine clinical and ultrasound predictors but developed models with or without CA125.

RESULTS

Most tumors were benign (3980 in development and 1688 in validation data), secondary metastatic tumors were least common (246 and 172). The c-statistic (AUROC) to discriminate benign from any type of malignant tumor ranged from 0.89 to 0.92 for models with CA125, from 0.89 to 0.91 for models without. The multiclass c-statistic ranged from 0.41 (SVM) to 0.55 (XGBoost) for models with CA125, and from 0.42 (SVM) to 0.51 (standard MLR) for models without. Multiclass calibration was best for RF and XGBoost. Estimated probabilities for a benign tumor in the same patient often differed by more than 0.2 (20% points) depending on the model. Net Benefit for diagnosing malignancy was similar for algorithms at the commonly used 10% risk threshold, but was slightly higher for RF at higher thresholds. Comparing models, between 3% (XGBoost vs. NN, with CA125) and 30% (NN vs. SVM, without CA125) of patients fell on opposite sides of the 10% threshold.

CONCLUSION

Although several models had similarly good performance, individual probability estimates varied substantially.

Detection of Dendritic Spines Using Wavelet-Based Conditional Symmetric Analysis and Regularized Morphological Shared-Weight Neural Networks

Identification and detection of dendritic spines in neuron images are of high interest in diagnosis and treatment of neurological and psychiatric disorders (e.g., Alzheimer's disease, Parkinson's diseases, and autism). In this paper, we have proposed a novel automatic approach using wavelet-based conditional symmetric analysis and regularized morphological shared-weight neural networks (RMSNN) for dendritic spine identification involving the following steps: backbone extraction, localization of dendritic spines, and classification. First, a new algorithm based on wavelet transform and conditional symmetric analysis has been developed to extract backbone and locate the dendrite boundary. Then, the RMSNN has been proposed to classify the spines into three predefined categories (mushroom, thin, and stubby). We have compared our proposed approach against the existing methods. The experimental result demonstrates that the proposed approach can accurately locate the dendrite and accurately classify the spines into three categories with the accuracy of 99.1% for “mushroom” spines, 97.6% for “stubby” spines, and 98.6% for “thin” spines.

Sensitivity and Specificity Can Change in Opposite Directions When New Predictive Markers Are Added to Risk Models

When comparing prediction models, it is essential to estimate the magnitude of change in performance rather than rely solely on statistical significance. In this paper we investigate measures that estimate change in classification performance, assuming 2-group classification based on a single risk threshold. We study the value of a new biomarker when added to a baseline risk prediction model. First, simulated data are used to investigate the change in sensitivity and specificity (ΔSe and ΔSp). Second, the influence of ΔSe and ΔSp on the net reclassification improvement (NRI; sum of ΔSe and ΔSp) and on decision-analytic measures (net benefit or relative utility) is studied. We assume normal distributions for the predictors and assume correctly specified models such that the extended model has a dominating receiver operating characteristic curve relative to the baseline model. Remarkably, we observe that even when a strong marker is added it is possible that either sensitivity (for thresholds below the event rate) or specificity (for thresholds above the event rate) decreases. In these cases, decision-analytic measures provide more modest support for improved classification than NRI, even though all measures confirm that adding the marker improved classification accuracy. Our results underscore the necessity of reporting ΔSe and ΔSp separately. When a single summary is desired, decision-analytic measures allow for a simple incorporation of the misclassification costs.

Improving strategies for diagnosing ovarian cancer: a summary of the International Ovarian Tumor Analysis (IOTA) studies

In order to ensure that ovarian cancer patients access appropriate treatment to improve the outcome of this disease, accurate characterization before any surgery on ovarian pathology is essential. The International Ovarian Tumor Analysis (IOTA) collaboration has standardized the approach to the ultrasound description of adnexal pathology. A prospectively collected large database enabled previously developed prediction models like the risk of malignancy index (RMI) to be tested and novel prediction models to be developed and externally validated in order to determine the optimal approach to characterize adnexal pathology preoperatively. The main IOTA prediction models (logistic regression model 1 (LR1) and logistic regression model 2 (LR2)) have both shown excellent diagnostic performance (area under the curve (AUC) values of 0.96 and 0.95, respectively) and outperform previous diagnostic algorithms. Their test performance almost matches subjective assessment by experienced examiners, which is accepted to be the best way to classify adnexal masses before surgery. A two-step strategy using the IOTA simple rules supplemented with subjective assessment of ultrasound findings when the rules do not apply, also reached excellent diagnostic performance (sensitivity 90%, specificity 93%) and misclassified fewer malignancies than did the RMI. An evidence-based approach to the preoperative characterization of ovarian and other adnexal masses should include the use of LR1, LR2 or IOTA simple rules and subjective assessment by an experienced examiner.

A mathematical model for interpretable clinical decision support with applications in gynecology

BACKGROUND

Over time, methods for the development of clinical decision support (CDS) systems have evolved from interpretable and easy-to-use scoring systems to very complex and non-interpretable mathematical models. In order to accomplish effective decision support, CDS systems should provide information on how the model arrives at a certain decision. To address the issue of incompatibility between performance, interpretability and applicability of CDS systems, this paper proposes an innovative model structure, automatically leading to interpretable and easily applicable models. The resulting models can be used to guide clinicians when deciding upon the appropriate treatment, estimating patient-specific risks and to improve communication with patients.

METHODS AND FINDINGS

We propose the interval coded scoring (ICS) system, which imposes that the effect of each variable on the estimated risk is constant within consecutive intervals. The number and position of the intervals are automatically obtained by solving an optimization problem, which additionally performs variable selection. The resulting model can be visualised by means of appealing scoring tables and color bars. ICS models can be used within software packages, in smartphone applications, or on paper, which is particularly useful for bedside medicine and home-monitoring. The ICS approach is illustrated on two gynecological problems: diagnosis of malignancy of ovarian tumors using a dataset containing 3,511 patients, and prediction of first trimester viability of pregnancies using a dataset of 1,435 women. Comparison of the performance of the ICS approach with a range of prediction models proposed in the literature illustrates the ability of ICS to combine optimal performance with the interpretability of simple scoring systems.

CONCLUSIONS

The ICS approach can improve patient-clinician communication and will provide additional insights in the importance and influence of available variables. Future challenges include extensions of the proposed methodology towards automated detection of interaction effects, multi-class decision support systems, prognosis and high-dimensional data.

Triaging women with ovarian masses for surgery: observational diagnostic study to compare RCOG guidelines with an International Ovarian Tumour Analysis (IOTA) group protocol

OBJECTIVE

To compare guidelines from the Royal College of Obstetricians and Gynaecologists (RCOG) based on the Risk of Malignancy Index (RMI) with a protocol based on logistic regression model LR2 developed by the International Ovarian Tumour Analysis (IOTA) group for triaging women with an ovarian mass as low, moderate, or high risk of malignancy.

DESIGN AND SETTING

Observational diagnostic study conducted between 2005 and 2007 at 21 oncology referral centres, referral centres for ultrasonography and general hospitals.

SAMPLE

In all, 1938 women undergoing surgery for an ovarian mass.

METHODS

RCOG guidelines use the RMI to triage women as low (RMI < 25), moderate (25-250), or high (above >250) risk. The IOTA protocol uses LR2s estimated probability of malignancy (<0.05 indicates low risk, ≥ 0.05 but <0.25 moderate risk, and ≥ 0.25 high risk).

MAIN OUTCOME MEASURE

Percentages of benign, borderline and invasive tumours classified as low, moderate or high risk.

RESULTS

The IOTA and RCOG protocols classified 71.1% and 62.1% of benign tumours as low risk, respectively (difference 9.0; 95% CI 6.2-11.9, P < 0.0001). Of invasive tumours, 88.6% and 73.6% were labelled high risk (difference 15.0; 10.6-19.4, P < 0.0001), and 3.0% and 5.2% were labelled low risk (difference -2.2; -4.6 to 0.2, P = 0.07) respectively by each protocol. Similar results were found after stratification for menopausal status.

CONCLUSIONS

The IOTA protocol was more accurate for triage than the RCOG protocol. The IOTA protocol would avoid major surgery for more women with benign tumours while still appropriately referring more women with an invasive tumour to a gynaecological oncologist.

Adnexal masses suspected to be benign treated with laparoscopy

OBJECTIVE

Guidelines for referring women with pelvic masses suspicious for ovarian cancer to gynecologic oncologists have been developed by the American College of Obstetrician Gynecologists (ACOG). We set out to evaluate the negative predictive value of these guidelines and to assess a modified algorithm involving minimally invasive surgery in the treatment of women with masses suspected to be benign.

METHODS

257 consecutive patients with adnexal masses of 8cm to 13cm on preoperative ultrasound examination meeting Triage Criteria set forth in ACOG Committee Opinion 280. Patients meeting the selection criteria were scheduled for operative laparoscopy, washings, adnexectomy, bagging, and colpotomy. A total of 240 patients successfully completed intended treatment (93.38%), and 234 of these did not require admission (97.5%). There was a low incidence of significant complications: 97.50% of women were successfully treated as outpatients, 97.92% of surgeries lasted <136 minutes, and <97.08% had blood loss <200mL. The negative predictive value of ACOG Committee Opinion 280 Triage Criteria as a deselector for having invasive ovarian malignancy in our population was 95.57% for premenopausal and 90.91% for postmenopausal women.

CONCLUSIONS

Laparoscopic adnexectomy, bagging, and colpotomy is a desirable goal for patients with ovarian masses in the 8cm to 13cm range meeting selection criteria affording a minimally invasive approach with attendant benefits including outpatient treatment (97.5%), few complications, low likelihood of iatrogenic rupture of the ovarian capsule (1.25%), and low necessity for reoperation after final pathology is evaluated (6.03%). Negative predictive value of ACOG Committee Opinion 280 is confirmed in a community gynecology practice and is recommended to form the basis of a new treatment algorithm for women with adnexal masses.

Improved modeling of clinical data with kernel methods

OBJECTIVE

Despite the rise of high-throughput technologies, clinical data such as age, gender and medical history guide clinical management for most diseases and examinations. To improve clinical management, available patient information should be fully exploited. This requires appropriate modeling of relevant parameters.

METHODS

When kernel methods are used, traditional kernel functions such as the linear kernel are often applied to the set of clinical parameters. These kernel functions, however, have their disadvantages due to the specific characteristics of clinical data, being a mix of variable types with each variable its own range. We propose a new kernel function specifically adapted to the characteristics of clinical data.

RESULTS

The clinical kernel function provides a better representation of patients' similarity by equalizing the influence of all variables and taking into account the range r of the variables. Moreover, it is robust with respect to changes in r. Incorporated in a least squares support vector machine, the new kernel function results in significantly improved diagnosis, prognosis and prediction of therapy response. This is illustrated on four clinical data sets within gynecology, with an average increase in test area under the ROC curve (AUC) of 0.023, 0.021, 0.122 and 0.019, respectively. Moreover, when combining clinical parameters and expression data in three case studies on breast cancer, results improved overall with use of the new kernel function and when considering both data types in a weighted fashion, with a larger weight assigned to the clinical parameters. The increase in AUC with respect to a standard kernel function and/or unweighted data combination was maximum 0.127, 0.042 and 0.118 for the three case studies.

CONCLUSION

For clinical data consisting of variables of different types, the proposed kernel function--which takes into account the type and range of each variable--has shown to be a better alternative for linear and non-linear classification problems.

External validation of diagnostic models to estimate the risk of malignancy in adnexal masses

PURPOSE

To externally validate and compare the performance of previously published diagnostic models developed to predict malignancy in adnexal masses.

EXPERIMENTAL DESIGN

We externally validated the diagnostic performance of 11 models developed by the International Ovarian Tumor Analysis (IOTA) group and 12 other (non-IOTA) models on 997 prospectively collected patients. The non-IOTA models included the original risk of malignancy index (RMI), three modified versions of the RMI, six logistic regression models, and two artificial neural networks. The ability of the models to discriminate between benign and malignant adnexal masses was expressed as the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, and likelihood ratios (LR(+), LR(-)).

RESULTS

Seven hundred and forty-two (74%) benign and 255 (26%) malignant masses were included. The IOTA models did better than the non-IOTA models (AUCs between 0.941 and 0.956 vs. 0.839 and 0.928). The difference in AUC between the best IOTA and the best non-IOTA model was 0.028 [95% confidence interval (CI), 0.011-0.044]. The AUC of the RMI was 0.911 (difference with the best IOTA model, 0.044; 95% CI, 0.024-0.064). The superior performance of the IOTA models was most pronounced in premenopausal patients but was also observed in postmenopausal patients. IOTA models were better able to detect stage I ovarian cancer.

CONCLUSION

External validation shows that the IOTA models outperform other models, including the current reference test RMI, for discriminating between benign and malignant adnexal masses.

A novel approach to predict the likelihood of specific ovarian tumor pathology based on serum CA-125: a multicenter observational study

BACKGROUND

The CA-125 tumor marker has limitations when used to distinguish between benign and malignant ovarian masses. We therefore establish likelihood curves of six subgroups of ovarian pathology based on CA-125 and menopausal status.

METHODS

This cross-sectional study conducted by the International Ovarian Tumor Analysis group involved 3,511 patients presenting with a persistent adnexal mass that underwent surgical intervention. CA-125 distributions for six tumor subgroups (endometriomas and abscesses, other benign tumors, borderline tumors, stage I invasive cancers, stage II-IV invasive cancers, and metastatic tumors) were estimated using kernel density estimation with stratification for menopausal status. Likelihood curves for the tumor subgroups were derived from the distributions.

RESULTS

Endometriomas and abscesses were the only benign pathologies with median CA-125 levels above 20 U/mL (43 and 45, respectively). Borderline and invasive stage I tumors had relatively low median CA-125 levels (29 and 81 U/mL, respectively). The CA-125 distributions of stage II-IV invasive cancers and benign tumors other than endometriomas or abscesses were well separated; the distributions of the other subgroups overlapped substantially. This held for premenopausal and postmenopausal patients. Likelihood curves and reference tables comprehensibly show how subgroup likelihoods change with CA-125 and menopausal status.

CONCLUSIONS AND IMPACT

Our results confirm the limited clinical value of CA-125 for preoperative discrimination between benign and malignant ovarian pathology. We have shown that CA-125 may be used in a different way. By using likelihood reference tables, we believe clinicians will be better able to interpret preoperative serum CA-125 results in patients with adnexal masses.

Effect of cancer prevalence on the use of risk-assessment cut-off levels and the performance of mathematical models to distinguish malignant from benign adnexal masses

OBJECTIVE

Two logistic regression models have been developed for the characterization of adnexal masses. The goal of this prospective analysis was to see whether these models perform differently according to the prevalence of malignancy and whether the cut-off levels of risk assessment for malignancy by the models require modification in different centers.

METHODS

Centers were categorized into those with a prevalence of malignancy below 15%, between 15 and 30% and above 30%. The areas under the receiver-operating characteristics curves (AUC) were compared using bootstrapping. The optimal cut-off level of risk assessment for malignancy was chosen per center, corresponding to the highest sensitivity level possible while still keeping a good specificity.

RESULTS

Both models performed better in centers with a lower prevalence of malignant cases. The AUCs of the two models for centers with fewer than 15% malignant cases were 0.97 and 0.95, those of centers with 15-30% malignancy were 0.95 and 0.93 and those of centers with more than 30% malignant cases were 0.94 and 0.92. This decrease in performance was due mainly to the decrease in specificity from over 90 to around 76%. In the centers with a higher percentage of malignant cases, a sensitivity of at least 90% with a good specificity could not be obtained by choosing a different cut-off level.

CONCLUSIONS

Overall the models performed well in all centers. The performance of the logistic regression models worsened with increasing prevalence of malignancy, due to a case mix with more borderline and complex benign masses seen in those centers. Because the cut-off of 0.10 is optimal for all three types of center, it seems reasonable to use this cut-off for both models in all centers.

Improving the preoperative classification of adnexal masses as benign or malignant by second-stage tests

OBJECTIVE

The aim of this study was to establish when a second-stage diagnostic test may be of value in cases where a primary diagnostic test has given an uncertain diagnosis of the benign or malignant nature of an adnexal mass.

METHODS

The diagnostic performance with regard to discrimination between benign and malignant adnexal masses for mathematical models including ultrasound variables and for subjective evaluation of ultrasound findings by an experienced ultrasound examiner was expressed as area under the receiver-operating characteristics curve (AUC), sensitivity and specificity. These were calculated for the total study population of 1938 patients with an adnexal mass as well as for subpopulations defined by the certainty with which the diagnosis of benignity or malignancy was made. The effect of applying a second-stage test to the tumors where risk estimation was uncertain was determined.

RESULTS

The best mathematical model (LR1) had an AUC of 0.95, sensitivity of 92% and specificity of 84% when applied to all tumors. When model LR1 was applied to the 10% of tumors in which the calculated risk fell closest to the risk cut-off of the model, the AUC was 0.59, sensitivity 90% and specificity 21%. A strategy where subjective evaluation was used to classify these 10% of tumors for which LR1 performed poorly and where LR1 was used in the other 90% of tumors resulted in a sensitivity of 91% and specificity of 90%. Applying subjective evaluation to all tumors yielded an AUC of 0.95, sensitivity of 90% and specificity of 93%. Sensitivity was 81% and specificity 47% for those patients where the ultrasound examiner was uncertain about the diagnosis (n = 115; 5.9%). No mathematical model performed better than did subjective evaluation among the 115 tumors where the ultrasound examiner was uncertain.

CONCLUSION

When model LR1 is used as a primary test for discriminating between benign and malignant adnexal masses, the use of subjective evaluation of ultrasound findings by an experienced examiner as a second-stage test in the 10% of cases for which the model yields a risk of malignancy closest to its risk cut-off will improve specificity without substantially decreasing sensitivity. However, none of the models tested proved suitable as a second-stage test in tumors where subjective evaluation yielded an uncertain result.

Preoperative evaluation of adnexal masses

This is a large multicenter trial designed to determine the efficacy of second stage diagnostic testing when the primary diagnostic test has provided an uncertain diagnosis of a benign or malignant adnexal mass. Women with adnexal masses were evaluated using ultrasonography. Clinical and ultrasonographic data were employed to determine the risk of malignancy using 11 statistical models and their performance was evaluated based on histologic findings at surgery. Overall, the ultrasound examiner's subjective evaluation had the highest area under the curve and specificity, with respect to diagnostic performance. Of the mathematical models evaluated, LR1 performed the best, with an area under the curve of 0.95, a sensitivity of 92% and a specificity of 84%. The accuracy of subjective evaluation did not improve with the addition of any second stage test. If the LR1 model was used as the primary diagnostic test, the addition of subjective evaluation as a secondary test was found to be beneficial.

Polytomous diagnosis of ovarian tumors as benign, borderline, primary invasive or metastatic: development and validation of standard and kernel-based risk prediction models

Background

Hitherto, risk prediction models for preoperative ultrasound-based diagnosis of ovarian tumors were dichotomous (benign versus malignant). We develop and validate polytomous models (models that predict more than two events) to diagnose ovarian tumors as benign, borderline, primary invasive or metastatic invasive. The main focus is on how different types of models perform and compare.

Methods

A multi-center dataset containing 1066 women was used for model development and internal validation, whilst another multi-center dataset of 1938 women was used for temporal and external validation. Models were based on standard logistic regression and on penalized kernel-based algorithms (least squares support vector machines and kernel logistic regression). We used true polytomous models as well as combinations of dichotomous models based on the 'pairwise coupling' technique to produce polytomous risk estimates. Careful variable selection was performed, based largely on cross-validated c-index estimates. Model performance was assessed with the dichotomous c-index (i.e. the area under the ROC curve) and a polytomous extension, and with calibration graphs.

Results

For all models, between 9 and 11 predictors were selected. Internal validation was successful with polytomous c-indexes between 0.64 and 0.69. For the best model dichotomous c-indexes were between 0.73 (primary invasive vs metastatic) and 0.96 (borderline vs metastatic). On temporal and external validation, overall discrimination performance was good with polytomous c-indexes between 0.57 and 0.64. However, discrimination between primary and metastatic invasive tumors decreased to near random levels. Standard logistic regression performed well in comparison with advanced algorithms, and combining dichotomous models performed well in comparison with true polytomous models. The best model was a combination of dichotomous logistic regression models. This model is available online.

Conclusions

We have developed models that successfully discriminate between benign, borderline, and invasive ovarian tumors. Methodologically, the combination of dichotomous models was an interesting approach to tackle the polytomous problem. Standard logistic regression models were not outperformed by regularized kernel-based alternatives, a finding to which the careful variable selection procedure will have contributed. The random discrimination between primary and metastatic invasive tumors on temporal/external validation demonstrated once more the necessity of validation studies.

Development of a kernel function for clinical data

For most diseases and examinations, clinical data such as age, gender and medical history guides clinical management, despite the rise of high-throughput technologies. To fully exploit such clinical information, appropriate modeling of relevant parameters is required. As the widely used linear kernel function has several disadvantages when applied to clinical data, we propose a new kernel function specifically developed for this data. This "clinical kernel function" more accurately represents similarities between patients. Evidently, three data sets were studied and significantly better performances were obtained with a Least Squares Support Vector Machine when based on the clinical kernel function compared to the linear kernel function.

Feature selection for classification based on fine motor signs of Parkinson's disease

Effective evaluation of potential neuroprotective interventions for Parkinson's disease (PD) requires precise quantification of the motor signs associated with this disease. We have created a protocol that uses force tracking in a simultaneous task paradigm to quantify the fine motor control deficits in individuals with PD. We have used this protocol to collect data from 30 individuals with early to moderate PD and 30 age-matched controls. Based on this data, we computed 60 variables. We generated all possible combinations of three of these variables, and we then computed the classification accuracy of a support vector machine (SVM) trained on each variable combination. We were able to correctly classify 85% of subjects as with or without PD. We found that root-mean-square error variables were the most important features for classification and that utilizing a simultaneous task paradigm improves classification accuracy.

Prospective internal validation of mathematical models to predict malignancy in adnexal masses: results from the international ovarian tumor analysis study

PURPOSE

To prospectively test the mathematical models for calculation of the risk of malignancy in adnexal masses that were developed on the International Ovarian Tumor Analysis (IOTA) phase 1 data set on a new data set and to compare their performance with that of pattern recognition, our standard method.

METHODS

Three IOTA centers included 507 new patients who all underwent a transvaginal ultrasound using the standardized IOTA protocol. The outcome measure was the histologic classification of excised tissue. The diagnostic performance of 11 mathematical models that had been developed on the phase 1 data set and of pattern recognition was expressed as area under the receiver operating characteristic curve (AUC) and as sensitivity and specificity when using the cutoffs recommended in the studies where the models had been created. For pattern recognition, an AUC was made based on level of diagnostic confidence.

RESULTS

All IOTA models performed very well and quite similarly, with sensitivity and specificity ranging between 92% and 96% and 74% and 84%, respectively, and AUCs between 0.945 and 0.950. A least squares support vector machine with linear kernel and a logistic regression model had the largest AUCs. For pattern recognition, the AUC was 0.963, sensitivity was 90.2%, and specificity was 92.9%.

CONCLUSION

This internal validation of mathematical models to estimate the malignancy risk in adnexal tumors shows that the IOTA models had a diagnostic performance similar to that in the original data set. Pattern recognition used by an expert sonologist remains the best method, although the difference in performance between the best mathematical model is not large.

Evaluation of Imputation Methods in Ovarian Tumor Diagnostic Models Using Generalized Linear Models and Support Vector Machines

Neglecting missing values in diagnostic models can result in unreliable and suboptimal performance on new data. In this study, the authors imputed missing values for the CA-125 tumor marker in a large data set of ovarian tumors that was used to develop models for predicting malignancy. Four imputation techniques were applied: regression imputation, expectation-maximization, data augmentation, and hotdeck. Models using the imputed data sets were compared with models without CA-125 to investigate the important clinical issue concerning the necessity of CA-125 information for diagnostic models and with models using only complete cases to investigate differences between imputation and complete case strategies for missing values. The models are based on Bayesian generalized linear models (GLMs) and Bayesian least squares support vector machines. Results indicate that the use of CA-125 resulted in small, clinically nonsignificant increases in the AUC of diagnostic models. Minor differences between imputation methods were observed, and imputing CA-125 resulted in minor differences in the AUC compared with complete case analysis (CCA). However, GLM parameter estimates of predictor variables often differed between CCA and models based on imputation. The authors conclude that CA-125 is not indispensable in diagnostic models for ovarian tumors and that missing value imputation is preferred over CCA.

The accuracy of risk scores in predicting ovarian malignancy: a systematic review

OBJECTIVE

To perform a systematic review of the literature on the accuracy of prediction models in the preoperative assessment of adnexal masses.

DATA SOURCES

Studies were identified through the MEDLINE and EMBASE databases from inception to March 2008. The MEDLINE search was performed using the keywords ["ovarian neoplasms"[MeSH] NOT "therapeutics"[MeSH] AND "model"] and ["ovarian neoplasms"[MeSH] NOT "therapeutics"[MeSH] AND "prediction"]. The Embase search was performed using the keywords [ovary tumor AND prediction], [ovary tumor AND Mathematical model], and [ovary tumor AND statistical model].

METHODS OF STUDY SELECTION

The search detected 1,161 publications; from the cross-references, another 116 studies were identified. Language restrictions were not applied. Eligible studies contained data on the accuracy of models predicting the risk of malignancy in ovarian masses. Models were required to combine at least two parameters.

TABULATION, INTEGRATION, AND RESULTS

Two independent reviewers selected studies and extracted study characteristics, study quality, and test accuracy. There were 109 accuracy studies that met the selection criteria. Accuracy data were used to form two-by-two contingency tables of the results of the risk score compared with definitive histology. We used bivariate meta-analysis to estimate pooled sensitivities and specificities and to fit summary receiver operating characteristic curves.Studies included in our analysis reported on 83 different prediction models. The model developed by Sassone was the most evaluated prediction model. All models has acceptable sensitivity and specificity. However, the Risk of Malignancy Index I and the Risk of Malignancy Index II, which use the product of the serum CA 125 level, an ultrasound scan result, and the menopausal state, were the best predictors. When 200 was used as the cutoff level, the pooled estimate for sensitivity was 78% for a specificity of 87%.

CONCLUSION

Based on our review, the Risk of Malignancy Index should be the prediction model of choice in the preoperative assessment of the adnexal mass.

Comparison of the risk of malignancy index and self-constructed logistic regression models in preoperative evaluation of adnexal masses

OBJECTIVE

The aim of this study was to evaluate women with adnexal masses in the preoperative period by creating 2 logistic regression models, 1 including sonographic morphologic characteristics and the other including both morphologic and color Doppler characteristics, to compare the diagnostic accuracy of these 2 models with the risk of malignancy index (RMI).

METHODS

This prospective study included 38 malignant, 7 borderline, and 244 benign ovarian masses. The menopausal status, presence of septa, presence of papillary projections, location of the tumor, presence of ascites, presence of metastases, cancer antigen 125 level, tumor volume, septa thickness, and percentage of the solid component were included in the initial analysis. A second regression analysis was performed with the addition of Doppler parameters (location of blood flow and lowest resistive index) in the data set. Diagnostic performance of the 2 regression models and RMI were described and compared by generating receiver operating characteristic curves for each model.

RESULTS

The area under the curve values for the morphologic model (model 1), Doppler model (model 2), and RMI were 0.907, 0.971, and 0.889, respectively. Significance levels of model 1 and the RMI were similar (P = .23), whereas model 2 had a significantly higher area under the curve compared with both model 1 (P = .037) and the RMI (P = .018).

CONCLUSIONS

The addition of Doppler parameters in the regression model significantly increases the predictive performance. Nevertheless, in low-resource settings, the RMI remains the method of choice for distinguishing adnexal masses and referral to gynecologic oncology clinics.

Epithelial ovarian tumors: value of dynamic contrast-enhanced MR imaging and correlation with tumor angiogenesis

PURPOSE

To retrospectively evaluate the diagnostic performance of dynamic contrast material-enhanced magnetic resonance (MR) imaging for the characterization of ovarian epithelial tumors, by using histologic findings as the reference standard, and to correlate dynamic contrast-enhanced MR imaging findings with angiogenesis biomarkers.

MATERIALS AND METHODS

Ethics committee approval was obtained, with waiver of informed consent. Patients consented to having their data used for future retrospective research. Forty-one women (age range, 22-73 years) with 48 epithelial ovarian tumors underwent dynamic contrast-enhanced MR imaging before surgical excision. In case of bilateral tumors (n = 7), only the most complex tumor was analyzed. Thus, 41 tumors (12 benign, 13 borderline, and 16 invasive) were examined with dynamic contrast-enhanced MR imaging and immunohistochemical methods. Dynamic contrast-enhanced MR imaging parameters (enhancement amplitude [EA], time of half rising [T(max)], and maximal slope [MS]) were analyzed according to histopathologic findings, microvessel density, pericyte coverage index (PCI), and vascular endothelial growth factor receptor 2 (VEGFR-2) expression. Statistical analyses were performed by using Kruskal-Wallis, Fisher exact, and Spearman tests and receiver operating curve analysis.

RESULTS

EA was higher for invasive tumors than for benign (P < .001) and borderline (P < .05) tumors. T(max) was longer for benign tumors than for borderline (P < .05) and invasive (P < .01) tumors. MS was steeper for invasive tumors than for benign (P < .001) and borderline (P < .001) tumors. PCI was lower in invasive tumors than in borderline (P < .05) and benign (P < .05) tumors. Microvessels showed stronger immunohistochemical VEGFR-2 expression in invasive tumors than in benign or borderline tumors (P < .05). MS correlated with a lower PCI (r = -0.34, P = .04) and stronger VEGFR-2 expression by using both epithelial (r = 0.41, P < .01) and endothelial (r = 0.66, P < .001) cells.

CONCLUSION

The early enhancement patterns of ovarian epithelial tumors on dynamic contrast-enhanced MR images can help distinguish among benign, borderline, and invasive tumors and were found to correlate with tumoral angiogenic status.

Chronic pain reconsidered

Chronic pain has been traditionally defined by pain duration, but this approach has limited empirical support and does not account for chronic pain's multi-dimensionality. This study compared duration-based and prospective approaches to defining chronic pain in terms of their ability to predict future pain course and outcomes for primary care patients with three common pain conditions: back pain (n=971), headache (n=1078), or orofacial pain (n=455). At baseline, their chronic pain was classified retrospectively based on Pain Days in the prior six months and prospectively with a prognostic Risk Score identifying patients with "possible" or "probable" chronic pain. The 0-28 Risk Score was based on pain intensity, pain-related activity limitations, depressive symptoms, number of pain sites, and Pain Days. Pain and behavioral outcomes were assessed at six-month follow-up, and long-term opioid use was assessed two to five years after baseline. Risk Score consistently predicted clinically significant pain at six months better than did Pain Days alone (area under the curve of 0.74-0.78 for Risk Score vs. 0.63-0.73 for Pain Days). Risk Score was a stronger predictor of future SF-36 Physical Function, pain-related worry, unemployment, and long-term opioid use than Pain Days alone. Thus, for these three common pain conditions, a prognostic Risk Score had better predictive validity for pain outcomes than did pain duration alone. However, chronic pain appears to be a continuum rather than a distinct class, because long-term pain outcomes are highly variable and inherently uncertain.