Comparison of two different artificial neural networks for prostate biopsy indication in two different patient populations

A systematic review of the applications of Expert Systems (ES) and machine learning (ML) in clinical urology

BACKGROUND

Testing a hypothesis for 'factors-outcome effect' is a common quest, but standard statistical regression analysis tools are rendered ineffective by data contaminated with too many noisy variables. Expert Systems (ES) can provide an alternative methodology in analysing data to identify variables with the highest correlation to the outcome. By applying their effective machine learning (ML) abilities, significant research time and costs can be saved. The study aims to systematically review the applications of ES in urological research and their methodological models for effective multi-variate analysis. Their domains, development and validity will be identified.

METHODS

The PRISMA methodology was applied to formulate an effective method for data gathering and analysis. This study search included seven most relevant information sources: WEB OF SCIENCE, EMBASE, BIOSIS CITATION INDEX, SCOPUS, PUBMED, Google Scholar and MEDLINE. Eligible articles were included if they applied one of the known ML models for a clear urological research question involving multivariate analysis. Only articles with pertinent research methods in ES models were included. The analysed data included the system model, applications, input/output variables, target user, validation, and outcomes. Both ML models and the variable analysis were comparatively reported for each system.

RESULTS

The search identified n = 1087 articles from all databases and n = 712 were eligible for examination against inclusion criteria. A total of 168 systems were finally included and systematically analysed demonstrating a recent increase in uptake of ES in academic urology in particular artificial neural networks with 31 systems. Most of the systems were applied in urological oncology (prostate cancer = 15, bladder cancer = 13) where diagnostic, prognostic and survival predictor markers were investigated. Due to the heterogeneity of models and their statistical tests, a meta-analysis was not feasible.

CONCLUSION

ES utility offers an effective ML potential and their applications in research have demonstrated a valid model for multi-variate analysis. The complexity of their development can challenge their uptake in urological clinics whilst the limitation of the statistical tools in this domain has created a gap for further research studies. Integration of computer scientists in academic units has promoted the use of ES in clinical urological research.

PSA-based machine learning model improves prostate cancer risk stratification in a screening population

CONTEXT

The majority of prostate cancer diagnoses are facilitated by testing serum Prostate Specific Antigen (PSA) levels. Despite this, there are limitations to the diagnostic accuracy of PSA. Consideration of patient demographic factors and biochemical adjuncts to PSA may improve prostate cancer risk stratification. We aimed to develop a contemporary, accurate and cost-effective model based on objective measures to improve the accuracy of prostate cancer risk stratification.

METHODS

Data were collated from a local institution and combined with patient data retrieved from the Prostate, Lung, Colorectal and Ovarian Cancer screening Trial (PLCO) database. Using a dataset of 4548 patients, a machine learning model was developed and trained using PSA, free-PSA, age and free-PSA to total PSA (FTR) ratio.

RESULTS

The model was trained on a dataset involving 3638 patients and was then tested on a separate set of 910 patients. The model improved prediction for prostate cancer (AUC 0.72) compared to PSA alone (AUC 0.63), age (AUC 0.52), free-PSA (AUC 0.50) and FTR alone (AUC 0.65). When an operating point is chosen such that the sensitivity of the model is 80% the specificity of the model is 45.3%. The benefit in AUC secondary to the model was related to sample size, with AUC of 0.64 observed when a subset of the cohort was assessed.

CONCLUSIONS

Development of a dense neural network model improved the diagnostic accuracy in screening for prostate cancer. These results demonstrate an additional utility of machine learning methods in prostate cancer risk stratification when using biochemical parameters.

Machine learning methods can more efficiently predict prostate cancer compared with prostate-specific antigen density and prostate-specific antigen velocity

Background

Prostate-specific antigen (PSA)–based screening for prostate cancer has been widely performed, but its accuracy is unsatisfactory. To improve accuracy, building an effective statistical model using machine learning methods (MLMs) is a promising approach.

Methods

Data on continuous changes in the PSA level over the past 2 years were accumulated from 512 patients who underwent prostate biopsy after PSA screening. The age of the patients, PSA level, prostate volumes, and white blood cell count in urinalysis were used as input data for the MLMs. As MLMs, we evaluated the efficacy of three different techniques: artificial neural networks (ANNs), random forest, and support vector machine. Model performance was evaluated using area under the receiver operating characteristic curve (AUC) and compared with the PSA level and the conventional PSA–based parameters: PSA density and PSA velocity.

Results

When using two annual PSA testing, all receiver operating characteristic curves of the three MLMs were above the curve for the PSA level, PSA density, and PSA velocity. The AUCs of ANNs, random forest, and support vector machine were 0.69, 0.64, and 0.63, respectively. Those values were higher than the AUCs of the PSA level, PSA density, and PSA velocity, 0.53, 0.41, and 0.55, respectively. The accuracies of the MLMs (71.6% to 72.1%) were also superior to those of the PSA level (39.1%), PSA density (49.7%), and PSA velocity (54.9%). Among the MLMs, ANNs showed the most favorable AUC. The MLMs showed higher sensitivity and specificity than conventional PSA–based parameters. The model performance did not improve when using three annual PSA testing.

Conclusion

The present retrospective study results indicate that machine learning techniques can predict prostate cancer with significantly better AUCs than those of PSA density and PSA velocity.

Biomarker development, from bench to bedside

This review describes studies performed by our group and other laboratories in the field aimed at development of biomarkers not only for cancer but also for other diseases. The markers covered include tumor-associated trypsin inhibitor (TATI), tumor-associated trypsin (TAT), human chorionic gonadotropin (hCG), prostate-specific antigen (PSA) and their various molecular forms, their biology and diagnostic use. The discovery of TATI was the result of a hypothesis-driven project aimed at finding new biomarkers for ovarian cancer among urinary peptides. TATI has since proved to be a useful prognostic marker for several cancers. Recently, it has been named Serine Peptidase Inhibitor Kazal Type 1 (SPINK1) after being rediscovered by several groups as a tumor-associated peptide by gene expression profiling and proteomic techniques and shown to promote tumor development by stimulating the EGF receptor. To explain why a trypsin inhibitor is strongly expressed in some cancers, research focused on the protease that it inhibited led to the finding of tumor-associated trypsin (TAT). Elevated serum concentrations of TAT-2 were found in some cancer types, but fairly high background levels of pancreatic trypsinogen-2 limited the use of TAT-2 for cancer diagnostics. However, trypsinogen-2 and its complex with α1-protease inhibitor proved to be very sensitive and specific markers for pancreatitis. Studies on hCG were initiated by the need to develop more rapid and sensitive pregnancy tests. These studies showed that serum from men and non-pregnant women contains measurable concentrations of hCG derived from the pituitary. Subsequent development of assays for the subunits of hCG showed that the β subunit of hCG (hCGβ) is expressed at low concentrations by most cancers and that it is a strong prognostic marker. These studies led to the formation of a working group for standardization of hCG determinations and the development of new reference reagents for several molecular forms of hCG. The preparation of intact hCG has been adopted as the fifth international standard by WHO. Availability of several well-defined forms of hCG made it possible to characterize the epitopes of nearly 100 monoclonal antibodies. This will facilitate design of immunoassays with pre-defined specificity. Finally, the discovery of different forms of immunoreactive PSA in serum from a prostate cancer patient led to identification of the complex between PSA and α1-antichymotrypsin, and the use of assays for free and total PSA in serum for improved diagnosis of prostate cancer. Epitope mapping of PSA antibodies and establishment of PSA standards has facilitated establishment well-standardized assays for the various forms of PSA.

Do prostate cancer risk models improve the predictive accuracy of PSA screening? A meta-analysis

BACKGROUND

Despite the extensive development of risk prediction models to aid patient decision-making on prostate screening, it is unknown whether these models could improve predictive accuracy of PSA testing to detect prostate cancer (PCa). The objective of this study was to perform a systematic review to identify PCa risk models and to assess the model's performance to predict PCa by conducting a meta-analysis.

DESIGN

A systematic literature search of Medline was conducted to identify PCa predictive risk models that used at least two variables, of which one of the variables was prostate-specific antigen (PSA) level. Model performance (discrimination and calibration) was assessed. Prediction models validated in ≥5 study populations and reported area under the curve (AUC) for prediction of any or clinically significant PCa were eligible for meta-analysis. Summary AUC and 95% CIs were calculated using a random-effects model.

RESULTS

The systematic review identified 127 unique PCa prediction models; however, only six models met study criteria for meta-analysis for predicting any PCa: Prostataclass, Finne, Karakiewcz, Prostate Cancer Prevention Trial (PCPT), Chun, and the European Randomized Study of Screening for Prostate Cancer Risk Calculator 3 (ERSPC RC3). Summary AUC estimates show that PCPT does not differ from PSA testing (0.66) despite performing better in studies validating both PSA and PCPT. Predictive accuracy to discriminate PCa increases with Finne (AUC = 0.74), Karakiewcz (AUC = 0.74), Chun (AUC = 0.76) and ERSPC RC3 and Prostataclass have the highest discriminative value (AUC = 0.79), which is equivalent to doubling the sensitivity of PSA testing (44% versus 21%) without loss of specificity. The discriminative accuracy of PCPT to detect clinically significant PCa was AUC = 0.71. Calibration measures of the models were poorly reported.

CONCLUSIONS

Risk prediction models improve the predictive accuracy of PSA testing to detect PCa. Future developments in the use of PCa risk models should evaluate its clinical effectiveness in practice.

Artificial neural networks and prostate cancer--tools for diagnosis and management

Artificial neural networks (ANNs) are mathematical models that are based on biological neural networks and are composed of interconnected groups of artificial neurons. ANNs are used to map and predict outcomes in complex relationships between given 'inputs' and sought-after 'outputs' and can also be used find patterns in datasets. In medicine, ANN applications have been used in cancer diagnosis, staging and recurrence prediction since the mid-1990s, when an enormous effort was initiated, especially in prostate cancer detection. Modern ANNs can incorporate new biomarkers and imaging data to improve their predictive power and can offer a number of advantages as clinical decision making tools, such as easy handling of distribution-free input parameters. Most importantly, ANNs consider nonlinear relationships among input data that cannot always be recognized by conventional analyses. In the future, complex medical diagnostic and treatment decisions will be increasingly based on ANNs and other multivariate models.

External validation of an artificial neural network and two nomograms for prostate cancer detection

Background. Multivariate models are used to increase prostate cancer (PCa) detection rate and to reduce unnecessary biopsies. An external validation of the artificial neural network (ANN) “ProstataClass” (ANN-Charité) was performed with daily routine data. Materials and Methods. The individual ANN predictions were generated with the use of the ANN application for PSA and free PSA assays, which rely on age, tPSA, %fPSA, prostate volume, and DRE (ANN-Charité). Diagnostic validity of tPSA, %fPSA, and the ANN was evaluated by ROC curve analysis and comparisons of observed versus predicted probabilities. Results. Overall, 101 (35.8%) PCa were detected. The areas under the ROC curve (AUCs) were 0.501 for tPSA, 0.669 for %fPSA, 0.694 for ANN-Charité, 0.713 for nomogram I, and 0.742 for nomogram II, showing a significant advantage for nomogram II (P = 0.009) compared with %fPSA while the other model did not differ from %fPSA (P = 0.15 and P = 0.41). All models overestimated the predicted PCa probability. Conclusions. Beside ROC analysis, calibration is an important tool to determine the true value of using a model in clinical practice. The worth of multivariate models is limited when external validations were performed without knowledge of the circumstances of the model's development.

Internal validation of an artificial neural network for prostate biopsy outcome

OBJECTIVES

To carry out an internal validation of the retrospectively trained artificial neural network (ANN) 'ProstataClass'.

METHODS

A prospectively collected database of 393 patients undergoing 8-12 core prostate biopsy was analyzed. Data of these patients were applied to the online available ANN 'ProstataClass' using the Elecsys total prostate-specific antigen (tPSA) and free PSA (fPSA) assays. Beside the internal validation of the ANN 'ProstataClass' an additional ANN (named as ANN internal validation: ANNiv) only using the 393 prospective patient data was evaluated. The new ANN model was constructed with the MATLAB Neural Network Toolbox. Diagnostic accuracy was evaluated by receiver operator characteristic (ROC) curves comparing the areas under the ROC curves (AUC) and specificities at 90% and 95% sensitivity.

RESULTS

Within a tPSA range of 1.0-22.8 ng/mL, 229 men (58.3%) had prostate cancer (PCa). tPSA, %fPSA and the number of positive digital rectal examinations (DRE) differed significantly from the cohort of patients of the ANN 'ProstataClass', whereas age and prostate volume were comparable. AUCs for tPSA, %fPSA and the ANN 'ProstataClass' were 0.527, 0.726 and 0.747 (P = 0.085 between %fPSA and ANN). The AUC of the ANNiv (0.754) was significantly better compared with %fPSA (P = 0.021), whereas the AUC of two ANN models built on external cohorts (0.726 and 0.729) showed no differences to %fPSA and the other ANN models.

CONCLUSIONS

Significant differences of DRE status and %fPSA medians decrease the power of the 'ProstataClass' ANN in the internal validation cohort. The effect of retrospective data evaluation the 'ProstataClass' cohort and prospective fPSA measurement may be responsible for %fPSA differences. All ANN models built with different PSA and fPSA assays performed equally if applied to the two cohorts.

Techniques and predictive models to improve prostate cancer detection

The use of prostate-specific antigen (PSA) as a screening test remains controversial. There have been several attempts to refine PSA measurements to improve its predictive value. These modifications, including PSA density, PSA kinetics, and the measurement of PSA isoforms, have met with limited success. Therefore, complex statistical and computational models have been created to assess an individual's risk of prostate cancer more accurately. In this review, the authors examined the methods used to modify PSA as well as various predictive models used in prostate cancer detection. They described the mathematical underpinnings of these techniques along with their intrinsic strengths and weaknesses, and they assessed the accuracy of these methods, which have been shown to be better than physicians' judgment at predicting a man's risk of cancer. Without understanding the design and limitations of these methods, they can be applied inappropriately, leading to incorrect conclusions. These models are important components in counseling patients on their risk of prostate cancer and also help in the design of clinical trials by stratifying patients into different risk categories. Thus, it is incumbent on both clinicians and researchers to become familiar with these tools. Cancer 2009;115(13 suppl):3085-99. (c) 2009 American Cancer Society.

A [-2]proPSA-based artificial neural network significantly improves differentiation between prostate cancer and benign prostatic diseases

BACKGROUND

The aim of this study was to combine the new automated Access [-2]proPSA (p2PSA) assay with a percent free PSA (%fPSA) based artificial neural network (ANN) or logistic regression (LR) model to enhance discrimination between patients with prostate cancer (PCa) and with no evidence of malignancy (NEM) and to detect aggressive PCa.

METHODS

Sera from 311 PCa patients and 275 NEM patients were measured with the p2PSA, total PSA (tPSA) and free PSA (fPSA) assays on Access immunoassay technology (Beckman Coulter, Fullerton, CA) within the 0-30 ng/ml tPSA range. Four hundred seventy-five patients (264 PCa, 211 NEM) had a tPSA of 2-10 ng/ml. LR models and leave-one-out (LOO) ANN models with Bayesian regularization by using tPSA, %fPSA, p2PSA/fPSA (%p2PSA), age and prostate volume were constructed and compared by receiver-operating characteristic (ROC) curve analysis.

RESULTS

The ANN and LR model each utilizing %p2PSA, %fPSA, tPSA and age, but without prostate volume, reached the highest AUCs (0.85 and 0.84) and best specificities (ANN: 62.1% and 45.5%; LR: 53.1% and 41.2%) compared with tPSA (22.7% and 11.4%) and %fPSA (45.5% and 26.1%) at 90% and 95% sensitivity. The %p2PSA furthermore distinguished better than tPSA and %fPSA between pT2 and pT3, and Gleason sum <7 and >or=7 PCa.

CONCLUSIONS

The automated p2PSA assay offers a new tool to improve PCa detection, and especially aggressive PCa detection. Incorporation of %p2PSA into an ANN and LR model further enhances the diagnostic accuracy to differentiate between malignant and non-malignant prostate diseases.

Emerging Biomarkers for the Diagnosis and Prognosis of Prostate Cancer

Background: Early detection of prostate cancer (CaP), the most prevalent cancer and the second-leading cause of death in men, has proved difficult, and current detection methods are inadequate. Prostate-specific antigen (PSA) testing is a significant advance for early diagnosis of patients with CaP.

Content: PSA is produced almost exclusively in the prostate, and abnormalities of this organ are frequently associated with increased serum concentrations. Because of PSA’s lack of specificity for CaP, however, many patients undergo unnecessary biopsies or treatments for benign or latent tumors, respectively. Thus, a more specific method of CaP detection is required to augment or replace screening with PSA. The focus recently has been on creating cost-effective assays for circulating protein biomarkers in the blood, but because of the heterogeneity of CaP, it has become clear that this effort will be a formidable challenge. Each marker will require proper validation to ensure clinical utility. Although much work has been done on variations of the PSA test (i.e., velocity, density, free vs bound, proisoforms) with limited usefulness, there are many emerging markers at various stages of development that show some promise for CaP diagnosis. These markers include kallikrein-related peptidase 2 (KLK2), early prostate cancer antigen (EPCA), PCA3, hepsin, prostate stem cell antigen, and α-methylacyl-CoA racemase (AMACR). We review biomarkers under investigation for the early diagnosis and management of prostate cancer.

Summary: It is hoped that the use of panels of markers can improve CaP diagnosis and prognosis and help predict the therapeutic response in CaP patients.

An artificial neural network for five different assay systems of prostate-specific antigen in prostate cancer diagnostics

OBJECTIVE

To compare separate prostate-specific antigen (PSA) assay-specific artificial neural networks (ANN) for discrimination between patients with prostate cancer (PCa) and no evidence of malignancy (NEM).

PATIENTS AND METHODS

In 780 patients (455 with PCa, 325 with NEM) we measured total PSA (tPSA) and free PSA (fPSA) with five different assays: from Abbott (AxSYM), Beckman Coulter (Access), DPC (Immulite 2000), and Roche (Elecsys 2010) and with tPSA and complexed PSA (cPSA) assays from Bayer (ADVIA Centaur). ANN models were developed with five input factors: tPSA, percentage free/total PSA (%fPSA), age, prostate volume and digital rectal examination status for each assay separately to examine two tPSA ranges of 0-10 and 10-27 ng/mL.

RESULTS

Compared with the median tPSA concentrations (range from 4.9 [Bayer] to 6.11 ng/mL [DPC]) and especially the median %fPSA values (range from 11.2 [DPC] to 17.4%[Abbott], for tPSA 0-10 ng/mL), the areas under the receiver operating characteristic curves (AUC) for all calculated ANN models did not significantly differ from each other. The AUC were: 0.894 (Abbott), 0.89 (Bayer), 0.895 (Beckman), 0.882 (DPC) and 0.892 (Roche). At 95% sensitivity the specificities were without significant differences, whereas the individual absolute ANN outputs differed markedly.

CONCLUSIONS

Despite only slight differences, PSA assay-specific ANN models should be used to optimize the ANN outcome to reduce the number of unnecessary prostate biopsies. We further developed the ANN named 'ProstataClass' to provide clinicians with an easy to use tool in making their decision about follow-up testing.

The comparability of models for predicting the risk of a positive prostate biopsy with prostate-specific antigen alone: a systematic review

CONTEXT

The sensitivity and specificity profile of measuring levels of prostate-specific antigen (PSA) to select men for prostate biopsy is not optimal. This has prompted the construction of nomograms and artificial neural networks (ANNs) to increase the performance of PSA measurements.

OBJECTIVE

A systematic review of nomograms and ANNs designed to predict the risk of a positive prostate biopsy for cancer was conducted in order to determine their value versus measuring PSA levels alone.

EVIDENCE ACQUISITION

Medical Literature Analysis and Retrieval System Online (U.S. National Library of Medicine's life science database; MEDLINE) was searched using the terms "nomogram" "artificial neural network" and "prostate cancer" for dates up to and including July 2007 and was supplemented by manual searches of reference lists. Included studies used an assessment tool to examine the risk of a positive prostate biopsy in a man without a known cancer diagnosis. Intramodel comparisons with evaluation of PSA levels alone, and intermodel comparisons of area under the curve (AUC) from receiver operating characteristic (ROC) curves were conducted. Individual case examples were also used for comparisons.

EVIDENCE SYNTHESIS

Twenty-three studies examining 36 models were included. With the exception of two studies, all the models had AUC values of 0.70 or greater, with eight reporting an AUC of >/=0.80 and four (all ANNs) reporting an AUC >/=0.85, with variable validation status. Fourteen studies compared the AUC with PSA levels alone: all showed a benefit from using AUCs which varied from 0.02 to 0.26. Of the 16 external validation comparisons, in 13 the AUC was lower in the external population than in the model population.

CONCLUSIONS

Nomograms and ANNs produce improvements in AUC over measurement of PSA levels alone, but many lack external validation. Where this is available, the benefits are often diminished, although most remain significantly better than with evaluation of PSA levels alone. In men without additional risk factors, PSA cutoff values alone provide a relatively precise risk estimate, but if additional risk factors are known, PSA values alone are less accurate.