Computer-aided diagnosis of breast cancer: artificial neural network approach for optimized merging of mammographic features

Integrating Artificial Intelligence and Machine Learning Into Cancer Clinical Trials

The practice of oncology requires analyzing and synthesizing abundant data. From the patient's workup to determine eligibility to the therapies received to the post-treatment surveillance, practitioners must constantly juggle, evaluate, and weigh decision-making based on their best understanding of information at hand. These complex, multifactorial decisions have a tremendous opportunity to benefit from data-driven machine learning (ML) methods to drive opportunities in artificial intelligence (AI). Within the past 5 years, we have seen AI move from simply a promising opportunity to being used in prospective trials. Here, we review recent efforts of AI in clinical trials that have moved the needle towards improved prediction of actionable outcomes, such as predicting acute care visits, short term mortality, and pathologic extranodal extension. We then pause and reflect on how these AI models ask a different question than traditional statistics models that readers may be more familiar with; how then should readers conceptualize and interpret AI models that they are not as familiar with. We end with what we believe are promising future opportunities for AI in oncology, with an eye towards allowing the data to inform us through unsupervised learning and generative models, rather than asking AI to perform specific functions.

Digital Mammography in Breast Cancer: Additive Value of Radiomics of Breast Parenchyma

Background Previous studies have suggested that breast parenchymal texture features may reflect the biologic risk factors associated with breast cancer development. Therefore, combining the characteristics of normal parenchyma from the contralateral breast with radiomic features of breast tumors may improve the accuracy of digital mammography in the diagnosis of breast cancer. Purpose To determine whether the addition of radiomic analysis of contralateral breast parenchyma to the characterization of breast lesions with digital mammography improves lesion classification over that with radiomic tumor features alone. Materials and Methods This HIPAA-compliant, retrospective study included 182 patients (age range, 25-90 years; mean age, 55.9 years ± 14.9) who underwent mammography between June 2002 and July 2009. There were 106 malignant and 76 benign lesions. Automatic lesion segmentation and radiomic analysis were performed for each breast lesion. Radiomic texture analysis was applied in the normal regions of interest in the contralateral breast parenchyma to assess the mammographic parenchymal patterns. The classification performance of both individual features and the output from a Bayesian artificial neural network classifier was evaluated with the leave-one-patient-out method by using the area under the receiver operating characteristic curve (AUC) as the figure of merit in the task of differentiating between malignant and benign lesions. Results The performance of the combined lesion and parenchyma classifier in the differentiation between malignant and benign mammographic lesions was better than that with the lesion features alone (AUC = 0.84 ± 0.03 vs 0.79 ± 0.03, respectively; P = .047). Overall, six radiomic features-spiculation, margin sharpness, size, circularity from the tumor feature set, and skewness and power law beta from the parenchymal feature set-were selected more than 50% of the time during the feature selection process on the combined feature set. Conclusion Combining quantitative radiomic data from tumors with contralateral parenchyma characterizations may improve diagnostic accuracy for breast cancer. © RSNA, 2019 Online supplemental material is available for this article. See also the editorial by Shaffer in this issue.

Machine Learning Approaches for Predicting Radiation Therapy Outcomes: A Clinician's Perspective

Radiation oncology has always been deeply rooted in modeling, from the early days of isoeffect curves to the contemporary Quantitative Analysis of Normal Tissue Effects in the Clinic (QUANTEC) initiative. In recent years, medical modeling for both prognostic and therapeutic purposes has exploded thanks to increasing availability of electronic data and genomics. One promising direction that medical modeling is moving toward is adopting the same machine learning methods used by companies such as Google and Facebook to combat disease. Broadly defined, machine learning is a branch of computer science that deals with making predictions from complex data through statistical models. These methods serve to uncover patterns in data and are actively used in areas such as speech recognition, handwriting recognition, face recognition, "spam" filtering (junk email), and targeted advertising. Although multiple radiation oncology research groups have shown the value of applied machine learning (ML), clinical adoption has been slow due to the high barrier to understanding these complex models by clinicians. Here, we present a review of the use of ML to predict radiation therapy outcomes from the clinician's point of view with the hope that it lowers the "barrier to entry" for those without formal training in ML. We begin by describing 7 principles that one should consider when evaluating (or creating) an ML model in radiation oncology. We next introduce 3 popular ML methods--logistic regression (LR), support vector machine (SVM), and artificial neural network (ANN)--and critique 3 seminal papers in the context of these principles. Although current studies are in exploratory stages, the overall methodology has progressively matured, and the field is ready for larger-scale further investigation.

Breast cancer risk estimation with artificial neural networks revisited: discrimination and calibration

BACKGROUND

Discriminating malignant breast lesions from benign ones and accurately predicting the risk of breast cancer for individual patients are crucial to successful clinical decisions. In the past, several artificial neural network (ANN) models have been developed for breast cancer-risk prediction. All studies have reported discrimination performance, but not one has assessed calibration, which is an equivalently important measure for accurate risk prediction. In this study, the authors have evaluated whether an artificial neural network (ANN) trained on a large prospectively collected dataset of consecutive mammography findings can discriminate between benign and malignant disease and accurately predict the probability of breast cancer for individual patients.

METHODS

Our dataset consisted of 62,219 consecutively collected mammography findings matched with the Wisconsin State Cancer Reporting System. The authors built a 3-layer feedforward ANN with 1000 hidden-layer nodes. The authors trained and tested their ANN by using 10-fold cross-validation to predict the risk of breast cancer. The authors used area the under the receiver-operating characteristic curve (AUC), sensitivity, and specificity to evaluate discriminative performance of the radiologists and their ANN. The authors assessed the accuracy of risk prediction (ie, calibration) of their ANN by using the Hosmer-Lemeshow (H-L) goodness-of-fit test.

RESULTS

Their ANN demonstrated superior discrimination (AUC, 0.965) compared with the radiologists (AUC, 0.939; P<.001). The authors' ANN was also well calibrated as shown by an H-L goodness of fit P-value of .13.

CONCLUSIONS

The authors' ANN can effectively discriminate malignant abnormalities from benign ones and accurately predict the risk of breast cancer for individual abnormalities.

Evaluation of computer-aided diagnosis on a large clinical full-field digital mammographic dataset

RATIONALE AND OBJECTIVES

To convert and optimize our previously developed computerized analysis methods for use with images from full-field digital mammography (FFDM) for breast mass classification to aid in the diagnosis of breast cancer.

MATERIALS AND METHODS

An institutional review board approved protocol was obtained, with waiver of consent for retrospective use of mammograms and pathology data. Seven hundred thirty-nine FFDM images, which contained 287 biopsy-proven breast mass lesions, of which 148 lesions were malignant and 139 lesions were benign, were retrospectively collected. Lesion margins were delineated by an expert breast radiologist and were used as the truth for lesion-segmentation evaluation. Our computerized image analysis method consisted of several steps: 1) identified lesions were automatically extracted from the parenchymal background using computerized segmentation methods; 2) a set of image characteristics (mathematic descriptors) were automatically extracted from image data of the lesions and surrounding tissues; and 3) selected features were merged into an estimate of the probability of malignancy using a Bayesian artificial neural network classifier. Performance of the analyses was evaluated at various stages of the conversion using receiver-operating characteristic analysis.

RESULTS

An area under the curve value of 0.81 was obtained in the task of distinguishing between malignant and benign mass lesions in a round-robin by case evaluation on the entire FFDM dataset. We failed to show a statistically significant difference (P = .83) compared to results from our previous study in which the computerized classification was performed on digitized screen-film mammograms.

CONCLUSIONS

Our computerized analysis methods developed on digitized screen-film mammography can be converted for use with FFDM. Results show that the computerized analysis methods for the diagnosis of breast mass lesions on FFDM are promising, and can potentially be used to aid clinicians in the diagnostic interpretation of FFDM.

Bayesian networks of BI-RADStrade mark descriptors for breast lesion classification

We investigated Bayesian network structure learning and probability estimation from mammographic feature data in order to classify breast lesions into different pathological categories. We compared the learned networks to naive Bayes classifiers, which are similar to the expert systems previously investigated for breast lesion classification. The learned network structures reflect the difference in the classification of biopsy outcome and the invasiveness of malignant lesions for breast masses and microcalcifications. The difference between masses and microcalcifications should be taken into consideration when interpreting systems for automatic pathological classification of breast lesions. The difference may also affect use of these systems for tasks such as estimating the sampling error of biopsy.

Computer-aided mass detection based on ipsilateral multiview mammograms

RATIONALE AND OBJECTIVES

Recent reports on advances in computer-aided detection (CAD) indicate that current schemes miss early-stage breast cancers and result in a relatively large false-positive detection rate in order to achieve a high sensitivity rate for mass detection. This paper is inspired by the interpretation procedure from mammographers. The abnormal diagnosis can be derived from multiple views but is not available through single-view image analysis.

MATERIALS AND METHODS

A new multiview CAD system for early-stage breast cancer detection, which is based on modifying the optimized CAD algorithms from our prior single-view CAD system for constructing an adaptive ipsilateral multiview concurrent CAD system, is presented in this paper. The selection and design for the training and testing ipsilateral multiview mammogram databases are described here.

RESULTS

The performance evaluation of the developed ipsilateral multiview CAD system using free-response receiver operating characteristic analysis and computerized receiver operating characteristic experiments are presented. The results indicated that the proposed multiview CAD system is significantly superior to the single-view CAD systems based on statistically standard P-values.

CONCLUSION

This paper addresses a very important and timely project. It is related to two main problems regarding the development of breast cancer detection and diagnosis: early-stage detection and diagnosis of breast cancer with digital mammogram, and overall improvement of CAD system performance for clinical implementation. In order to improve the efficacy, accuracy, and efficiency of the current CAD scheme, an entirely new class of CAD method is required. This paper is unique in that a comprehensive and state-of-the-art approach is proposed for the CAD scheme of digital mammography. From the design aspect of the CAD scheme, the proposed ipsilateral multiview CAD method is innovative and quite different from current single-view CAD methods.

Artificial neural network predictive model for allergic disease using single nucleotide polymorphisms data

The purpose of this study was to develop a novel diagnostic prediction method for allergic diseases from the data of single nucleotide polymorphisms (SNPs) using an artificial neural network (ANN). We applied the prediction method to four allergic diseases, such as atopic dermatitis (AD), allergic conjunctivitis (AC), allergic rhinitis (AR) and bronchial asthma (BA), and verified its predictive ability. Almost all the learning data were precisely predicted. Regarding the evaluation data, the learned ANN model could correctly predict a diagnosis with more than 78% accuracy. We also analyzed the SNP data using multiple regression analysis (MRA). Using the MRA model, less than 10% of patients with the above allergic diseases were correctly diagnosed, while this figure was more than 75% for persons without allergic diseases. From these results, it was shown that the ANN model was superior to the MRA model with respect to predictive ability of allergic diseases. Moreover, we used two different methods to convert the genetic polymorphism data into numerical data. Using both methods, diagnostic predictions were quite precise and almost the same predictive abilities were observed. This is the first study showing the application and usefulness of an ANN for the prediction of allergic diseases based on SNP data.

Predicting radiotherapy-induced cardiac perfusion defects

The purpose of this work is to compare the efficacy of mathematical models in predicting the occurrence of radiotherapy-induced left ventricular perfusion defects assessed using single-photon emission computed tomography (SPECT). The basis of this study is data from 73 left-sided breast/ chestwall patients treated with tangential photon fields. The mathematical models compared were three commonly used parametric models [Lyman normal tissue complication probability (LNTCP), relative serialty (RS), generalized equivalent uniform dose (gEUD)] and a nonparametric model (Linear discriminant analysis--LDA). Data used by the models were the left ventricular dose--volume histograms, or SPECT-based dose-function histograms, and the presence/absence of SPECT perfusion defects 6 months postradiation therapy (21 patients developed defects). For the parametric models, maximum likelihood estimation and F-tests were used to fit the model parameters. The nonparametric LDA model step-wise selected features (volumes/function above dose levels) using a method based on receiver operating characteristics (ROC) analysis to best separate the groups with and without defects. Optimistic (upper bound) and pessimistic (lower bound) estimates of each model's predictive capability were generated using ROC curves. A higher area under the ROC curve indicates a more accurate model (a model that is always accurate has area = 1). The areas under these curves for different models were used to statistically test for differences between them. Pessimistic estimates of areas under the ROC curve using dose-volume histogram/ dose-function histogram inputs, in order of increasing prediction accuracy, were LNTCP (0.79/0.75), RS (0.80/0.77), gEUD (0.81/0.78), and LDA (0.84/0.86). Only the LDA model benefited from SPECT-based regional functional information. In general, the LDA model was statistically superior to the parametric models. The LDA model selected as features the left ventricular volumes above approximately 23 Gy (V23), essentially volume in field, and 33 Gy (V33), as best separating the groups with and without defects. In conclusion, the nonparametric LDA model appears to be a more accurate predictor of radiotherapy-induced left ventricular perfusion defects than commonly used parametric models.

Qualitative diagnosis of calvarial metastasis by neural network and logistic regression

RATIONALE AND OBJECTIVES

To simplify the diagnostic features used by an artificial neural network compared with logistic regression (LR) in the diagnosis of calvarial metastasis with computed tomography and analyze their accuracy.

MATERIALS AND METHODS

Twenty-one of 167 patients with calvarial lesions were found to have metastasis. Clinical and computed tomography data were used for LR and neural network models. Both models were tested with the leave-one-out method. The final results of each model were compared using the area under receiver operating characteristic curve (Az).

RESULTS

The neural network identified metastasis significantly more successfully than LR with an Az of 0.9324 +/- 0.0386 versus 0.9192 +/- 0.0373, P = .01. The most important features selected by the LR and neural network were age and edge definition.

CONCLUSION

Neural networks offer wide possibilities over statistics for the study of calvarial metastases other than their minimum clinical and radiologic features for diagnosis.

Application of artificial neural networks to the analysis of dynamic MR imaging features of the breast

The discriminative ability of established diagnostic criteria for MRI of the breast is assessed, and their relative relevance using artificial neural networks (ANNs) is determined. A total of 89 women with 105 histopathologically verified breast lesions (73 invasive cancers, 2 in situ cancers, and 30 benign lesions) were included in this study. A T1-weighted 3D FLASH sequence was acquired before and seven times after the intravenous administration of gadopentetate dimeglumine at a dose of 0.2 mmol/kg body weight. ANN models were built to test the discriminative ability of kinetic, morphologic, and combined MR features. The subjects were randomly divided into two parts: a training set of 59 lesions and a verification set of 46 lesions. The training set was used for learning, and the performance of each model was evaluated on the verification set by measuring the area under the ROC curve (Az). An optimally minimized model was constructed using the most relevant input variables that were determined by the automatic relevance determination (ARD) method. ANN models were compared with the performance of a human reader. Margin type, time-to-peak enhancement, and washout ratio showed the highest discriminative ability among diagnostic criteria and comprised the minimized model. Compared with the expert radiologist (Az = 0.799), using the same prediction scale, the minimized ANN model performed best (Az = 0.771), followed by the best kinetic (Az = 0.743), the maximized (Az = 0.727), and the morphologic model (Az = 0.678). The performance of a neural network prediction model is comparable to that of an expert radiologist. A neurostatistical approach is preferred for the analysis of diagnostic criteria when many parameters are involved and complex nonlinear relationships exist in the data set.

Advances in biomedical informatics for the management of cancer

Increased access to health care, and advances in education and technology have resulted in a larger proportion of the population having longer life expectancy. The strong correlation between age and cancer has resulted in a major healthcare problem for this century, and until recently cancer has defied any long-lasting cure. However, progress, especially in the field of biomedical informatics, promises a successful prediction and possibly a permanent cure for cancer within the next two decades. Biomedical informatics-with its roots in computer science, biomedical engineering, biostatistics, and mathematics-helps to bring the patient closer to the physician, facilitates access to specialist information and knowledge bases across the world, and makes it possible to identify genetic expression profiles for malignant or cancerous cells. This paper reviews the new research findings in biomedical informatics, working toward the ultimate goal of successfully predicting cancer, solving complex problems in prevention and treatment of cancer, and perhaps completely curing the scourge of cancer.

Does training in the Breast Imaging Reporting and Data System (BI-RADS) improve biopsy recommendations or feature analysis agreement with experienced breast imagers at mammography?

PURPOSE

To determine whether training in the Breast Imaging Reporting and Data System (BI-RADS) improves observer performance and agreement with the consensus of experienced breast imagers with regard to mammographic feature analysis and final assessment.

MATERIALS AND METHODS

A test set of mammograms was developed, with 54 proven lesions consisting of 28 masses (nine [32%] malignancies) and 26 microcalcifications (10 [38%] malignancies). Three experienced breast imagers reviewed cases independently and by means of consensus. Twenty-three practicing mammogram-interpreting physicians reviewed mammograms before and after a day's lectures on BI-RADS. Observer performance before and after training was measured by means of agreement (kappa) with consensus description and assessments, rate of biopsy of malignant and benign lesions, and areas under receiver operating characteristic (ROC) curves. Performance was also measured for 11 participants 2-3 months after training.

RESULTS

Improved agreement with consensus feature analysis was found for mass margins and/or asymmetries, with a pretraining generalized kappa value of 0.36 and a posttraining generalized kappa value of 0.41. Similar improvement was seen for description of calcification morphology (pretraining kappa value of 0.36 improving to 0.44 after training). No improvement was seen in describing calcification distribution. Final assessments were more consistent after training, with a pretraining kappa value of 0.31, as compared with 0.45 after training. The mean biopsy rate for malignant lesions improved from 73% (range, 53%-89%) before training to 88% (range, 74%-100%) after training, with minimal increase in mean biopsy rate of benign lesions (43% [range, 26%-60%] before to 51% [range, 31%-63%] after training), and no net change in area under the ROC curve, as compared with histopathologic findings. For the subset of participants with delayed follow-up, no significant decline in posttraining results was seen.

CONCLUSION

BI-RADS training resulted in improved agreement with the consensus of experienced breast imagers for feature analysis and final assessment. It is important that trainees showed improved rates of recommending biopsy for malignant lesions. This effect was maintained over 2-3 months.

Perceptron error surface analysis: a case study in breast cancer diagnosis

Perceptrons are typically trained to minimize mean square error (MSE). In computer-aided diagnosis (CAD), model performance is usually evaluated according to other more clinically relevant measures. The purpose of this study was to investigate the relationship between MSE and the area (A(z)) under the receiver operating characteristic (ROC) curve and the high-sensitivity partial ROC area ((0.90)A'(z)). A perceptron was used to predict lesion malignancy based on two mammographic findings and patient age. For each performance measure, the error surface in weight space was visualized. Comparison of the surfaces indicated that minimizing MSE tended to maximize A(z), but not (0.90)A'(z).

Cross-institutional evaluation of BI-RADS predictive model for mammographic diagnosis of breast cancer

OBJECTIVE

Given a predictive model for identifying very likely benign breast lesions on the basis of Breast Imaging Reporting and Data System (BI-RADS) mammographic findings, this study evaluated the model's ability to generalize to a patient data set from a different institution.

MATERIALS AND METHODS

The artificial neural network model underwent three trials: it was optimized over 500 biopsy-proven lesions from Duke University Medical Center or "Duke," evaluated on 1,000 similar cases from the University of Pennsylvania Health System or "Penn," and reoptimized for Penn.

RESULTS

Trial A's Duke-only model yielded 98% sensitivity, 36% specificity, area index (A(z)) of 0.86, and partial A(z) of 0.51. The cross-institutional trial B yielded 96% sensitivity, 28% specificity, A(z) of 0.79, and partial A(z) of 0.28. The decreases were significant for both A(z) (p = 0.017) and partial A(z) (p < 0.001). In trial C, the model reoptimized for the Penn data yielded 96% sensitivity, 35% specificity, A(z) of 0.83, and partial A(z) of 0.32. There were no significant differences compared with trial B for specificity (p = 0.44) or partial A(z) (p = 0.46), suggesting that the Penn data were inherently more difficult to characterize.

CONCLUSION

The BI-RADS lexicon facilitated the cross-institutional test of a breast cancer prediction model. The model generalized reasonably well, but there were significant performance decreases. The cross-institutional performance was encouraging because it was not significantly different from that of a reoptimized model using the second data set at high sensitivities. This study indicates the need for further work to collect more data and to improve the robustness of the model.

Application of the mutual information criterion for feature selection in computer-aided diagnosis

The purpose of this study was to investigate an information theoretic approach to feature selection for computer-aided diagnosis (CAD). The approach is based on the mutual information (MI) concept. MI measures the general dependence of random variables without making any assumptions about the nature of their underlying relationships. Consequently, MI can potentially offer some advantages over feature selection techniques that focus only on the linear relationships of variables. This study was based on a database of statistical texture features extracted from perfusion lung scans. The ultimate goal was to select the optimal subset of features for the computer-aided diagnosis of acute pulmonary embolism (PE). Initially, the study addressed issues regarding the approximation of MI in a limited dataset as it is often the case in CAD applications. The MI selected features were compared to those features selected using stepwise linear discriminant analysis and genetic algorithms for the same PE database. Linear and nonlinear decision models were implemented to merge the selected features into a final diagnosis. Results showed that the MI is an effective feature selection criterion for nonlinear CAD models overcoming some of the well-known limitations and computational complexities of other popular feature selection techniques in the field.

Improvement of mammographic mass characterization using spiculation meausures and morphological features

We are developing new computer vision techniques for characterization of breast masses on mammograms. We had previously developed a characterization method based on texture features. The goal of the present work was to improve our characterization method by making use of morphological features. Toward this goal, we have developed a fully automated, three-stage segmentation method that includes clustering, active contour, and spiculation detection stages. After segmentation, morphological features describing the shape of the mass were extracted. Texture features were also extracted from a band of pixels surrounding the mass. Stepwise feature selection and linear discriminant analysis were employed in the morphological, texture, and combined feature spaces for classifier design. The classification accuracy was evaluated using the area Az under the receiver operating characteristic curve. A data set containing 249 films from 102 patients was used. When the leave-one-case-out method was applied to partition the data set into trainers and testers, the average test Az for the task of classifying the mass on a single mammographic view was 0.83 +/- 0.02, 0.84 +/- 0.02, and 0.87 +/- 0.02 in the morphological, texture, and combined feature spaces, respectively. The improvement obtained by supplementing texture features with morphological features in classification was statistically significant (p = 0.04). For classifying a mass as malignant or benign, we combined the leave-one-case-out discriminant scores from different views of a mass to obtain a summary score. In this task, the test Az value using the combined feature space was 0.91 +/- 0.02. Our results indicate that combining texture features with morphological features extracted from automatically segmented mass boundaries will be an effective approach for computer-aided characterization of mammographic masses.

Overview of breast imaging

In summary, it is an exciting time in breast imaging with many tools being brought to bear on an ever more common problem. The challenge for this decade will be to develop optimal cost-effective strategies to use all the tools now available with minimal discomfort and disfigurement to the patient.

The development and evaluation of CADMIUM: a prototype system to assist in the interpretation of mammograms

We have developed CADMIUM, a novel approach for the design of systems to assist in the interpretation of medical images. CADMIUM uses symbolic reasoning to relate information obtained from image processing to the decisions radiologists take. The approach is based on a symbolic decision procedure which has already been used successfully in a variety of nonimaging clinical decision systems. In CADMIUM this decision procedure is extended with models of three generic image interpretation tasks: detection, measurement and classification of image features. The extended procedure is used to construct the lines of reasoning needed in each task and to control the acquisition of information by image processing. CADMIUM has been evaluated as an aid to the differential diagnosis of microcalcifications on mammographic images. Radiographers who had been trained to interpret images performed better when using the advice provided by the system.

A neural network to predict symptomatic lung injury

A nonlinear neural network that simultaneously uses pre-radiotherapy (RT) biological and physical data was developed to predict symptomatic lung injury. The input data were pre-RT pulmonary function, three-dimensional treatment plan doses and demographics. The output was a single value between 0 (asymptomatic) and 1 (symptomatic) to predict the likelihood that a particular patient would become symptomatic. The network was trained on data from 97 patients for 400 iterations with the goal to minimize the mean-squared error. Statistical analysis was performed on the resulting network to determine the model's accuracy. Results from the neural network were compared with those given by traditional linear discriminate analysis and the dose-volume histogram reduction (DVHR) scheme of Kutcher. Receiver-operator characteristic (ROC) analysis was performed on the resulting network which had Az = 0.833 +/- 0.04. (Az is the area under the ROC curve.) Linear discriminate multivariate analysis yielded an Az = 0.813 +/- 0.06. The DVHR method had Az = 0.521 +/- 0.08. The network was also used to rank the significance of the input variables. Future studies will be conducted to improve network accuracy and to include functional imaging data.

Evaluating the performance of detection algorithms in digital mammography

The initial and relative evaluation of computer methodologies developed for assisting diagnosis in mammography is usually done by comparing the computer output to ground truth data provided by experts and/or biopsy. Reported studies, however, give little information on how the performance indices of computer assisted diagnosis (CAD) algorithms are determined in this initial stage of evaluation. Several strategies exist in the estimation of the true positive (TP) and false positive (FP) rates with respect to ground truth. Adopting one strategy over another yields different performance rates that can be over- or underestimates of the true performance. Furthermore, the estimation of pairs of TP and FP rates gives a partial picture of the performance of an algorithm. It is shown in this work that new performance indices are needed to fully describe the degree of detection (part or whole) and the type of detection (single calcification, cluster of calcifications, mass, or artifact). Several evaluation strategies were tested. The one that yielded the most realistic performances included the following criteria: The detected area should be at least 50% of the true area and no more than four times the true area in order to be considered TP. At least three true calcifications should be detected to within 1 cm2 with nearest neighbor distances of less than square root(2) cm for a cluster to be considered TP. Separate detection measures should be established and used for artifacts and naturally occurring structures to maximize the benefits of the evaluation. Finally, it is critical that CAD investigators provide information on the tested image set as well as the criteria used for the evaluation of the algorithms to allow comparisons and better understanding of their methodologies.

Effect of patient history data on the prediction of breast cancer from mammographic findings with artificial neural networks

RATIONALE AND OBJECTIVES

The authors evaluated the contribution of medical history data to the prediction of breast cancer with artificial neural network (ANN) models based on mammographic findings.

MATERIALS AND METHODS

Three ANNs were developed: The first used 10 Breast Imaging Reporting and Data System (BI-RADS) variables; the second, the BI-RADS variables plus patient age; the third, the BI-RADS variables, patient age, and seven other history variables, for a total of 18 inputs. Performance of the ANNs and the original radiologist's impression were evaluated with five metrics: receiver operating characteristic area index (Az); specificity at given sensitivities of 100%, 98%, and 95%; and positive predictive value.

RESULTS

All three ANNs consistently outperformed the radiologist's impression over all five performance metrics. The patient-age variable was particularly valuable. Adding the age variable to the basic ANN model, which used only the BI-RADS findings, significantly improved Az (P = .028). In fact, replacing all history data with just the age variable resulted in virtually no changes for Az or specificity at 98% sensitivity (P = .324 and P = .410, respectively).

CONCLUSION

Patient age was an important variable for the prediction of breast cancer from mammographic findings with the ANNs. For this data set, all history data could be replaced with age alone.

Simulation model of mammographic calcifications based on the American College of Radiology Breast Imaging Reporting and Data System, or BIRADS

RATIONALE AND OBJECTIVES

The authors developed and evaluated a method for the simulation of calcification clusters based on the guidelines of the Breast Imaging Reporting and Data System of the American College of Radiology. They aimed to reproduce accurately the relative and absolute size, shape, location, number, and intensity of real calcifications associated with both benign and malignant disease.

MATERIALS AND METHODS

Thirty calcification clusters were simulated by using the proposed model and were superimposed on real, negative mammograms digitized at 30 microns and 16 bits per pixel. The accuracy of the simulation was evaluated by three radiologists in a blinded study.

RESULTS

No statistically significant difference was observed in the observers' evaluation of the simulated clusters and the real clusters. The observers' classification of the cluster types seemed to be a good approximation of the intended types from the simulation design.

CONCLUSION

This model can provide simulated calcification clusters with well-defined morphologic, distributional, and contrast characteristics for a variety of applications in digital mammography.

Predictive model for the diagnosis of intraabdominal abscess

RATIONALE AND OBJECTIVES

The authors investigated the use of an artificial neural network (ANN) to aid in the diagnosis of intraabdominal abscess.

MATERIALS AND METHODS

An ANN was constructed based on data from 140 patients who underwent abdominal and pelvic computed tomography (CT) between January and December 1995. Input nodes included data from clinical history, physical examination, laboratory investigation, and radiographic study. The ANN was trained and tested on data from all 140 cases by using a round-robin method and was compared with linear discriminate analysis. A receiver operating characteristic curve was generated to evaluate both predictive models.

RESULTS

CT examinations in 50 cases were positive for abscess. This finding was confirmed by means of laboratory culture of aspirations from CT-guided percutaneous drainage in 38 patients, ultrasound-guided percutaneous drainage in five patients, surgery in five patients, and characteristic appearance on CT scans without aspiration in two patients. CT scans in 90 cases were negative for abscess. The sensitivity and specificity of the ANN in predicting the presence of intraabdominal abscess were 90% and 51%, respectively. Receiver operating characteristic analysis showed no statistically significant difference in performance between the two predictive models.

CONCLUSION

The ANN is a useful tool for determining whether an intraabdominal abscess is present. It can be used to set priorities for CT examinations in order to expedite treatment in patients believed to be more likely to have an abscess.

Calvarial eosinophilic granuloma: diagnostic models and image feature selection with a neural network

RATIONALE AND OBJECTIVES

The authors analyzed the accuracy of diagnostic features used by an artificial neural network compared with logistic-regression analysis in the diagnosis with computed tomography (CT) of calvarial eosinophilic granuloma.

MATERIALS AND METHODS

Thirty-one of 167 patients with calvarial lesions were found to have eosinophilic granuloma. Clinical and CT data were used for logistic-regression and neural network models. Both models were tested by using the leave-one-out method. The final results of each model were compared by means of the area under the receiver operating characteristic curve (Az).

RESULTS

Identification of eosinophilic granuloma was significantly more accurate with the neural network than with logistic regression (Az = 0.9846 +/- 0.0157 [standard deviation] vs 0.9117 +/- 0.0373) (P = .001). The most important diagnostic features identified with the neural network were patient age and marginal sclerosis. For logistic regression, the most important features were age, shape, and lobularity.

CONCLUSION

The neural network is a useful tool for analyzing the features of calvarial eosinophilic granuloma. Age and marginal sclerosis are important diagnostic features.

Breast masses in women treated for childhood cancer: incidence and screening guidelines

BACKGROUND

The authors' aims were to define the frequency, characteristics, and methods of detection of breast masses in young women treated for childhood cancer and to develop screening guidelines for the growing population of long term survivors.

METHODS

The authors reviewed medical records of all female patients treated for malignancy at a childhood cancer center over a 34-year period to identify those who developed a breast mass and to determine the cumulative incidence of breast cancer as a second primary cancer.

RESULTS

A breast mass was identified in 66 patients who had been diagnosed with a malignancy at a median age of 13.8 years (range, 0.4-24.4 years). Masses were initially detected by breast self-detection in 32 and clinical examination in 28; the method of detection was unknown in 6 cases. Breast lesions were benign in 41 patients and malignant in 26; 1 patient had both a benign and a malignant lesion. Of the 26 malignant masses, 14 represented metastases of the primary malignancy and 12 were primary breast cancers as second primary cancers. The median interval to a primary breast cancer as a second primary cancer was 13.6 years (range, 9.2-24.4 years), and the median age at detection was 27.7 years (range, 12.5-43.1 years). The 25-year cumulative incidence of primary breast cancer as a second primary cancer in this cohort was 1.7% (95% CI, 0.4%-2.9%). This represented a 20-fold increase (95% CI, 10-36) over the expected incidence in age-matched and race-matched controls.

CONCLUSIONS

Young women treated for childhood cancer have a significantly increased risk of breast cancer compared with age-matched controls. For this group of patients, the authors recommend patient education regarding this risk and the importance of properly conducted self-examination as the foundation of breast cancer screening. In addition, clinical and mammographic screening should be instituted at a younger age and performed more frequently than recommended for the general population of women.

Computerized classification of malignant and benign microcalcifications on mammograms: texture analysis using an artificial neural network

We investigated the feasibility of using texture features extracted from mammograms to predict whether the presence of microcalcifications is associated with malignant or benign pathology. Eighty-six mammograms from 54 cases (26 benign and 28 malignant) were used as case samples. All lesions had been recommended for surgical biopsy by specialists in breast imaging. A region of interest (ROI) containing the microcalcifications was first corrected for the low-frequency background density variation. Spatial grey level dependence (SGLD) matrices at ten different pixel distances in both the axial and diagonal directions were constructed from the background-corrected ROI. Thirteen texture measures were extracted from each SGLD matrix. Using a stepwise feature selection technique, which maximized the separation of the two class distributions, subsets of texture features were selected from the multi-dimensional feature space. A backpropagation artificial neural network (ANN) classifier was trained and tested with a leave-one-case-out method to recognize the malignant or benign microcalcification clusters. The performance of the ANN was analysed with receiver operating characteristic (ROC) methodology. It was found that a subset of six texture features provided the highest classification accuracy among the feature sets studied. The ANN classifier achieved an area under the ROC curve of 0.88. By setting an appropriate decision threshold, 11 of the 28 benign cases were correctly identified (39% specificity) without missing any malignant cases (100% sensitivity) for patients who had undergone biopsy. This preliminary result indicates that computerized texture analysis can extract mammographic information that is not apparent by visual inspection. The computer-extracted texture information may be used to assist in mammographic interpretation, with the potential to reduce biopsies of benign cases and improve the positive predictive value of mammography.

Construction of a Bayesian network for mammographic diagnosis of breast cancer

Bayesian networks use the techniques of probability theory to reason under uncertainty, and have become an important formalism for medical decision support systems. We describe the development and validation of a Bayesian network (MammoNet) to assist in mammographic diagnosis of breast cancer. MammoNet integrates five patient-history features, two physical findings, and 15 mammographic features extracted by experienced radiologists to determine the probability of malignancy. We outline the methods and issues in the system's design, implementation, and evaluation. Bayesian networks provide a potentially useful tool for mammographic decision support.