The Relative Accuracy of a Variety of Medical Diagnostic Programs

Acute abdominal pain is one of the most widely studied applications of computer-aided diagnosis. The usual approach is to apply Bayes’ theorem with the assumption of conditional independence (“independence Bayes”). We compared various approaches to designing diagnostic programs for abdominal pain of suspected gynaecological origin. The methods range from statistical to knowledge-based. All programs were evaluated using a database of 1,270 cases collected retrospectively. Our results suggest that in this application no significant improvement in accuracy can be made by taking interactions into account, either by statistical or by knowledge-based means; independence Bayes is near-optimal. As far as accuracy is concerned, there appears to be little point in pursuing knowledge-based approaches. However, the “nearest neighbours” method using a new metric appears to be at least as accurate as independence Bayes. We argue that the nearest neighbours method is more suitable than independence Bayes for clinical use because of greater accountability.

Case-Based Explanation for Medical Diagnostic Programs, with an Example from Gynaecology

One of the most accountable methods of providing machine assistance in medical diagnosis is to retrieve and display similar previously diagnosed cases from a database. In practice, however, classifying cases according to the diagnoses of their nearest neighbours is often significantly less accurate than other statistical classifiers. In this paper the transparency of the nearest neighbours method is combined with the accuracy of another statistical method. This is achieved by using the other statistical method to define a measure of similarity between the presentations of two cases. The diagnosis of abdominal pain of suspected gynaecological origin is used as a case study to evaluate this method. Bayes’ theorem, with the usual assumption of conditional independence, is used to define a metric on cases. This new metric was found to correspond as well as Hamming distance to the clinical notion of “similarity” between cases, while significantly increasing accuracy to that of the Bayes’ method itself.

Role of Central Corneal Thickness on Baseline Parameters and Progression of Visual Fields in Open Angle Glaucoma

Purpose

To evaluate the relationship of central corneal thickness (CCT) to baseline visual field parameters and visual field progression in patients with primary open-angle glaucoma (POAG).

Methods

Charts of consecutive patients with POAG were reviewed to obtain visual field data. Visual field was measured by standard threshold static perimetry. Variables analyzed included mean deviation (MD) and pattern standard deviation (PSD).

Results

A total of 121 eyes examined over 4 years were evaluated. A significant negative relationship between CCT and PSD (correlation coefficient: −0.02, p<0.05) was found. Analyses comparing CCT to change in PSD and MD (visual field progression) were statistically not significant.

Conclusions

Patients with thinner corneas initially present with a greater visual field defect, indicating that thin corneas may contribute to advanced glaucomatous damage at the time of diagnosis. However, CCT does not seem to be a significant risk factor for progression of the disease.

Prevent Blindness America visual field screening study. The Prevent Blindness America Glaucoma Advisory Committee

PURPOSE

To assess the screening efficacy and practical use of two portable devices to detect moderate to severe visual field loss rapidly in population screening.

METHODS

Henson visual field analysis and Damato campimetry for glaucoma were performed in a healthy adult population, to determine false-positive rates; in established glaucoma patients and suspects, to determine false-negative rates; and in a general adult population, to assess practical use in actual screenings.

RESULTS

There were no false-positive test failures among the 82 normal subjects who completed the Henson two-step screening. Eighty of 83 normal subjects passed Damato campimetry, resulting in a false-positive rate of 3.6%. Among 83 glaucoma suspects and patients, the Henson test identified 49 (84%) of 58 subjects whose full-threshold fields from Humphrey perimetry were abnormal, 38 (97%) of 39 of whom had moderate to severe visual field loss. The Damato campimeter detected 55 (81%) of 68 subjects with any pathologic loss on full-threshold visual fields, 44 (92%) of 48 of whom had moderate to severe visual field loss. Among 1,278 subjects tested in general population screenings, 55 subjects (4.3%) failed either or both tests.

CONCLUSIONS

The Henson visual field analyzer can discriminate moderately to severely diseased from normal visual fields with high sensitivity and specificity. The Damato campimeter can reliably detect moderate to severe visual field loss with a tolerably low false-positive rate. To overcome the weakness of glaucoma screening by tonometry alone, some forms of visual field testing may be acceptably brief (cost effective) and accurate (sensitive and specific).

The relative accuracy of a variety of medical diagnostic programs

Acute abdominal pain is one of the most widely studied applications of computer-aided diagnosis. The usual approach is to apply Bayes' theorem with the assumption of conditional independence ("independence Bayes"). We compared various approaches to designing diagnostic programs for abdominal pain of suspected gynaecological origin. The methods range from statistical to knowledge-based. All programs were evaluated using a database of 1,270 cases collected retrospectively. Our results suggest that in this application no significant improvement in accuracy can be made by taking interactions into account, either by statistical or by knowledge-based means; independence Bayes is near-optimal. As far as accuracy is concerned, there appears to be little point in pursuing knowledge-based approaches. However, the "nearest neighbours" method using a new metric appears to be at least as accurate as independence Bayes. We argue that the nearest neighbours method is more suitable than independence Bayes for clinical use because of greater accountability.

Case-based explanation for medical diagnostic programs, with an example from gynaecology

One of the most accountable methods of providing machine assistance in medical diagnosis is to retrieve and display similar previously diagnosed cases from a database. In practice, however, classifying cases according to the diagnoses of their nearest neighbours is often significantly less accurate than other statistical classifiers. In this paper the transparency of the nearest neighbours method is combined with the accuracy of another statistical method. This is achieved by using the other statistical method to define a measure of similarity between the presentations of two cases. The diagnosis of abdominal pain of suspected gynaecological origin is used as a case study to evaluate this method. Bayes' theorem, with the usual assumption of conditional independence, is used to define a metric on cases. This new metric was found to correspond as well as Hamming distance to the clinical notion of "similarity" between cases, while significantly increasing accuracy to that of the Bayes' method itself.

The design and construction of a medical simulation model

This paper describes the design, construction and validation of a probabilistic simulation model of patients who present with abdominal pain. The model incorporates text-book medical knowledge, clinical judgment, and statistics collected from real cases. The knowledge representation combines techniques of Bayesian network modelling with ideas of logistic discrimination. The model is shown to generate convincing, realistic cases; large numbers of artificial cases with no missing observations can be generated quickly. This should make the model a useful tool for investigating factors which limit achievable computer accuracy in the diagnosis of abdominal pain.

A probabilistic rule-based expert system

This paper explores a medical expert system combining techniques of Bayesian network modelling with ideas of weighted inference rules. The weights of the individual rules can be estimated objectively from a training set of actual cases; and they can be used in a Monte Carlo stimulation to estimate objectively conditional probabilities of diagnosis given particular combinations of symptoms. The paper describes and evaluates a medical expert system built according to this design. The diagnostic accuracy of the program was found to be similar to that obtained through the usual application of Bayes theorem with the assumption of conditional independence of symptoms given disease, even though the Bayesian classifier has more than 70 times as many numerical parameters. The method may be promising in cases where small training sets do not permit accurate estimation of large numbers of parameters.

Limits to diagnostic accuracy

This paper explores the limits to computer-aided medical diagnosis. A specific application area (the diagnosis of abdominal pain of suspected gynaecological origin) is chosen, and the factors limiting the accuracy of computer programs are investigated by means of a simulation model which has been shown previously to generate realistic cases. The model is used to generate arbitrarily large training and test sets. The results suggest that, while statistical dependencies exist amongst symptoms and signs, there is little to be gained by taking interactions into account. However, failure to record all possible observations does limit diagnostic accuracy significantly. The results suggest that near-optimal diagnostic accuracy (75-80%) can be obtained with a training set size of 10(5) cases simply by applying Bayes' theorem with the usual assumption of conditional independence.

The relation between visual sensitivity and intraocular pressure in normal eyes

Intraocular pressure and flicker modulation sensitivity at 25 and 40 Hz were measured in 22 normal observers, with an age range from 20-71 years. Significant correlations up to 0.67 were found between intraocular pressure and flicker sensitivity at several points in the visual field. There was no correlation between flicker sensitivity and age of the observers. Thus intraocular pressure may affect neuronal function in the normal eye.

Spectral sensitivity and color discrimination changes in glaucoma and glaucoma-suspect patients

Color vision changes may occur early in the course of glaucoma and may precede visual field loss. Glaucoma suspects, having raised intraocular pressure and no diagnostic optic nerve head or visual field changes, may also have color vision loss. Unfortunately, the instruments used in the studies that have demonstrated these color vision changes were not feasible for routine clinical use; likewise, the studies did not carefully control for the effects of small pupil size and age or did not point to the underlying mechanisms responsible. We studied 19 glaucoma patients, 19 glaucoma suspects, and age-matched controls for each group by means of the Farnsworth D-15 panel test, a desaturated version of the D-15 test, and by measures of spectral increment threshold. Minor modifications of the Farnsworth D-15 panel test produce highly significant differentiation of glaucoma and glaucoma-suspect patients from age-matched normal groups. Further, spectral increment thresholds, with a two-degree spectral target flashed at either 1 or 25 Hz on a bright white background, show that both achromatic and chromatic sensitivity are significantly reduced when compared with their age-matched normals. Pupil size does not seem to be a significant factor. These results suggest that the function of two different ganglion cell populations is affected in glaucoma and that glaucoma may produce functional loss in the central foveal area earlier in the disease process than previously believed.

Decreasing sample volume used in the “Sequential Multichannel Analyzer Computerized” (SMAC)

We describe modifications to two cartridges, those for alkaline phosphatase and creatinine, on an 18-channel SMAC (Sequential Multichannel Analyzer Computerized), which allow all 18 tests to be run with a commonly prediluted sample. This decreases the required sample size and eliminates the problem of fibrin clots in the stream-splitter on the main sample line. The modifications can be made rapidly and require few new components. Accuracy, precision, linearity, and carryover are acceptable after these modifications, as judged by comparison with premodification data. Some financial savings ensue from the decreased consumption of reagents and the fewer repeat analyses necessitated by clotting problems.

Decreasing sample volume used in the "Sequential Multichannel Analyzer Computerized" (SMAC)

We describe modifications to two cartridges, those for alkaline phosphatase and creatinine, on an 18-channel SMAC (Sequential Multichannel Analyzer Computerized), which allow all 18 tests to be run with a commonly prediluted sample. This decreases the required sample size and eliminates the problem of fibrin clots in the stream-splitter on the main sample line. The modifications can be made rapidly and require few new components. Accuracy, precision, linearity, and carryover are acceptable after these modifications, as judged by comparison with premodification data. Some financial savings ensue from the decreased consumption of reagents and the fewer repeat analyses necessitated by clotting problems.

Decreasing sample volume used in the “Sequential Multichannel Analyzer Computerized” (SMAC)

We describe modifications to two cartridges, those for alkaline phosphatase and creatinine, on an 18-channel SMAC (Sequential Multichannel Analyzer Computerized), which allow all 18 tests to be run with a commonly prediluted sample. This decreases the required sample size and eliminates the problem of fibrin clots in the stream-splitter on the main sample line. The modifications can be made rapidly and require few new components. Accuracy, precision, linearity, and carryover are acceptable after these modifications, as judged by comparison with premodification data. Some financial savings ensue from the decreased consumption of reagents and the fewer repeat analyses necessitated by clotting problems.