A method for evaluating computer programs for electrocardiographic interpretation. II. Application to version D of the PHS program and the Mayo Clinic program of 1968

Comparison of PR, QRS, and QT interval measurements by seven ECG interpretation programs

BACKGROUND

Electrocardiograph-generated measurements of PR, QRS, and QT intervals are generally thought to be more precise than manual measurements on paper records. However, the performance of different programs has not been well compared.

METHODS

Routinely obtained digital electrocardiograms (ECGs), including over 500 pediatric ECGs, were used to create over 2000 10 s analog ECGs that were replayed through seven commercially available electrocardiographs. The measurements for PR interval, QRS duration, and QT interval made by each program were extracted and compared against each other (using the median of the programs after correction for program bias) and the population mean values.

RESULTS

Small but significant systematic biases were seen between programs. The smallest and largest variation from the population mean differed by 4.7 ms for PR intervals, 5.8 ms for QRS duration, and 12.4 ms for QT intervals. In pairwise comparison programs showed similar accuracy for most ECGs, with the average absolute errors at the 75th percentile for PR intervals being 4-6 ms from the median, QRS duration 4-8 ms, and QT interval 6-10 ms. However, substantial differences were present in the numbers and extent of large, clinically significant errors (e.g at the 98th percentile), for which programs differed by a factor of two for absolute errors, as well as differences in the mix of overestimations and underestimations.

CONCLUSIONS

When reading digital ECGs, users should be aware that small systematic differences exist between programs and that there may be large clinically important errors in difficult cases.

Diagnostic performance of a computer-based ECG rhythm algorithm

We examined the accuracy of computer-based rhythm interpretation from one electrocardiograph manufacturer (GE Healthcare Technologies MUSE software 005C) in 4297 consecutive recordings in a university hospital setting. Overreading was performed by either of 2 experienced cardiologists, and all disagreements with the initial computer rhythm statement were reviewed by the second cardiologist to achieve physician consensus used as the "gold standard" for rhythm diagnosis. Overall, 13.2% (565/4297) of computer-based rhythm statements required revision, but excluding tracings with pacemakers, the revision rate was 7.8% (307/3954), including 3.8% involving the primary rhythm diagnosis and 3.9% involving definition of ectopic complexes. The false-negative rate for sinus rhythm was only 1.3%, but a computer diagnosis of sinus rhythm was incorrect in 9.9% of other rhythms. The false-negative rate for atrial fibrillation was 9.2%, whereas a computer diagnosis of atrial fibrillation was incorrect in 1.1% of other rhythms, including sinus. Computer diagnosis of paced rhythms remains problematic, and physician overreading to correct computer-based electrocardiogram rhythm diagnoses remains mandatory.

A Patient With Bicuspid Aorta and Intercoronary Continuity A Rare Variant of Coronary Circulation

Bidirectional flow in patients with normal coronary arteries is an indicator of intercoronary continuity, a rare variant of coronary circulation, distinct from collaterals. The case of an 18 year old Turkish male with bicuspid aorta and intercoronary artery is reported and different aspects of this interesting entity are emphasized.

Extended protocol for demonstration of dual AV nodal physiology

The incidence of dual atrioventricular (AV) nodal physiology was evaluated in 22 patients (14 males, 8 females, age 52 +/- 18 years) undergoing electrophysiology studies for evaluation of ventricular tachycardia/nonsustained ventricular tachycardia (n = 11), supraventricular tachycardia (n = 5), and syncope (n = 6). Patients with AV node reentrant tachycardia were excluded. Thirteen patients had normal left ventricular function and nine patients (seven with coronary artery disease, two with dilated cardiomyopathy) had depressed left ventricular function. Single atrial extrastimuli (A2) were introduced after eight-beat drives at paced cycle lengths of 550 msec and 400 or 450 msec beginning at coupling intervals of 650 and 500 or 550 msec, respectively. The coupling interval was decreased at 10-msec intervals until AV node or atrial refractoriness. A second atrial extrastimulus (A3) was then added. A2 was fixed at 50 msec greater than the atrial or AV nodal refractory period. A3 was coupled to A2 at 650 and 500 or 550 msec and decremented as with single extrastimulation. Dual AV nodal physiology was defined by a 50-msec increase in A2H2 or A3H3 with a 10-msec decrement in the coupling interval or a discontinuous H1H2 versus A1A2 or H2H3 versus A2A3 curve. Using a single extrastimulus, 1 of 22 patients demonstrated dual AV nodal physiology. Using double extrastimuli, an additional four patients with dual AV nodal physiology were identified. The occurrence of dual AV nodal physiology determined using double extrastimuli is increased compared to using only a single extrastimulus (P = 0.03).(ABSTRACT TRUNCATED AT 250 WORDS)

Stability of computer ECG amplitude measurements in the presence of noise. The CSE Working Party

An important feature of an ECG analysis program is its ability to provide reliable measurements under various operating conditions, e.g., on noise-free and noisy ECGs. Therefore, within the European cooperative project "Common Standards For Quantitative Electrocardiography" (CSE), the accuracy and stability of ECG measurements obtained by several computer programs has been compared. To investigate the stability of measurements two sets of 10 ECGs with and without seven different high- and low-frequency types of noise--altogether 160 electrocardiograms and 160 vectorcardiograms--have been analyzed by eight electrocardiographic and five vectorcardiographic computer programs. The stability of measurement was tested with respect to results obtained for the noise-free recordings. In a previous paper, the influence of noise on wave boundary recognition has been reported. In the present paper, the effect of noise on amplitude measurements and on problems of waveform definitions within the QRS complex are described. The results indicate that programs analyzing an averaged beat exhibit less variability than programs which measure every complex or a selected beat. Comparability and stability of measurements could be improved if a standardized procedure for amplitude references were to be introduced. In addition, the stability of QRS waveform labelling could be improved if waveforms' minimum amplitude and duration were to be validated against the noise level which itself should be determined by a standardized procedure.

A reference data base for multilead electrocardiographic computer measurement programs

In an effort to standardize and evaluate the performance of electrocardiographic computer measurement programs, a 15 lead reference library has been developed based on simultaneously recorded standard 12 lead and orthogonal XYZ lead data. A set of 250 electrocardiograms (ECGs) with selected abnormalities was analyzed by a group of five referee cardiologists and 11 different 12 lead and 6 XYZ computer programs. Attention was focused on the exact determination of the onsets and offsets of P, QRS and T waves. The referees performed their task on highly amplified, selected complexes from the library in a two round process. Median results of the referees coincided best with the median derived from all programs. An analysis of stability proved that the combined program median was a robust reference. However, some individual program results were widely divergent. Paired t tests demonstrated earlier onset for P and QRS (p less than 0.001), as well as later offset for P and T waves in the median 12 lead than in the XYZ results. Significant differences also existed among results obtained by programs analyzing all standard ECG leads at one time, the so-called multilead programs, and those obtained by the conventional standard three lead analysis programs. As a consequence, the derived P, PR, QRS and QT interval measurements varied quite widely among the various programs. Significant differences were also observed among measurements of Q, R and S duration. Some programs showed Q waves that were on the average 6 ms (p less than 0.001) longer than those of others. This may significantly influence diagnostic performance.(ABSTRACT TRUNCATED AT 250 WORDS)

Signal quality of resting electrocardiograms

Artifact may cause errors of technical origin when ECGs are interpreted by automatic methods. Baseline shift and high-frequency noise content of minimal and typical length ECG records from pediatric and adult populations were measured to allow prediction of both the likelihood of interpretation errors of technical origin and the number of reacquistions needed to obtain an artifact-free record. Ages of the 708 subjects in this study ranged from 2 weeks to 27 years. When a baseline shift of 0.25 mv (exceeded in 7% of the R-R intervals in the database) or a noise content greater than 15 muv RMS (exceeded in 6% of the R-R intervals in the database) within six seconds of three simultaneous leads was declared an unacceptable artifact, then 68% of the records from 0-4 year olds and 31% of the records from adults (greater than 19 years), were rejected on the basis of technical quality. These failure rates mean that, on the average, 3.1 tries would be needed to obtain an artifact-free record from 0-4 year olds; 1.4 tries would be needed for adults. If acquisition is done interactively, the measurement time for a 6-second, 3-lead group would be increased by 13 seconds for 0-4 years olds and by three seconds for adults in order to assure adequate signal quality for computer-assisted analysis.

The influence of editing on the performance of a computer program for serial comparison of electrocardiograms

In this study, three tests of computer ECG analysis function were conducted: 1) interpretation of serial tracings using the single record analysis program, i.e. without serial comparisons; 2) interpretation of the same tracings using a computer-comparison program; and 3) interpretation of these tracings using not only a computer-comparison program but also an editing program which allows amending of each serial record and insertion of any corrections into the computer data base before the next comparison. There was an over-all reduction of 84% in the number of statement changes needed to produce a correct final report by using the combined comparison and editing program. When this combined program was used, 81% of all computer interpretations required no alteration, --a considerable saving of time in analyzing serial electrocardiographic tracings. Serial comparison was required in in 27--45% of all ECG's taken in four medical centers.

Comparative diagnostic performance of the Telemed computer ECG program

One thousand consecutive ECG's from an ambulatory population of patients with suspected or proven cardiac disease were evaluated using two versions of the Telemed computerized ECG system. Only minor differences were found between the two programs. In version 6 vs. version 5, 87% vs. 90% of 287 normal ECG's were correctly classified and 93% vs. 96% of abnormal ECG's were correctly classified; the percent of acceptable diagnostic agreement was 86.2% and 87.4% respectively (NS). The sensitivity for arrhythmia detection, transmural inferior infarction and ST-T wave abnormalities was slightly greater in version 6. The increased sensitivity was not accompanied by decreased specificity. The sensitivity for left ventricular hypertrophy decreased from 95.2% to 91.4% in version 6 with a slight increase in specificity (95.2% to 97.0%). In conclusion, criteria changes in the most recent version of the Telemed program have not resulted in a major change in diagnostic performance. Arrhythmia detection is slightly but not significantly improved.

Computer evaluation of human circulation based on non-invasive methods

A discriminating program was developed on the basis of time, dynamic and simply calculated parameters of non-invasive tracings recorded in the supine position. Data were derived from ECG, PCG and the indirect carotid pulse curve. The optimal program, formed after 40 experimental processes, was in 85% agreement with the clinical diagnosis. To improve the decision process, we created a new 'test again' group, in addition to the healthy and sick groups. The 'test again' group included 16.5% of the examined subjects. At the same time, there was 75.6% agreement with the clinical diagnosis, and 7.9% disagreement. The risk factors, which could be demonstrated as part of the 'errors' called attention to undetected heart failure. The descriminating function found to be best, was fed into a small computer (R-10). Records for evaluation were entered on magnetic tape to the computer which measured automatically the necessary parameters and printed out the 'decision': 'healthy', 'test again!', or 'cardiac patient', as well as other data, such as systolic time intervals, etc. There is a wide potential application for automated computer system based on non-invasive parameters.

Limitations of the computer in electrocardiographic interpretation

Use of the computer for electrocardiographic interpretation has steadily increased over the past decade. Although acceptance by a majority of physicians has been slower than originally anticipated, it now appears assured. Nevertheless, the approach has limitations. These are primarily a result of the lack of objectivity in clinical electrocardiographic criteria for both measurement and diagnosis. The limitations are best judged by reviewing experience with a variety of programs. Currently this still involves abstracting from the reports of developers. Although not all developers have analyzed every portion of their programs, in general the data from one program apply to others; that is, all are with minor exceptions at the same "state of the art." Awareness of the limits of computer performance allows a physician to use the computer properly in his current electrocardiographic practice.