High Precision Digitization of Paper-Based ECG Records: A Step Toward Machine Learning

Automatic detection of cardiac conditions from photos of electrocardiogram captured by smartphones

BACKGROUND

Researchers have developed machine learning-based ECG diagnostic algorithms that match or even surpass cardiologist level of performance. However, most of them cannot be used in real-world, as older generation ECG machines do not permit installation of new algorithms.

OBJECTIVE

To develop a smartphone application that automatically extract ECG waveforms from photos and to convert them to voltage-time series for downstream analysis by a variety of diagnostic algorithms built by researchers.

METHODS

A novel approach of using objective detection and image segmentation models to automatically extract ECG waveforms from photos taken by clinicians was devised. Modular machine learning models were developed to sequentially perform waveform identification, gridline removal, and scale calibration. The extracted data were then analysed using a machine learning-based cardiac rhythm classifier.

RESULTS

Waveforms from 40 516 scanned and 444 photographed ECGs were automatically extracted. 12 828 of 13 258 (96.8%) scanned and 5399 of 5743 (94.0%) photographed waveforms were correctly cropped and labelled. 11 604 of 12 735 (91.1%) scanned and 5062 of 5752 (88.0%) photographed waveforms achieved successful voltage-time signal extraction after automatic gridline and background noise removal. In a proof-of-concept demonstration, an atrial fibrillation diagnostic algorithm achieved 91.3% sensitivity, 94.2% specificity, 95.6% positive predictive value, 88.6% negative predictive value and 93.4% F1 score, using photos of ECGs as input.

CONCLUSION

Object detection and image segmentation models allow automatic extraction of ECG signals from photos for downstream diagnostics. This novel pipeline circumvents the need for costly ECG hardware upgrades, thereby paving the way for large-scale implementation of machine learning-based diagnostic algorithms.

ECG-based data-driven solutions for diagnosis and prognosis of cardiovascular diseases: A systematic review

Cardiovascular diseases (CVD) are a leading cause of death globally, and result in significant morbidity and reduced quality of life. The electrocardiogram (ECG) plays a crucial role in CVD diagnosis, prognosis, and prevention; however, different challenges still remain, such as an increasing unmet demand for skilled cardiologists capable of accurately interpreting ECG. This leads to higher workload and potential diagnostic inaccuracies. Data-driven approaches, such as machine learning (ML) and deep learning (DL) have emerged to improve existing computer-assisted solutions and enhance physicians' ECG interpretation of the complex mechanisms underlying CVD. However, many ML and DL models used to detect ECG-based CVD suffer from a lack of explainability, bias, as well as ethical, legal, and societal implications (ELSI). Despite the critical importance of these Trustworthy Artificial Intelligence (AI) aspects, there is a lack of comprehensive literature reviews that examine the current trends in ECG-based solutions for CVD diagnosis or prognosis that use ML and DL models and address the Trustworthy AI requirements. This review aims to bridge this knowledge gap by providing a systematic review to undertake a holistic analysis across multiple dimensions of these data-driven models such as type of CVD addressed, dataset characteristics, data input modalities, ML and DL algorithms (with a focus on DL), and aspects of Trustworthy AI like explainability, bias and ethical considerations. Additionally, within the analyzed dimensions, various challenges are identified. To these, we provide concrete recommendations, equipping other researchers with valuable insights to understand the current state of the field comprehensively.

ECGMiner: A flexible software for accurately digitizing ECG

BACKGROUND AND OBJECTIVE

The electrocardiogram (ECG) is the most important non-invasive method for elucidating information about heart and cardiovascular disease diagnosis. Typically, the ECG system manufacturing companies provide ECG images, but store the numerical data in a proprietary format that is not interpretable and is not therefore useful for automatic diagnosis. There have been many efforts to digitize paper-based ECGs. The main limitations of previous works in ECG digitization are that they require manual selection of the regions of interest, only partly provide signal digitization, and offer limited accuracy.

METHODS

We have developed the ECGMiner, an open-source software to digitize ECG images. It is precise, fast, and simple to use. This software digitizes ECGs in four steps: 1) recognizing the image composition; 2) removing the gridline; 3) extracting the signals; 4) post-processing and storing the data.

RESULTS

We have evaluated the ECGMiner digitization capabilities using the Pearson Correlation Coefficient (PCC) and the Root Mean Square Error (RMSE) measures, and we consider ECG from two large, public, and widely used databases, LUDB and PTB-XL. The actual and digitized values of signals in both databases have been compared. The software's ability to correctly identify the location of characteristic waves has also been validated. Specifically, the PCC values are between 0.971 and 0.995, and the RMSE values are between 0.011 and 0.031 mV.

CONCLUSIONS

The ECGMiner software presented in this paper is open access, easy to install, easy to use, and capable of precisely recovering the paper-based/digital ECG signal data, regardless of the input format and signal complexity. ECGMiner outperforms existing digitization algorithms in terms of PCC and RMSE values.

Impact of ECG data format on the performance of machine learning models for the prediction of myocardial infarction

Background We aim to determine which electrocardiogram (ECG) data format is optimal for ML modelling, in the context of myocardial infarction prediction. We will also address the auxiliary objective of evaluating the viability of using digitised ECG signals for ML modelling. Methods Two ECG arrangements displaying 10s and 2.5 s of data for each lead were used. For each arrangement, conservative and speculative data cohorts were generated from the PTB-XL dataset. All ECGs were represented in three different data formats: Signal ECGs, Image ECGs, and Extracted Signal ECGs, with 8358 and 11,621 ECGs in the conservative and speculative cohorts, respectively. ML models were trained using the three data formats in both data cohorts. Results For ECGs that contained 10s of data, Signal and Extracted Signal ECGs were optimal and statistically similar, with AUCs [95% CI] of 0.971 [0.961, 0.981] and 0.974 [0.965, 0.984], respectively, for the conservative cohort; and 0.931 [0.918, 0.945] and 0.919 [0.903, 0.934], respectively, for the speculative cohort. For ECGs that contained 2.5 s of data, the Image ECG format was optimal, with AUCs of 0.960 [0.948, 0.973] and 0.903 [0.886, 0.920], for the conservative and speculative cohorts, respectively. Conclusion When available, the Signal ECG data should be preferred for ML modelling. If not, the optimal format depends on the data arrangement within the ECG: If the Image ECG contains 10s of data for each lead, the Extracted Signal ECG is optimal, however, if it only uses 2.5 s, then using the Image ECG data is optimal for ML performance.

Exploring alternative approaches to precision medicine through genomics and artificial intelligence - a systematic review

The core idea behind precision medicine is to pinpoint the subpopulations that differ from one another in terms of disease risk, drug responsiveness, and treatment outcomes due to differences in biology and other traits. Biomarkers are found through genomic sequencing. Multi-dimensional clinical and biological data are created using these biomarkers. Better analytic methods are needed for these multidimensional data, which can be accomplished by using artificial intelligence (AI). An updated review of 80 latest original publications is presented on four main fronts-preventive medicine, medication development, treatment outcomes, and diagnostic medicine-All these studies effectively illustrated the significance of AI in precision medicine. Artificial intelligence (AI) has revolutionized precision medicine by swiftly analyzing vast amounts of data to provide tailored treatments and predictive diagnostics. Through machine learning algorithms and high-resolution imaging, AI assists in precise diagnoses and early disease detection. AI's ability to decode complex biological factors aids in identifying novel therapeutic targets, allowing personalized interventions and optimizing treatment outcomes. Furthermore, AI accelerates drug discovery by navigating chemical structures and predicting drug-target interactions, expediting the development of life-saving medications. With its unrivaled capacity to comprehend and interpret data, AI stands as an invaluable tool in the pursuit of enhanced patient care and improved health outcomes. It's evident that AI can open a new horizon for precision medicine by translating complex data into actionable information. To get better results in this regard and to fully exploit the great potential of AI, further research is required on this pressing subject.

Extraction and Digitization of ECG Signals from Standard Clinical Portable Document Format Files for the Principal Component Analysis of T-wave Morphology

INTRODUCTION

T-wave analysis from standard electrocardiogram (ECG) remains one of the most available clinical and research methods for evaluating myocardial repolarization. T-wave morphology was recently evaluated to aid with diagnosis and characterization of diastolic dysfunction. Unfortunately, PDF stored ECG datasets limit additional numerical post-processing of ECG waveforms. In this study, we apply a simple custom process pipeline to extract and re-digitize T-wave signals and subject them to principal component analysis (PCA) to define primary T-wave shape variations.

METHODS

We propose simple pre-processing and digitization algorithms programmable as a MATLAB tool using standard thresholding functions without the need for advanced signal analysis. To validate digitized datasets, we compared clinically standard measurements in 20 different ECGs with the original ECG machine interpreted values as a gold standard. Afterwards, we analyzed 212 individual ECGs for T-wave shape analysis using PCA.

RESULTS

The re-digitized signal was shown to preserve the original information as evidenced by excellent agreement between original - machine interpreted and re-digitized clinical variables including heart rate: bias ~ 1 bpm (95% CI: -1.0 to 3.5), QT interval: bias ~ 0.000 ms (95% CI: -0.012 to 0.012), PR interval: bias = -0.015 ms (95% CI: -0.015 to 0.003), and QRS duration: bias = -0.001 ms (95% CI: -0.007 to 0.006). PCA revealed that the first principal component universally modulates the T-wave height or amount of repolarization voltage regardless of the investigated ECG lead. The second and third principal components described variation in the T-wave peak onset and the T-wave peak morphology, respectively.

CONCLUSION

This study presents a straightforward method for re-digitizing ECGs stored in the PDF format utilized in many academic electronic medical record systems. This process can yield re-digitized lead specific signals which can be retrospectively analyzed using advanced custom post-processing numerical analysis independent of commercially available platforms.

Automatic digitization of paper electrocardiograms - A systematic review

The digitization of electrocardiogram paper records is an essential step to preserve and analyze cardiac data. This digitization process is not flawless as it involves several challenges, such as skew correction, binarization, and signal extraction. Various approaches have been proposed to address these challenges and recent studies have introduced innovative solutions, such as deep learning models and automation processes. Although existing approaches have shown promising results, there is a lack of common databases and metrics where authors could evaluate and compare their methods. Furthermore, the limited accessibility of code or software hinders the comparison process. Overall, while digitization of paper ECG recordings is important in advancing cardiology research, additional efforts are needed to standardize the evaluation process while improving code accessibility. This article provides a systematic review of this process.

Artificial intelligence based approach for categorization of COVID-19 ECG images in presence of other cardiovascular disorders

Coronavirus disease (COVID-19) is a class of SARS-CoV-2 virus which is initially identified in the later half of the year 2019 and then evolved as a pandemic. If it is not identified in the early stage then the infection and mortality rates increase with time. A timely and reliable approach for COVID-19 identification has become important in order to prevent the disease from spreading rapidly. In recent times, many methods have been suggested for the detection of COVID-19 disease have various flaws, to increase diagnosis performance, fresh investigations are required. In this article, automatically diagnosing COVID-19 using ECG images and deep learning approaches like as Visual Geometry Group (VGG) and AlexNet architectures have been proposed. The proposed method is able to classify between COVID-19, myocardial infarction, normal sinus rhythm, and other abnormal heart beats using Lead-II ECG image only. The efficacy of the technique proposed is validated by using a publicly available ECG image database. We have achieved an accuracy of 77.42% using Alexnet model and 75% accuracy with the help of VGG19 model.

A fully-automated paper ECG digitisation algorithm using deep learning

There is increasing focus on applying deep learning methods to electrocardiograms (ECGs), with recent studies showing that neural networks (NNs) can predict future heart failure or atrial fibrillation from the ECG alone. However, large numbers of ECGs are needed to train NNs, and many ECGs are currently only in paper format, which are not suitable for NN training. We developed a fully-automated online ECG digitisation tool to convert scanned paper ECGs into digital signals. Using automated horizontal and vertical anchor point detection, the algorithm automatically segments the ECG image into separate images for the 12 leads and a dynamical morphological algorithm is then applied to extract the signal of interest. We then validated the performance of the algorithm on 515 digital ECGs, of which 45 were printed, scanned and redigitised. The automated digitisation tool achieved 99.0% correlation between the digitised signals and the ground truth ECG (n = 515 standard 3-by-4 ECGs) after excluding ECGs with overlap of lead signals. Without exclusion, the performance of average correlation was from 90 to 97% across the leads on all 3-by-4 ECGs. There was a 97% correlation for 12-by-1 and 3-by-1 ECG formats after excluding ECGs with overlap of lead signals. Without exclusion, the average correlation of some leads in 12-by-1 ECGs was 60-70% and the average correlation of 3-by-1 ECGs achieved 80-90%. ECGs that were printed, scanned, and redigitised, our tool achieved 96% correlation with the original signals. We have developed and validated a fully-automated, user-friendly, online ECG digitisation tool. Unlike other available tools, this does not require any manual segmentation of ECG signals. Our tool can facilitate the rapid and automated digitisation of large repositories of paper ECGs to allow them to be used for deep learning projects.

A multi-label classification system for anomaly classification in electrocardiogram

Automatic classification of ECG signals has become a research hotspot, and most of the research work in this field is currently aimed at single-label classification. However, a segment of ECG signal may contain more than two cardiac diseases, and single-label classification cannot accurately judge all possibilities. Besides, single-label classification performs classification in units of segmented beats, which destroys the contextual relevance of signal data. Therefore, studying the multi-label classification of ECG signals becomes more critical. This study proposes a method based on the multi-label question transformation method-binary correlation and classifies ECG signals by constructing a deep sequence model. Binary correlation simplifies the learning difficulty of deep learning models and converts multi-label problems into multiple binary classification problems. The experimental results are as follows: F1 score is 0.767, Hamming Loss is 0.073, Coverage is 3.4, and Ranking Loss is 0.262. It performs better than existing work.

Analysis of digitalized ECG signals based on artificial intelligence and spectral analysis methods specialized in ARVC

Arrhythmogenic right ventricular cardiomyopathy (ARVC) is an inherited heart muscle disease that appears between the second and forth decade of a patient's life, being responsible for 20% of sudden cardiac deaths before the age of 35. The effective and punctual diagnosis of this disease based on electrocardiograms (ECGs) could have a vital role in reducing premature cardiovascular mortality. In our analysis, we first outline the digitalization process of paper-based ECG signals enhanced by a spatial filter aiming to eliminate dark regions in the dataset's images that do not correspond to ECG waveform, producing undesirable noise. Next, we propose the utilization of a low-complexity convolutional neural network for the detection of an arrhythmogenic heart disease, that has not been studied through the usage of deep learning methodology to date, achieving high classification accuracy, namely 99.98% training and 98.6% testing accuracy, on a disease the major identification criterion of which are infinitesimal millivolt variations in the ECG's morphology, in contrast with other arrhythmogenic abnormalities. Finally, by performing spectral analysis we investigate significant differentiations in the field of frequencies between normal ECGs and ECGs corresponding to patients suffering from ARVC. In 16 out of the 18 frequencies where we encounter statistically significant differentiations, the normal ECGs are characterized by greater normalized amplitudes compared to the abnormal ones. The overall research carried out in this article highlights the importance of integrating mathematical methods into the examination and effective diagnosis of various diseases, aiming to a substantial contribution to their successful treatment.

Digital signal extraction approach for cardiotocography image

BACKGROUND AND OBJECTIVE

Cardiotocography, commonly called CTG, has become an indispensable auxiliary examination in obstetrics. Generally, CTG is provided in the form of a report, so the fetal heart rate and uterine contraction signals have to be extracted from the CTG images. However, most studies focused on reading data for a single curve, and the influence of complex backgrounds was usually not considered.

METHODS

An efficient signal extraction method was proposed for the binary CTG images with complex backgrounds. Firstly, the images' background grids and symbol noise were removed by templates. Then a morphological method was used to fill breakpoints of curves. Moreover, the projection map was utilized to localize the area and the starting and ending positions of curves. Subsequently, data of the curves were extracted by column scanning. Finally, the amplitude of the extracted signal was calibrated.

RESULTS

This study had tested 552 CTG images simulated using the CTU-UHB database. The correlation coefficient between the extracted and original signals was 0.9991 ± 0.0030 for fetal heart rate and 0.9904 ± 0.0208 for uterine contraction, and the mean absolute error of fetal heart rate and uterine contraction were 2.4658 ± 1.8446 and 1.8025 ± 0.6155, and the root mean square error of fetal heart rate and uterine contraction were 4.2930 ± 2.9771 and 2.5214 ± 0.9640, respectively. After being validated using 293 clinical authentic CTG images, the extracted signals were remarkably similar to the original counterparts, and no significant differences were observed.

CONCLUSIONS

The proposed method could effectively extract the fetal heart rate and uterine contraction signals from the binary CTG images with complex backgrounds.

Digitizing ECG image: A new method and open-source software code

BACKGROUND AND OBJECTIVE

We aimed to develop and validate an open-source code ECG-digitizing tool and assess agreements of ECG measurements across three types of median beats, comprised of digitally recorded simultaneous and asynchronous ECG leads and digitized asynchronous ECG leads.

METHODS

We used the data of clinical studies participants (n = 230; mean age 30±15 y; 25% female; 52% had the cardiovascular disease) with available both digitally recorded and printed on paper and then scanned ECGs, split into development (n = 150) and validation (n = 80) datasets. The agreement between ECG and VCG measurements on the digitally recorded time-coherent median beat, representative asynchronous digitized, and digitally recorded beats was assessed by Bland-Altman analysis.

RESULTS

The sample-per-sample comparison of digitally recorded and digitized signals showed a very high correlation (0.977), a small mean difference (9.3 µV), and root mean squared error (25.9 µV). Agreement between digitally recorded and digitized representative beat was high [area spatial ventricular gradient (SVG) elevation bias 2.5(95% limits of agreement [LOA] -7.9-13.0)°; precision 96.8%; inter-class correlation [ICC] 0.988; Lin's concordance coefficient ρ_c 0.97(95% confidence interval [CI] 0.95-0.98)]. Agreement between digitally recorded asynchronous and time-coherent median beats was moderate for area-based VCG metrics (spatial QRS-T angle bias 1.4(95%LOA -33.2-30.3)°; precision 94.8%; ICC 0.95; Lin's concordance coefficient ρ_c 0.90(95%CI 0.82-0.95)].

CONCLUSIONS

We developed and validated an open-source software tool for paper-ECG digitization. Asynchronous ECG leads are the primary source of disagreement in measurements on digitally recorded and digitized ECGs.

Combining Optical Character Recognition With Paper ECG Digitization

Objective: We propose a MATLAB-based tool to convert electrocardiography (ECG) waveforms from paper-based ECG records into digitized ECG signals that is vendor-agnostic. The tool is packaged as an open source standalone graphical user interface (GUI) based application. Methods and procedures: To reach this objective we: (1) preprocess the ECG records, which includes skew correction, background grid removal and linear filtering; (2) segment ECG signals using Connected Components Analysis (CCA); (3) implement Optical Character Recognition (OCR) for removal of overlapping ECG lead characters and for interfacing of patients’ demographic information with their research records or their electronic medical record (EMR). The ECG digitization results are validated through a reader study where clinically salient features, such as intervals of QRST complex, between the paper ECG records and the digitized ECG records are compared. Results: Comparison of clinically important features between the paper-based ECG records and the digitized ECG signals, reveals intra- and inter-observer correlations of 0.86–0.99 and 0.79–0.94, respectively. The kappa statistic was found to average at 0.86 and 0.72 for intra- and inter-observer correlations, respectively. Conclusion: The clinically salient features of the ECG waveforms such as the intervals of QRST complex, are preserved during the digitization procedure. Clinical and Healthcare Impact: This open-source digitization tool can be used as a research resource to digitize paper ECG records thereby enabling development of new prediction algorithms to risk stratify individuals with cardiovascular disease, and/or allow for development of ECG-based cardiovascular diagnoses relying upon automated digital algorithms.

Classification of COVID-19 electrocardiograms by using hexaxial feature mapping and deep learning

BACKGROUND

Coronavirus disease 2019 (COVID-19) has become a pandemic since its first appearance in late 2019. Deaths caused by COVID-19 are still increasing day by day and early diagnosis has become crucial. Since current diagnostic methods have many disadvantages, new investigations are needed to improve the performance of diagnosis.

METHODS

A novel method is proposed to automatically diagnose COVID-19 by using Electrocardiogram (ECG) data with deep learning for the first time. Moreover, a new and effective method called hexaxial feature mapping is proposed to represent 12-lead ECG to 2D colorful images. Gray-Level Co-Occurrence Matrix (GLCM) method is used to extract features and generate hexaxial mapping images. These generated images are then fed into a new Convolutional Neural Network (CNN) architecture to diagnose COVID-19.

RESULTS

Two different classification scenarios are conducted on a publicly available paper-based ECG image dataset to reveal the diagnostic capability and performance of the proposed approach. In the first scenario, ECG data labeled as COVID-19 and No-Findings (normal) are classified to evaluate COVID-19 classification ability. According to results, the proposed approach provides encouraging COVID-19 detection performance with an accuracy of 96.20% and F1-Score of 96.30%. In the second scenario, ECG data labeled as Negative (normal, abnormal, and myocardial infarction) and Positive (COVID-19) are classified to evaluate COVID-19 diagnostic ability. The experimental results demonstrated that the proposed approach provides satisfactory COVID-19 prediction performance with an accuracy of 93.00% and F1-Score of 93.20%. Furthermore, different experimental studies are conducted to evaluate the robustness of the proposed approach.

CONCLUSION

Automatic detection of cardiovascular changes caused by COVID-19 can be possible with a deep learning framework through ECG data. This not only proves the presence of cardiovascular changes caused by COVID-19 but also reveals that ECG can potentially be used in the diagnosis of COVID-19. We believe the proposed study may provide a crucial decision-making system for healthcare professionals.

SOURCE CODE

All source codes are made publicly available at: https://github.com/mkfzdmr/COVID-19-ECG-Classification.