1
|
Ouyang D, Theurer J, Stein NR, Hughes JW, Elias P, He B, Yuan N, Duffy G, Sandhu RK, Ebinger J, Botting P, Jujjavarapu M, Claggett B, Tooley JE, Poterucha T, Chen JH, Nurok M, Perez M, Perotte A, Zou JY, Cook NR, Chugh SS, Cheng S, Albert CM. Electrocardiographic deep learning for predicting post-procedural mortality: a model development and validation study. Lancet Digit Health 2024; 6:e70-e78. [PMID: 38065778 DOI: 10.1016/s2589-7500(23)00220-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2022] [Revised: 10/01/2023] [Accepted: 10/18/2023] [Indexed: 12/22/2023]
Abstract
BACKGROUND Preoperative risk assessments used in clinical practice are insufficient in their ability to identify risk for postoperative mortality. Deep-learning analysis of electrocardiography can identify hidden risk markers that can help to prognosticate postoperative mortality. We aimed to develop a prognostic model that accurately predicts postoperative mortality in patients undergoing medical procedures and who had received preoperative electrocardiographic diagnostic testing. METHODS In a derivation cohort of preoperative patients with available electrocardiograms (ECGs) from Cedars-Sinai Medical Center (Los Angeles, CA, USA) between Jan 1, 2015 and Dec 31, 2019, a deep-learning algorithm was developed to leverage waveform signals to discriminate postoperative mortality. We randomly split patients (8:1:1) into subsets for training, internal validation, and final algorithm test analyses. Model performance was assessed using area under the receiver operating characteristic curve (AUC) values in the hold-out test dataset and in two external hospital cohorts and compared with the established Revised Cardiac Risk Index (RCRI) score. The primary outcome was post-procedural mortality across three health-care systems. FINDINGS 45 969 patients had a complete ECG waveform image available for at least one 12-lead ECG performed within the 30 days before the procedure date (59 975 inpatient procedures and 112 794 ECGs): 36 839 patients in the training dataset, 4549 in the internal validation dataset, and 4581 in the internal test dataset. In the held-out internal test cohort, the algorithm discriminates mortality with an AUC value of 0·83 (95% CI 0·79-0·87), surpassing the discrimination of the RCRI score with an AUC of 0·67 (0·61-0·72). The algorithm similarly discriminated risk for mortality in two independent US health-care systems, with AUCs of 0·79 (0·75-0·83) and 0·75 (0·74-0·76), respectively. Patients determined to be high risk by the deep-learning model had an unadjusted odds ratio (OR) of 8·83 (5·57-13·20) for postoperative mortality compared with an unadjusted OR of 2·08 (0·77-3·50) for postoperative mortality for RCRI scores of more than 2. The deep-learning algorithm performed similarly for patients undergoing cardiac surgery (AUC 0·85 [0·77-0·92]), non-cardiac surgery (AUC 0·83 [0·79-0·88]), and catheterisation or endoscopy suite procedures (AUC 0·76 [0·72-0·81]). INTERPRETATION A deep-learning algorithm interpreting preoperative ECGs can improve discrimination of postoperative mortality. The deep-learning algorithm worked equally well for risk stratification of cardiac surgeries, non-cardiac surgeries, and catheterisation laboratory procedures, and was validated in three independent health-care systems. This algorithm can provide additional information to clinicians making the decision to perform medical procedures and stratify the risk of future complications. FUNDING National Heart, Lung, and Blood Institute.
Collapse
Affiliation(s)
- David Ouyang
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA; Division of Artificial Intelligence in Medicine, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
| | - John Theurer
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Nathan R Stein
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - J Weston Hughes
- Department of Computer Science, Stanford University, Palo Alto, CA, USA
| | - Pierre Elias
- Milstein Division of Cardiology, Department of Medicine, Columbia University Irving Medical Center, New York, NY, USA; Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY, USA
| | - Bryan He
- Department of Computer Science, Stanford University, Palo Alto, CA, USA
| | - Neal Yuan
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Grant Duffy
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Roopinder K Sandhu
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Joseph Ebinger
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Patrick Botting
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Melvin Jujjavarapu
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Brian Claggett
- Division of Cardiovascular Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, MA, USA
| | - James E Tooley
- Division of Cardiology, Stanford University, Palo Alto, CA, USA
| | - Tim Poterucha
- Milstein Division of Cardiology, Department of Medicine, Columbia University Irving Medical Center, New York, NY, USA
| | - Jonathan H Chen
- Division of Bioinformatics Research, Stanford University, Palo Alto, CA, USA
| | - Michael Nurok
- Division of Anesthesia, Department of Surgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Marco Perez
- Division of Cardiology, Stanford University, Palo Alto, CA, USA
| | - Adler Perotte
- Milstein Division of Cardiology, Department of Medicine, Columbia University Irving Medical Center, New York, NY, USA
| | - James Y Zou
- Department of Computer Science, Stanford University, Palo Alto, CA, USA; Department of Medicine, and Department of Biomedical Data Science, Stanford University, Palo Alto, CA, USA
| | - Nancy R Cook
- Division of Preventive Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, MA, USA
| | - Sumeet S Chugh
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA; Division of Artificial Intelligence in Medicine, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Susan Cheng
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Christine M Albert
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| |
Collapse
|
2
|
Hughes JW, Tooley J, Torres Soto J, Ostropolets A, Poterucha T, Christensen MK, Yuan N, Ehlert B, Kaur D, Kang G, Rogers A, Narayan S, Elias P, Ouyang D, Ashley E, Zou J, Perez MV. A deep learning-based electrocardiogram risk score for long term cardiovascular death and disease. NPJ Digit Med 2023; 6:169. [PMID: 37700032 PMCID: PMC10497604 DOI: 10.1038/s41746-023-00916-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Accepted: 08/30/2023] [Indexed: 09/14/2023] Open
Abstract
The electrocardiogram (ECG) is the most frequently performed cardiovascular diagnostic test, but it is unclear how much information resting ECGs contain about long term cardiovascular risk. Here we report that a deep convolutional neural network can accurately predict the long-term risk of cardiovascular mortality and disease based on a resting ECG alone. Using a large dataset of resting 12-lead ECGs collected at Stanford University Medical Center, we developed SEER, the Stanford Estimator of Electrocardiogram Risk. SEER predicts 5-year cardiovascular mortality with an area under the receiver operator characteristic curve (AUC) of 0.83 in a held-out test set at Stanford, and with AUCs of 0.78 and 0.83 respectively when independently evaluated at Cedars-Sinai Medical Center and Columbia University Irving Medical Center. SEER predicts 5-year atherosclerotic disease (ASCVD) with an AUC of 0.67, similar to the Pooled Cohort Equations for ASCVD Risk, while being only modestly correlated. When used in conjunction with the Pooled Cohort Equations, SEER accurately reclassified 16% of patients from low to moderate risk, uncovering a group with an actual average 9.9% 10-year ASCVD risk who would not have otherwise been indicated for statin therapy. SEER can also predict several other cardiovascular conditions such as heart failure and atrial fibrillation. Using only lead I of the ECG it predicts 5-year cardiovascular mortality with an AUC of 0.80. SEER, used alongside the Pooled Cohort Equations and other risk tools, can substantially improve cardiovascular risk stratification and aid in medical decision making.
Collapse
Affiliation(s)
- J Weston Hughes
- Department of Computer Science, Stanford University, Palo Alto, CA, USA.
| | - James Tooley
- Department of Medicine, Stanford University, Palo Alto, CA, USA
| | - Jessica Torres Soto
- Department of Biomedical Informatics, Stanford University, Palo Alto, CA, USA
| | - Anna Ostropolets
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY, USA
| | - Tim Poterucha
- Milstein Division of Cardiology, Department of Medicine, Columbia University Irving Medical Center, New York, NY, USA
| | - Matthew Kai Christensen
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Neal Yuan
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Ben Ehlert
- Department of Biomedical Informatics, Stanford University, Palo Alto, CA, USA
| | | | - Guson Kang
- Department of Medicine, Stanford University, Palo Alto, CA, USA
| | - Albert Rogers
- Department of Medicine, Stanford University, Palo Alto, CA, USA
| | - Sanjiv Narayan
- Department of Medicine, Stanford University, Palo Alto, CA, USA
| | - Pierre Elias
- Department of Biomedical Informatics, Columbia University Irving Medical Center, New York, NY, USA
- Milstein Division of Cardiology, Department of Medicine, Columbia University Irving Medical Center, New York, NY, USA
| | - David Ouyang
- Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Euan Ashley
- Department of Medicine, Stanford University, Palo Alto, CA, USA
| | - James Zou
- Department of Computer Science, Stanford University, Palo Alto, CA, USA
- Department of Biomedical Data Science, Stanford University, Palo Alto, CA, USA
| | - Marco V Perez
- Department of Medicine, Stanford University, Palo Alto, CA, USA
| |
Collapse
|