1
|
Roumeliotis S, Schurgers J, Tsalikakis DG, D'Arrigo G, Gori M, Pitino A, Leonardis D, Tripepi G, Liakopoulos V. ROC curve analysis: a useful statistic multi-tool in the research of nephrology. Int Urol Nephrol 2024:10.1007/s11255-024-04022-8. [PMID: 38530584 DOI: 10.1007/s11255-024-04022-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/15/2024] [Accepted: 03/04/2024] [Indexed: 03/28/2024]
Abstract
In the past decade, scientific research in the area of Nephrology has focused on evaluating the clinical utility and performance of various biomarkers for diagnosis, risk stratification and prognosis. Before implementing a biomarker in everyday clinical practice for screening a specific disease context, specific statistic measures are necessary to evaluate the diagnostic accuracy and performance of this biomarker. Receiver Operating Characteristic (ROC) Curve analysis is an important statistical method used to estimate the discriminatory performance of a novel diagnostic test, identify the optimal cut-off value for a test that maximizes sensitivity and specificity, and evaluate the predictive value of a certain biomarker or risk, prediction score. Herein, through practical examples, we aim to present a simple methodological approach to explain in detail the principles and applications of ROC curve analysis in the field of nephrology pertaining diagnosis and prognosis.
Collapse
Affiliation(s)
- Stefanos Roumeliotis
- 2nd Department of Nephrology, Medical School, AHEPA Hospital, Aristotle University of Thessaloniki, 54636, Thessaloniki, Greece
| | - Juul Schurgers
- 2nd Department of Nephrology, Medical School, AHEPA Hospital, Aristotle University of Thessaloniki, 54636, Thessaloniki, Greece
| | - Dimitrios G Tsalikakis
- Department of Electrical and Computer Engineering, University of Western Macedonia, Kozani, Greece
| | - Graziella D'Arrigo
- Institute of Clinical Physiology (IFC), National Research Council (CNR), 89124, Reggio Calabria, Italy
| | - Mercedes Gori
- Institute of Clinical Physiology (IFC), National Research Council (CNR), 00100, Rome, Italy
| | - Annalisa Pitino
- Institute of Clinical Physiology (IFC), National Research Council (CNR), 89124, Reggio Calabria, Italy
| | - Daniela Leonardis
- Institute of Clinical Physiology (IFC), National Research Council (CNR), 89124, Reggio Calabria, Italy
| | - Giovanni Tripepi
- Institute of Clinical Physiology (IFC), National Research Council (CNR), 89124, Reggio Calabria, Italy
| | - Vassilios Liakopoulos
- 2nd Department of Nephrology, Medical School, AHEPA Hospital, Aristotle University of Thessaloniki, 54636, Thessaloniki, Greece.
| |
Collapse
|
2
|
Suri JS, Bhagawati M, Paul S, Protogerou AD, Sfikakis PP, Kitas GD, Khanna NN, Ruzsa Z, Sharma AM, Saxena S, Faa G, Laird JR, Johri AM, Kalra MK, Paraskevas KI, Saba L. A Powerful Paradigm for Cardiovascular Risk Stratification Using Multiclass, Multi-Label, and Ensemble-Based Machine Learning Paradigms: A Narrative Review. Diagnostics (Basel) 2022; 12:diagnostics12030722. [PMID: 35328275 PMCID: PMC8947682 DOI: 10.3390/diagnostics12030722] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2022] [Revised: 03/10/2022] [Accepted: 03/13/2022] [Indexed: 12/16/2022] Open
Abstract
Background and Motivation: Cardiovascular disease (CVD) causes the highest mortality globally. With escalating healthcare costs, early non-invasive CVD risk assessment is vital. Conventional methods have shown poor performance compared to more recent and fast-evolving Artificial Intelligence (AI) methods. The proposed study reviews the three most recent paradigms for CVD risk assessment, namely multiclass, multi-label, and ensemble-based methods in (i) office-based and (ii) stress-test laboratories. Methods: A total of 265 CVD-based studies were selected using the preferred reporting items for systematic reviews and meta-analyses (PRISMA) model. Due to its popularity and recent development, the study analyzed the above three paradigms using machine learning (ML) frameworks. We review comprehensively these three methods using attributes, such as architecture, applications, pro-and-cons, scientific validation, clinical evaluation, and AI risk-of-bias (RoB) in the CVD framework. These ML techniques were then extended under mobile and cloud-based infrastructure. Findings: Most popular biomarkers used were office-based, laboratory-based, image-based phenotypes, and medication usage. Surrogate carotid scanning for coronary artery risk prediction had shown promising results. Ground truth (GT) selection for AI-based training along with scientific and clinical validation is very important for CVD stratification to avoid RoB. It was observed that the most popular classification paradigm is multiclass followed by the ensemble, and multi-label. The use of deep learning techniques in CVD risk stratification is in a very early stage of development. Mobile and cloud-based AI technologies are more likely to be the future. Conclusions: AI-based methods for CVD risk assessment are most promising and successful. Choice of GT is most vital in AI-based models to prevent the RoB. The amalgamation of image-based strategies with conventional risk factors provides the highest stability when using the three CVD paradigms in non-cloud and cloud-based frameworks.
Collapse
Affiliation(s)
- Jasjit S. Suri
- Stroke Diagnostic and Monitoring Division, AtheroPoint™, Roseville, CA 95661, USA
- Correspondence: ; Tel.: +1-(916)-749-5628
| | - Mrinalini Bhagawati
- Department of Biomedical Engineering, North-Eastern Hill University, Shillong 793022, India; (M.B.); (S.P.)
| | - Sudip Paul
- Department of Biomedical Engineering, North-Eastern Hill University, Shillong 793022, India; (M.B.); (S.P.)
| | - Athanasios D. Protogerou
- Research Unit Clinic, Laboratory of Pathophysiology, Department of Cardiovascular Prevention, National and Kapodistrian University of Athens, 11527 Athens, Greece;
| | - Petros P. Sfikakis
- Rheumatology Unit, National Kapodistrian University of Athens, 11527 Athens, Greece;
| | - George D. Kitas
- Arthritis Research UK Centre for Epidemiology, Manchester University, Manchester 46962, UK;
| | - Narendra N. Khanna
- Department of Cardiology, Indraprastha APOLLO Hospitals, New Delhi 110020, India;
| | - Zoltan Ruzsa
- Department of Internal Medicines, Invasive Cardiology Division, University of Szeged, 6720 Szeged, Hungary;
| | - Aditya M. Sharma
- Division of Cardiovascular Medicine, University of Virginia, Charlottesville, VA 22903, USA;
| | - Sanjay Saxena
- Department of CSE, International Institute of Information Technology, Bhubaneswar 751003, India;
| | - Gavino Faa
- Department of Pathology, A.O.U., di Cagliari-Polo di Monserrato s.s., 09045 Cagliari, Italy;
| | - John R. Laird
- Cardiology Department, St. Helena Hospital, St. Helena, CA 94574, USA;
| | - Amer M. Johri
- Department of Medicine, Division of Cardiology, Queen’s University, Kingston, ON K7L 3N6, Canada;
| | - Manudeep K. Kalra
- Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, USA;
| | - Kosmas I. Paraskevas
- Department of Vascular Surgery, Central Clinic of Athens, N. Iraklio, 14122 Athens, Greece;
| | - Luca Saba
- Department of Radiology, A.O.U., di Cagliari-Polo di Monserrato s.s., 09045 Cagliari, Italy;
| |
Collapse
|
3
|
Roumeliotis S, Georgianos PI, Roumeliotis A, Eleftheriadis T, Stamou A, Manolopoulos VG, Panagoutsos S, Liakopoulos V. Oxidized LDL Modifies the Association between Proteinuria and Deterioration of Kidney Function in Proteinuric Diabetic Kidney Disease. Life (Basel) 2021; 11:life11060504. [PMID: 34072583 PMCID: PMC8226768 DOI: 10.3390/life11060504] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2021] [Revised: 05/25/2021] [Accepted: 05/28/2021] [Indexed: 12/12/2022] Open
Abstract
Proteinuria is characterized by low accuracy for predicting onset and development of diabetic kidney disease (DKD) because it is not directly associated with molecular changes that promote DKD, but is a result of kidney damage. Oxidized low-density lipoprotein (ox-LDL) reflects oxidative stress and endothelial dysfunction, both underlying the development of proteinuria and loss of kidney function in DKD. We aimed to investigate whether ox-LDL modifies the association between proteinuria and progression of DKD in a cohort of 91 patients with proteinuric DKD and diabetic retinopathy, followed for 10 years. The primary endpoint was a combined kidney outcome of eGFR decline ≥30% or progression to end-stage kidney disease. After the end of the study, we considered the percentage change of eGFR over time as our secondary outcome. Proteinuria was associated with both outcomes, and ox-LDL amplified the magnitude of this link (p < 0.0001 for primary and p < 0.0001 for secondary outcome, respectively). After adjustment for duration of diabetes, history of cardiovascular disease and serum albumin, ox-LDL remained a significant effect modifier of the association between proteinuria and eGFR decline over time (p = 0.04). Our study shows that in proteinuric DKD, circulating ox-LDL levels amplified the magnitude of the association between proteinuria and progression of DKD.
Collapse
Affiliation(s)
- Stefanos Roumeliotis
- Division of Nephrology and Hypertension, 1st Department of Internal Medicine, AHEPA Hospital, School of Medicine, Aristotle University of Thessaloniki, 54636 Thessaloniki, Greece; (P.I.G.); (A.R.); (V.L.)
- Correspondence: ; Tel.: +30-231-099-4694
| | - Panagiotis I. Georgianos
- Division of Nephrology and Hypertension, 1st Department of Internal Medicine, AHEPA Hospital, School of Medicine, Aristotle University of Thessaloniki, 54636 Thessaloniki, Greece; (P.I.G.); (A.R.); (V.L.)
| | - Athanasios Roumeliotis
- Division of Nephrology and Hypertension, 1st Department of Internal Medicine, AHEPA Hospital, School of Medicine, Aristotle University of Thessaloniki, 54636 Thessaloniki, Greece; (P.I.G.); (A.R.); (V.L.)
| | - Theodoros Eleftheriadis
- Department of Nephrology, School of Medicine, University of Thessaly, 38221 Larissa, Greece;
| | - Aikaterini Stamou
- Department of Microbiology, AHEPA Hospital, School of Medicine, Aristotle University of Thessaloniki, 54636 Thessaloniki, Greece;
| | - Vangelis G. Manolopoulos
- Laboratory of Pharmacology, Medical School, Democritus University of Thrace, 68100 Alexandroupolis, Greece;
| | - Stylianos Panagoutsos
- Department of Nephrology, Medical School, Democritus University of Thrace, 68100 Alexandroupolis, Greece;
| | - Vassilios Liakopoulos
- Division of Nephrology and Hypertension, 1st Department of Internal Medicine, AHEPA Hospital, School of Medicine, Aristotle University of Thessaloniki, 54636 Thessaloniki, Greece; (P.I.G.); (A.R.); (V.L.)
| |
Collapse
|