1
|
Harish KB, Chervonski E, Speranza G, Maldonado TS, Garg K, Sadek M, Rockman CB, Jacobowitz GR, Berland TL. Prior authorization requirements in the office-based laboratory setting are administratively inefficient and threaten timeliness of care. J Vasc Surg 2024; 79:1195-1203. [PMID: 38135169 DOI: 10.1016/j.jvs.2023.10.066] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/25/2023] [Revised: 09/12/2023] [Accepted: 10/30/2023] [Indexed: 12/24/2023]
Abstract
OBJECTIVE The objective of this study was to investigate the administrative and clinical impacts of prior authorization (PA) processes in the office-based laboratory (OBL) setting. METHODS This single-institution, retrospective analysis studied all OBL PAs pursued between January 2018 and March 2022. Case, PA, and coding information was obtained from the practice's scheduling database. RESULTS Over the study period, 1854 OBL cases were scheduled; 8% (n = 146) required PA. Of these, 75% (n = 110) were for lower extremity arterial interventions, 19% (n = 27) were for deep venous interventions, and 6% (n = 9) were for other interventions. Of 146 PAs, 19% (n = 27) were initially denied but 74.1% (n = 7) of these were overturned on appeal. Deep venous procedures were initially denied, at 43.8% (n = 14), more often than were arterial procedures, at 11.8% (n = 13). Of 146 requested procedures, 4% (n = 6) were delayed due to pending PA determination by a mean 14.2 ± 18.3 working days. An additional 6% (n = 8) of procedures were performed in the interest of time before final determination. Of the seven terminally denied procedures, 57% (n = 4) were performed at cost to the practice based on clinical judgment. CONCLUSIONS Using PA appeals mechanisms, while administratively onerous, resulted in the overturning of most initial denials.
Collapse
Affiliation(s)
| | | | | | - Thomas S Maldonado
- Division of Vascular Surgery, Department of Surgery, New York University Langone Health, New York, NY
| | - Karan Garg
- Division of Vascular Surgery, Department of Surgery, New York University Langone Health, New York, NY
| | - Mikel Sadek
- Division of Vascular Surgery, Department of Surgery, New York University Langone Health, New York, NY
| | - Caron B Rockman
- Division of Vascular Surgery, Department of Surgery, New York University Langone Health, New York, NY
| | - Glenn R Jacobowitz
- Division of Vascular Surgery, Department of Surgery, New York University Langone Health, New York, NY
| | - Todd L Berland
- Division of Vascular Surgery, Department of Surgery, New York University Langone Health, New York, NY.
| |
Collapse
|
2
|
Zhang J, Ratner M, Harish KB, Speranza G, Hartwell CA, Rao A, Garg K, Maldonado T, Sadek M, Jacobowitz G, Rockman C. The natural history and long-term follow-up of splenic artery aneurysms. J Vasc Surg 2024; 79:801-807.e3. [PMID: 38081394 DOI: 10.1016/j.jvs.2023.11.065] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2023] [Revised: 11/09/2023] [Accepted: 11/10/2023] [Indexed: 01/09/2024]
Abstract
OBJECTIVE Although splenic artery aneurysms (SAAs) are the most common visceral aneurysm, there is a paucity of literature on the behavior of these entities. The objective of this study was to review the natural history of patients with SAA. METHODS This single-institution, retrospective analysis studied patients with SAA diagnosed by computed tomography imaging between 2015 and 2019, identified by our institutional radiology database. Imaging, demographic, and clinical data were obtained via the electronic medical record. The growth rate was calculated for patients with radiologic follow-up. RESULTS The cohort consisted of 853 patients with 890 SAAs, of whom 692 were female (81.2%). There were 37 women (5.3%) of childbearing age (15-50 years). The mean age at diagnosis was 70.9 years (range: 28-100 years). Frequently observed medical comorbidities included hypertension (70.2%), hypercholesterolemia (54.7%), and prior smoking (32.2%). Imaging indications included abdominal pain (37.3%), unrelated follow-up (28.0%), and follow-up of a previously noted visceral artery aneurysm (8.6%). The mean diameter at diagnosis was 13.3 ± 6.3 mm. Anatomic locations included the splenic hilum (36.0%), distal splenic artery (30.3%), midsplenic artery (23.9%), and proximal splenic artery (9.7%). Radiographically, the majority were saccular aneurysms (72.4%) with calcifications (88.5%). One patient (38-year-old woman) was initially diagnosed at the time of rupture of a 25 mm aneurysm; this patient underwent immediate endovascular intervention with no complications. The mean clinical follow-up among 812 patients was 4.1 ± 4.0 years, and the mean radiological follow-up among 514 patients was 3.8 ± 6.8 years. Of the latter, 122 patients (23.7%) experienced growth. Aneurysm growth rates for initial sizes <10 mm (n = 123), 10 to 19 mm (n = 353), 20 to 29 mm (n = 34), and >30 mm (n = 4) were 0.166 mm/y, 0.172 mm/y, 0.383 mm/y, and 0.246 mm/y, respectively. Of the entire cohort, 27 patients (3.2%) eventually underwent intervention (81.5% endovascular), with the most common indications including size/growth criteria (70.4%) and symptom development (18.5%). On multivariate analysis, only prior tobacco use was significantly associated with aneurysm growth (P = .028). CONCLUSIONS The majority of SAAs in this cohort remained stable in size, with few patients requiring intervention over a mean follow-up of 4 years. Current guidelines recommending treatment of asymptomatic aneurysms >30 mm appear appropriate given their slow progression. Despite societal recommendations for intervention for all SAAs among women of childbearing age, only a minority underwent vascular surgical consultation and intervention in this series, indicating that these recommendations are likely not well known in the general medical community.
Collapse
Affiliation(s)
- Jason Zhang
- Division of Vascular and Endovascular Surgery, Department of Surgery, New York University Langone Medical Center, New York, NY
| | - Molly Ratner
- Division of Vascular and Endovascular Surgery, Department of Surgery, New York University Langone Medical Center, New York, NY
| | - Keerthi B Harish
- Division of Vascular and Endovascular Surgery, Department of Surgery, New York University Langone Medical Center, New York, NY
| | - Giancarlo Speranza
- Division of Vascular and Endovascular Surgery, Department of Surgery, New York University Langone Medical Center, New York, NY
| | - C Austen Hartwell
- Department of Radiology, New York University Langone Medical Center, New York, NY
| | - Abhishek Rao
- Division of Vascular and Endovascular Surgery, Department of Surgery, New York University Langone Medical Center, New York, NY
| | - Karan Garg
- Division of Vascular and Endovascular Surgery, Department of Surgery, New York University Langone Medical Center, New York, NY
| | - Thomas Maldonado
- Division of Vascular and Endovascular Surgery, Department of Surgery, New York University Langone Medical Center, New York, NY
| | - Mikel Sadek
- Division of Vascular and Endovascular Surgery, Department of Surgery, New York University Langone Medical Center, New York, NY
| | - Glenn Jacobowitz
- Division of Vascular and Endovascular Surgery, Department of Surgery, New York University Langone Medical Center, New York, NY
| | - Caron Rockman
- Division of Vascular and Endovascular Surgery, Department of Surgery, New York University Langone Medical Center, New York, NY.
| |
Collapse
|
3
|
Chervonski E, Harish KB, Rockman CB, Sadek M, Teter KA, Jacobowitz GR, Berland TL, Lohr J, Moore C, Maldonado TS. Generative artificial intelligence chatbots may provide appropriate informational responses to common vascular surgery questions by patients. Vascular 2024:17085381241240550. [PMID: 38500300 DOI: 10.1177/17085381241240550] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/20/2024]
Abstract
OBJECTIVES Generative artificial intelligence (AI) has emerged as a promising tool to engage with patients. The objective of this study was to assess the quality of AI responses to common patient questions regarding vascular surgery disease processes. METHODS OpenAI's ChatGPT-3.5 and Google Bard were queried with 24 mock patient questions spanning seven vascular surgery disease domains. Six experienced vascular surgery faculty at a tertiary academic center independently graded AI responses on their accuracy (rated 1-4 from completely inaccurate to completely accurate), completeness (rated 1-4 from totally incomplete to totally complete), and appropriateness (binary). Responses were also evaluated with three readability scales. RESULTS ChatGPT responses were rated, on average, more accurate than Bard responses (3.08 ± 0.33 vs 2.82 ± 0.40, p < .01). ChatGPT responses were scored, on average, more complete than Bard responses (2.98 ± 0.34 vs 2.62 ± 0.36, p < .01). Most ChatGPT responses (75.0%, n = 18) and almost half of Bard responses (45.8%, n = 11) were unanimously deemed appropriate. Almost one-third of Bard responses (29.2%, n = 7) were deemed inappropriate by at least two reviewers (29.2%), and two Bard responses (8.4%) were considered inappropriate by the majority. The mean Flesch Reading Ease, Flesch-Kincaid Grade Level, and Gunning Fog Index of ChatGPT responses were 29.4 ± 10.8, 14.5 ± 2.2, and 17.7 ± 3.1, respectively, indicating that responses were readable with a post-secondary education. Bard's mean readability scores were 58.9 ± 10.5, 8.2 ± 1.7, and 11.0 ± 2.0, respectively, indicating that responses were readable with a high-school education (p < .0001 for three metrics). ChatGPT's mean response length (332 ± 79 words) was higher than Bard's mean response length (183 ± 53 words, p < .001). There was no difference in the accuracy, completeness, readability, or response length of ChatGPT or Bard between disease domains (p > .05 for all analyses). CONCLUSIONS AI offers a novel means of educating patients that avoids the inundation of information from "Dr Google" and the time barriers of physician-patient encounters. ChatGPT provides largely valid, though imperfect, responses to myriad patient questions at the expense of readability. While Bard responses are more readable and concise, their quality is poorer. Further research is warranted to better understand failure points for large language models in vascular surgery patient education.
Collapse
Affiliation(s)
- Ethan Chervonski
- New York University Grossman School of Medicine, New York, NY, USA
| | - Keerthi B Harish
- New York University Grossman School of Medicine, New York, NY, USA
| | - Caron B Rockman
- Division of Vascular & Endovascular Surgery, Department of Surgery, New York University Langone Health, New York, NY, USA
| | - Mikel Sadek
- Division of Vascular & Endovascular Surgery, Department of Surgery, New York University Langone Health, New York, NY, USA
| | - Katherine A Teter
- Division of Vascular & Endovascular Surgery, Department of Surgery, New York University Langone Health, New York, NY, USA
| | - Glenn R Jacobowitz
- Division of Vascular & Endovascular Surgery, Department of Surgery, New York University Langone Health, New York, NY, USA
| | - Todd L Berland
- Division of Vascular & Endovascular Surgery, Department of Surgery, New York University Langone Health, New York, NY, USA
| | - Joann Lohr
- Dorn Veterans Affairs Medical Center, Columbia, SC, USA
| | | | - Thomas S Maldonado
- Division of Vascular & Endovascular Surgery, Department of Surgery, New York University Langone Health, New York, NY, USA
| |
Collapse
|
4
|
Ng GW, Gan KF, Liew H, Ge L, Ang G, Molina J, Sun Y, Prakash PS, Harish KB, Lo ZJ. A Systematic Review and Classification of Factors Influencing Diabetic Foot Ulcer Treatment Adherence, in Accordance With the WHO Dimensions of Adherence to Long-Term Therapies. INT J LOW EXTR WOUND 2024:15347346241233962. [PMID: 38377963 DOI: 10.1177/15347346241233962] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/22/2024]
Abstract
PURPOSE Effective treatment of diabetic foot ulcers (DFUs) involves a multidisciplinary treatment plan to promote wound healing and prevent complications. Given the lack of consensus data on the factors affecting patient adherence, a systematic review was performed to identify and classify factors according to the WHO Dimensions of Adherence to Long-Term Therapies. METHODS Six hundred and forty-three articles from PubMed, Embase, and Scopus were reviewed. The inclusion criteria included qualitative and quantitative studies which discussed factors affecting patient adherence to DFU treatment, had study populations that comprised patients with either prior history of or existing DFU, and had either prior history of DFU treatment or were currently receiving treatment. Factors, and associated measures of adherence, were extracted and organized according to the WHO Dimensions of Adherence to Long-Term Therapies. RESULTS Seven quantitative and eight qualitative studies were included. Eleven patient-related factors, seven condition-related factors, three therapy-related factors, five socioeconomic factors, and five health system-related factors were investigated by the included studies. The largest proportion of factors studied was patient-related, such as patient insight on DFU treatment, patient motivation, and patient perception of DFU treatment. There was notable overlap in the range of discussed factors across various domains, in the socioeconomic (including social support, income, social and cultural acceptability of DFU therapy, cost) and therapy-related domains (including duration of treatment, offloading footwear, and reminder devices). Different studies found that specific factors, such as gender and patients having a low internal locus of control, had differing effects on adherence on different cohorts. CONCLUSION Current literature presents heterogeneous findings regarding factors affecting patient adherence. It would be useful for future studies to categorize factors as such to provide more comprehensive understanding and personalized care to patients. Further research can be done to explore how significant factors can be addressed universally across different cohort populations in different cultural and socioeconomic contexts.
Collapse
Affiliation(s)
- Gwyneth Wy Ng
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Keith F Gan
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Huiling Liew
- Department of Endocrinology, Tan Tock Seng Hospital, Singapore, Singapore
| | - Lixia Ge
- National Healthcare Group Health Services and Outcomes Research Unit, Singapore, Singapore
| | - Gary Ang
- National Healthcare Group Health Services and Outcomes Research Unit, Singapore, Singapore
| | - Joseph Molina
- National Healthcare Group Health Services and Outcomes Research Unit, Singapore, Singapore
| | - Yan Sun
- National Healthcare Group Health Services and Outcomes Research Unit, Singapore, Singapore
| | - Prajwala S Prakash
- Department of General Surgery, Tan Tock Seng Hospital, Singapore, Singapore
| | | | - Zhiwen Joseph Lo
- Department of Surgery, Woodlands Health, Singapore, Singapore
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| |
Collapse
|
5
|
Harish KB, Speranza G, Rockman CB, Sadek M, Jacobowitz GR, Garg K, Teter KA, Maldonado TS. Natural history of internal carotid artery stenosis progression. J Vasc Surg 2024; 79:297-304. [PMID: 37925038 DOI: 10.1016/j.jvs.2023.10.059] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Revised: 10/23/2023] [Accepted: 10/30/2023] [Indexed: 11/06/2023]
Abstract
OBJECTIVE The aim of this study was to investigate the natural history of internal carotid artery (ICA) stenosis progression. METHODS This single-institution retrospective cohort study analyzed patients diagnosed with ICA stenosis of 50% or greater on duplex ultrasound from 2015 to 2022. Subjects were drawn from our institutional Intersocietal Accreditation Commission-accredited noninterventional vascular laboratory database. Primary outcomes were incidences of disease progression, and stroke or revascularization after index study. Progression was defined as an increase in stenosis classification category. Imaging, demographic, and clinical data was obtained from our institutional electronic medical record via a database mining query. Cases were analyzed at the patient and artery levels, with severity corresponding to the greatest degree of ICA stenosis on index and follow-up studies. RESULTS Of 577 arteries in 467 patients, mean cohort age was 73.5 ± 8.9 years at the time of the index study, and 45.0% (n = 210) were female. Patients were followed with duplex ultrasound for a mean of 42.2 ± 22.7 months. Of 577 arteries, 65.5% (n = 378) at the index imaging study had moderate (50%-69%) stenosis, 23.7% (n = 137) had severe (70%-99%) stenosis, and 10.7% (n = 62) were occluded. These three groups had significant differences in age, hypertension, hyperlipidemia prevalence, and proportion on best medical therapy. Of the 467-patient cohort, 56.5% (n = 264) were on best medical therapy, defined as smoking cessation, treatment with an antiplatelet agent, statin, and antihypertensive and glycemic agents as indicated. Mean time to progression for affected arteries was 28.0 ± 20.5 months. Of those arteries with nonocclusive disease at diagnosis, 21.3% (n = 123) progressed in their level of stenosis. Older age, diabetes, and a history of vasculitis were associated with stenosis progression, whereas antiplatelet agent use trended towards decreased progression rates. Of the 467 patients, 5.6% (n = 26) developed symptoms; of those, 38.5% (n = 10) had ischemic strokes, 26.9% (n = 7) had hemispheric transient ischemic attacks, 11.5% (n = 3) had amaurosis fugax, and 23.1% (n = 6) had other symptoms. A history of head and neck cancer was positively associated with symptom development. Of 577 affected arteries, 16.6% (n = 96) underwent intervention; 81% (n = 78) of interventions were for asymptomatic disease and 19% (n = 18) were for symptomatic disease. No patient-level factors were associated with risk of intervention. CONCLUSIONS A significant number of carotid stenosis patients experience progression of disease. Physicians should consider long-term surveillance on all patients with carotid disease, with increased attention paid to those with risk factors for progression, particularly those with diabetes and a history of vasculitis.
Collapse
Affiliation(s)
| | | | - Caron B Rockman
- Division of Vascular Surgery, Department of Surgery, New York University Langone Health, New York, NY
| | - Mikel Sadek
- Division of Vascular Surgery, Department of Surgery, New York University Langone Health, New York, NY
| | - Glenn R Jacobowitz
- Division of Vascular Surgery, Department of Surgery, New York University Langone Health, New York, NY
| | - Karan Garg
- Division of Vascular Surgery, Department of Surgery, New York University Langone Health, New York, NY
| | - Katherine A Teter
- Division of Vascular Surgery, Department of Surgery, New York University Langone Health, New York, NY
| | - Thomas S Maldonado
- Division of Vascular Surgery, Department of Surgery, New York University Langone Health, New York, NY.
| |
Collapse
|
6
|
Lo ZJ, Harish KB, Tan E, Zhu J, Chan S, Liew H, Hoi WH, Liang S, Cho YT, Koo HY, Wu K, Car J. A feasibility study on the efficacy of a patient-owned wound surveillance system for diabetic foot ulcer care (ePOWS study). Digit Health 2023; 9:20552076231205747. [PMID: 37808235 PMCID: PMC10559723 DOI: 10.1177/20552076231205747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Accepted: 09/19/2023] [Indexed: 10/10/2023] Open
Abstract
Objective Wound image analysis tools hold promise in helping patients to monitor their wounds. We aim to perform a novel feasibility study on the efficacy of a patient-owned wound surveillance system for diabetic foot ulcer (DFU) care. Methods This two-institutional, prospective, single-arm pilot study examined patients with DFU. An artificial intelligence-enabled image analysis app calculating the wound surface area was installed and patients or caregivers were instructed to take pictures of wounds during dressing changes. Patients were followed until wound deterioration, wound healing, or wound stability at 6 months occurred and the outcomes of interest included study adherence, algorithm performance, and user experience. Results Between January 2021 and December 2021, 39 patients were enrolled in the study, with a mean age of 61.6 ± 8.6 years, and 69% (n = 27) of subjects were male. All patients had documented diabetes and 85% (n = 33) of them had peripheral arterial disease. A mean follow-up for those completing the study was 12.0 ± 8.5 weeks. At the conclusion of the study, 80% of patients (n = 20) had primary wound healing whilst 20% (n = 5) had wound deterioration. The study completion rate was 64% (n = 25). Usage of the app for surveillance of DFU healing, as compared to physician evaluation, yielded a sensitivity of 100%, specificity of 20%, positive predictive value of 83%, and negative predictive value of 100%. Of those who provided user experience feedback, 59% (n = 10) felt the app was easy to use, 47% (n = 8) would recommend the wound analysis app to others but only 6% would pay for the app out of pocket (n = 1). Conclusion Implementation of a patient-owned wound surveillance system is feasible. Most patients were able to effectively monitor wounds using a smartphone app-based solution. The image analysis algorithm demonstrates strong performance in identifying wound healing and is capable of detecting deterioration prior to interval evaluation by a physician. Patients generally found the app easy to use but were reluctant to pay for the use of the solution out of pocket.
Collapse
Affiliation(s)
- Zhiwen J Lo
- Department of Surgery, Vascular Surgery Service, Woodlands Health, Singapore, Singapore
- Centre for Population Health Sciences, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | | | - Elaine Tan
- National Healthcare Group Polyclinics, Singapore, Singapore
| | - Julia Zhu
- National Healthcare Group Polyclinics, Singapore, Singapore
| | - Shaun Chan
- Department of General Surgery, Vascular Surgery Service, Khoo Teck Puat Hospital, Singapore, Singapore
| | - Huiling Liew
- Department of Endocrinology, Tan Tock Seng Hospital, Singapore, Singapore
| | - Wai H Hoi
- Department of Endocrinology, Woodlands Health, Singapore, Singapore
| | - Shanying Liang
- Department of Surgery, Vascular Surgery Service, Woodlands Health, Singapore, Singapore
| | - Yuan T Cho
- Department of Surgery, Vascular Surgery Service, Woodlands Health, Singapore, Singapore
| | - Hui Y Koo
- Group Integrated Care, National Healthcare Group, Singapore, Singapore
| | | | - Josip Car
- Centre for Population Health Sciences, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
- Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, UK
| |
Collapse
|
7
|
Harish KB, Price WN, Aphinyanaphongs Y. Open-Source Clinical Machine Learning Models: Critical Appraisal of Feasibility, Advantages, and Challenges. JMIR Form Res 2022; 6:e33970. [PMID: 35404258 PMCID: PMC9039816 DOI: 10.2196/33970] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2021] [Revised: 01/05/2022] [Accepted: 01/19/2022] [Indexed: 12/12/2022] Open
Abstract
Machine learning applications promise to augment clinical capabilities and at least 64 models have already been approved by the US Food and Drug Administration. These tools are developed, shared, and used in an environment in which regulations and market forces remain immature. An important consideration when evaluating this environment is the introduction of open-source solutions in which innovations are freely shared; such solutions have long been a facet of digital culture. We discuss the feasibility and implications of open-source machine learning in a health care infrastructure built upon proprietary information. The decreased cost of development as compared to drugs and devices, a longstanding culture of open-source products in other industries, and the beginnings of machine learning–friendly regulatory pathways together allow for the development and deployment of open-source machine learning models. Such tools have distinct advantages including enhanced product integrity, customizability, and lower cost, leading to increased access. However, significant questions regarding engineering concerns about implementation infrastructure and model safety, a lack of incentives from intellectual property protection, and nebulous liability rules significantly complicate the ability to develop such open-source models. Ultimately, the reconciliation of open-source machine learning and the proprietary information–driven health care environment requires that policymakers, regulators, and health care organizations actively craft a conducive market in which innovative developers will continue to both work and collaborate.
Collapse
Affiliation(s)
- Keerthi B Harish
- Grossman School of Medicine, New York University, New York, NY, United States
| | - W Nicholson Price
- Law School, University of Michigan, Ann Arbor, MI, United States.,Centre for Advanced Studies In Biomedical Innovation Law, University of Copenhagen, Copenhagen, Denmark
| | | |
Collapse
|
8
|
Razavian N, Major VJ, Sudarshan M, Burk-Rafel J, Stella P, Randhawa H, Bilaloglu S, Chen J, Nguy V, Wang W, Zhang H, Reinstein I, Kudlowitz D, Zenger C, Cao M, Zhang R, Dogra S, Harish KB, Bosworth B, Francois F, Horwitz LI, Ranganath R, Austrian J, Aphinyanaphongs Y. A validated, real-time prediction model for favorable outcomes in hospitalized COVID-19 patients. NPJ Digit Med 2020; 3:130. [PMID: 33083565 PMCID: PMC7538971 DOI: 10.1038/s41746-020-00343-x] [Citation(s) in RCA: 37] [Impact Index Per Article: 9.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2020] [Accepted: 09/17/2020] [Indexed: 12/26/2022] Open
Abstract
The COVID-19 pandemic has challenged front-line clinical decision-making, leading to numerous published prognostic tools. However, few models have been prospectively validated and none report implementation in practice. Here, we use 3345 retrospective and 474 prospective hospitalizations to develop and validate a parsimonious model to identify patients with favorable outcomes within 96 h of a prediction, based on real-time lab values, vital signs, and oxygen support variables. In retrospective and prospective validation, the model achieves high average precision (88.6% 95% CI: [88.4-88.7] and 90.8% [90.8-90.8]) and discrimination (95.1% [95.1-95.2] and 86.8% [86.8-86.9]) respectively. We implemented and integrated the model into the EHR, achieving a positive predictive value of 93.3% with 41% sensitivity. Preliminary results suggest clinicians are adopting these scores into their clinical workflows.
Collapse
Affiliation(s)
- Narges Razavian
- Department of Population Health, NYU Grossman School of Medicine, New York, NY USA
- Center for Healthcare Innovation and Delivery Science, NYU Langone Health, New York, NY USA
- Center for Data Science, New York University, New York, NY USA
| | - Vincent J. Major
- Department of Population Health, NYU Grossman School of Medicine, New York, NY USA
| | - Mukund Sudarshan
- Courant Institute of Mathematical Sciences, New York University, New York, NY USA
| | - Jesse Burk-Rafel
- Department of Medicine, NYU Grossman School of Medicine, New York, NY USA
| | - Peter Stella
- Department of Pediatrics, NYU Grossman School of Medicine, New York, NY USA
| | | | - Seda Bilaloglu
- Department of Population Health, NYU Grossman School of Medicine, New York, NY USA
| | - Ji Chen
- Department of Population Health, NYU Grossman School of Medicine, New York, NY USA
| | - Vuthy Nguy
- Department of Population Health, NYU Grossman School of Medicine, New York, NY USA
| | - Walter Wang
- Department of Population Health, NYU Grossman School of Medicine, New York, NY USA
| | - Hao Zhang
- Department of Population Health, NYU Grossman School of Medicine, New York, NY USA
| | - Ilan Reinstein
- Institute for Innovations in Medical Education, NYU Grossman School of Medicine, New York, NY USA
| | - David Kudlowitz
- Department of Medicine, NYU Grossman School of Medicine, New York, NY USA
| | - Cameron Zenger
- Department of Medicine, NYU Grossman School of Medicine, New York, NY USA
| | - Meng Cao
- Department of Medicine, NYU Grossman School of Medicine, New York, NY USA
| | - Ruina Zhang
- Department of Medicine, NYU Grossman School of Medicine, New York, NY USA
| | - Siddhant Dogra
- Department of Medicine, NYU Grossman School of Medicine, New York, NY USA
| | - Keerthi B. Harish
- Department of Population Health, NYU Grossman School of Medicine, New York, NY USA
| | - Brian Bosworth
- Department of Medicine, NYU Grossman School of Medicine, New York, NY USA
- NYU Langone Health, New York, NY USA
| | - Fritz Francois
- Department of Medicine, NYU Grossman School of Medicine, New York, NY USA
- NYU Langone Health, New York, NY USA
| | - Leora I. Horwitz
- Department of Population Health, NYU Grossman School of Medicine, New York, NY USA
- Center for Healthcare Innovation and Delivery Science, NYU Langone Health, New York, NY USA
- Department of Medicine, NYU Grossman School of Medicine, New York, NY USA
| | - Rajesh Ranganath
- Department of Population Health, NYU Grossman School of Medicine, New York, NY USA
- Center for Data Science, New York University, New York, NY USA
- Courant Institute of Mathematical Sciences, New York University, New York, NY USA
| | - Jonathan Austrian
- Department of Medicine, NYU Grossman School of Medicine, New York, NY USA
- Medical Center IT, NYU Langone Health, New York, NY USA
| | - Yindalon Aphinyanaphongs
- Department of Population Health, NYU Grossman School of Medicine, New York, NY USA
- Center for Healthcare Innovation and Delivery Science, NYU Langone Health, New York, NY USA
| |
Collapse
|