1
|
Paul E, Chakraborty B, Sikorskii A, Ghosh S. A framework for testing non-inferiority in a three-arm, sequential, multiple assignment randomized trial. Stat Methods Med Res 2024; 33:611-633. [PMID: 38400576 DOI: 10.1177/09622802241232124] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/25/2024]
Abstract
Sequential multiple assignment randomized trial design is becoming increasingly used in the field of precision medicine. This design allows comparisons of sequences of adaptive interventions tailored to the individual patient. Superiority testing is usually the initial goal in order to determine which embedded adaptive intervention yields the best primary outcome on average. When direct superiority is not evident, yet an adaptive intervention poses other benefits, then non-inferiority testing is warranted. Non-inferiority testing in the sequential multiple assignment randomized trial setup is rather new and involves the specification of non-inferiority margin and other important assumptions that are often unverifiable internally. These challenges are not specific to sequential multiple assignment randomized trial and apply to two-arm non-inferiority trials that do not include a standard-of-care (or placebo) arm. To address some of these challenges, three-arm non-inferiority trials that include the standard-of-care arm are proposed. However, methods developed so far for three-arm non-inferiority trials are not sequential multiple assignment randomized trial-specific. This is because apart from embedded adaptive interventions, sequential multiple assignment randomized trial typically does not include a third standard-of-care arm. In this article, we consider a three-arm sequential multiple assignment randomized trial from an National Institutes of Health-funded study of symptom management strategies among people undergoing cancer treatment. Motivated by that example, we propose a novel data analytic method for non-inferiority testing in the framework of three-arm sequential multiple assignment randomized trial for the first time. Sample size and power considerations are discussed through extensive simulation studies to elucidate our method.
Collapse
Affiliation(s)
- Erina Paul
- Biostatistics and Research Decision Sciences, Merck & Co., Inc., Kenilworth, NJ, USA
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
- Department of Statistics and Data Science, National University of Singapore, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| | - Alla Sikorskii
- Department of Psychiatry, Michigan State University, East Lansing, MI, USA
| | - Samiran Ghosh
- Department of Biostatistics & Data Science and Institute for Implementation Science, School of Public Health, University of Texas, Houston, TX, USA
| |
Collapse
|
2
|
Aravindhan A, Fenwick EK, Chan AWD, Man REK, Tan NC, Wong WT, Soo WF, Lim SW, Wee SYM, Sabanayagam C, Finkelstein E, Tan G, Hamzah H, Chakraborty B, Acharyya S, Shyong TE, Scanlon P, Wong TY, Lamoureux EL. Extending the diabetic retinopathy screening intervals in Singapore: methodology and preliminary findings of a cohort study. BMC Public Health 2024; 24:786. [PMID: 38481239 PMCID: PMC10935797 DOI: 10.1186/s12889-024-18287-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2023] [Accepted: 03/05/2024] [Indexed: 03/17/2024] Open
Abstract
BACKGROUND The Diabetic Retinopathy Extended Screening Study (DRESS) aims to develop and validate a new DR/diabetic macular edema (DME) risk stratification model in patients with Type 2 diabetes (DM) to identify low-risk groups who can be safely assigned to biennial or triennial screening intervals. We describe the study methodology, participants' baseline characteristics, and preliminary DR progression rates at the first annual follow-up. METHODS DRESS is a 3-year ongoing longitudinal study of patients with T2DM and no or mild non-proliferative DR (NPDR, non-referable) who underwent teleophthalmic screening under the Singapore integrated Diabetic Retinopathy Programme (SiDRP) at four SingHealth Polyclinics. Patients with referable DR/DME (> mild NPDR) or ungradable fundus images were excluded. Sociodemographic, lifestyle, medical and clinical information was obtained from medical records and interviewer-administered questionnaires at baseline. These data are extracted from medical records at 12, 24 and 36 months post-enrollment. Baseline descriptive characteristics stratified by DR severity at baseline and rates of progression to referable DR at 12-month follow-up were calculated. RESULTS Of 5,840 eligible patients, 78.3% (n = 4,570, median [interquartile range [IQR] age 61.0 [55-67] years; 54.7% male; 68.0% Chinese) completed the baseline assessment. At baseline, 97.4% and 2.6% had none and mild NPDR (worse eye), respectively. Most participants had hypertension (79.2%) and dyslipidemia (92.8%); and almost half were obese (43.4%, BMI ≥ 27.5 kg/m2). Participants without DR (vs mild DR) reported shorter DM duration, and had lower haemoglobin A1c, triglycerides and urine albumin/creatinine ratio (all p < 0.05). To date, we have extracted 41.8% (n = 1909) of the 12-month follow-up data. Of these, 99.7% (n = 1,904) did not progress to referable DR. Those who progressed to referable DR status (0.3%) had no DR at baseline. CONCLUSIONS In our prospective study of patients with T2DM and non-referable DR attending polyclinics, we found extremely low annual DR progression rates. These preliminary results suggest that extending screening intervals beyond 12 months may be viable and safe for most participants, although our 3-year follow up data are needed to substantiate this claim and develop the risk stratification model to identify low-risk patients with T2DM who can be assigned biennial or triennial screening intervals.
Collapse
Affiliation(s)
- Amudha Aravindhan
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| | - Eva K Fenwick
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| | - Aurora Wing Dan Chan
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
| | - Ryan Eyn Kidd Man
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| | | | | | | | | | | | - Charumathi Sabanayagam
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| | | | - Gavin Tan
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
- Duke-NUS Medical School, Singapore, Singapore
| | - Haslina Hamzah
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore
| | | | | | - Tai E Shyong
- Saw Swee Hock School of Public Health, National University of Singapore and National University Health System, Singapore, Singapore
| | - Peter Scanlon
- Gloucestershire Hospitals NHS Foundation Trust, Gloucester, UK
| | | | - Ecosse L Lamoureux
- Singapore Eye Research Institute and Singapore National Eye Centre, Singapore, Singapore.
- Duke-NUS Medical School, Singapore, Singapore.
- The University of Melbourne, Melbourne, Australia.
| |
Collapse
|
3
|
Arévalo Avalos MR, Xu J, Figueroa CA, Haro-Ramos AY, Chakraborty B, Aguilera A. The effect of cognitive behavioral therapy text messages on mood: A micro-randomized trial. PLOS Digit Health 2024; 3:e0000449. [PMID: 38381747 PMCID: PMC10880955 DOI: 10.1371/journal.pdig.0000449] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/20/2023] [Accepted: 01/15/2024] [Indexed: 02/23/2024]
Abstract
The StayWell at Home intervention, a 60-day text-messaging program based on Cognitive Behavioral Therapy (CBT) principles, was developed to help adults cope with the adverse effects of the global pandemic. Participants in StayWell at Home were found to show reduced depressive and anxiety symptoms after participation. However, it remains unclear whether the intervention improved mood and which intervention components were most effective at improving user mood during the pandemic. Thus, utilizing a micro-randomized trial (MRT) design, we examined two intervention components to inform the mechanisms of action that improve mood: 1) text messages delivering CBT-informed coping strategies (i.e., behavioral activation, other coping skills, or social support); 2) time at which messages were sent. Data from two independent trials of StayWell are included in this paper. The first trial included 303 adults aged 18 or older, and the second included 266 adults aged 18 or older. Participants were recruited via online platforms (e.g., Facebook ads) and partnerships with community-based agencies aiming to reach diverse populations, including low-income individuals and people of color. The results of this paper indicate that participating in the program improved and sustained self-reported mood ratings among participants. We did not find significant differences between the type of message delivered and mood ratings. On the other hand, the results from Phase 1 indicated that delivering any type of message in the 3 pm-6 pm time window improved mood significantly over sending a message in the 9 am-12 pm time window. The StayWell at Home program increases in mood ratings appeared more pronounced during the first two to three weeks of the intervention and were maintained for the remainder of the study period. The current paper provides evidence that low-burden text-message interventions may effectively address behavioral health concerns among diverse communities.
Collapse
Affiliation(s)
- Marvyn R. Arévalo Avalos
- School of Social Welfare, University of California Berkeley, Berkeley, California, United States of America
| | - Jing Xu
- Centre for Quantitative Medicine, Duke-NUS Medical School, National University of Singapore, Singapore
| | - Caroline Astrid Figueroa
- School of Social Welfare, University of California Berkeley, Berkeley, California, United States of America
- Faculty of Technology, Policy, and Management, Delft Technical University, Delft, The Netherlands
| | - Alein Y. Haro-Ramos
- School of Public Health, Health Policy and Management, University of California Berkeley, Berkeley, California, United States of America
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, National University of Singapore, Singapore
- Department of Statistics and Data Science, National University of Singapore, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, United States of America
| | - Adrian Aguilera
- School of Social Welfare, University of California Berkeley, Berkeley, California, United States of America
- Department of Psychiatry and Behavioral Sciences, University of California–San Francisco, San Francisco, California, United States of America
| |
Collapse
|
4
|
Wang X, Deliu N, Narita Y, Chakraborty B. Incorporating participants' welfare into sequential multiple assignment randomized trials. Biometrics 2024; 80:ujad004. [PMID: 38364800 DOI: 10.1093/biomtc/ujad004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 07/02/2023] [Accepted: 11/03/2023] [Indexed: 02/18/2024]
Abstract
Dynamic treatment regimes (DTRs) are sequences of decision rules that recommend treatments based on patients' time-varying clinical conditions. The sequential, multiple assignment, randomized trial (SMART) is an experimental design that can provide high-quality evidence for constructing optimal DTRs. In a conventional SMART, participants are randomized to available treatments at multiple stages with balanced randomization probabilities. Despite its relative simplicity of implementation and desirable performance in comparing embedded DTRs, the conventional SMART faces inevitable ethical issues, including assigning many participants to the empirically inferior treatment or the treatment they dislike, which might slow down the recruitment procedure and lead to higher attrition rates, ultimately leading to poor internal and external validities of the trial results. In this context, we propose a SMART under the Experiment-as-Market framework (SMART-EXAM), a novel SMART design that holds the potential to improve participants' welfare by incorporating their preferences and predicted treatment effects into the randomization procedure. We describe the steps of conducting a SMART-EXAM and evaluate its performance compared to the conventional SMART. The results indicate that the SMART-EXAM can improve the welfare of the participants enrolled in the trial, while also achieving a desirable ability to construct an optimal DTR when the experimental parameters are suitably specified. We finally illustrate the practical potential of the SMART-EXAM design using data from a SMART for children with attention-deficit/hyperactivity disorder.
Collapse
Affiliation(s)
- Xinru Wang
- Centre for Quantitative Medicine, Duke-NUS Medical School, 169857, Singapore
| | - Nina Deliu
- MEMOTEF Department, Sapienza University of Rome, Rome, 00161, Italy
- MRC-Biostatistics Unit, University of Cambridge, Cambridge, CB2 0SR, United Kingdom
| | - Yusuke Narita
- Department of Economics and Cowles Foundation, Yale University, New Haven, CT, 06520-8281, United States
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, 169857, Singapore
- Program in Health Services and Systems Research, Duke-NUS Medical School, 169857, Singapore
- Department of Statistics and Data Science, National University of Singapore, 117546, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, 27710, United States
| |
Collapse
|
5
|
Li S, Liu P, Nascimento GG, Wang X, Leite FRM, Chakraborty B, Hong C, Ning Y, Xie F, Teo ZL, Ting DSW, Haddadi H, Ong MEH, Peres MA, Liu N. Federated and distributed learning applications for electronic health records and structured medical data: a scoping review. J Am Med Inform Assoc 2023; 30:2041-2049. [PMID: 37639629 PMCID: PMC10654866 DOI: 10.1093/jamia/ocad170] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Revised: 07/19/2023] [Indexed: 08/31/2023] Open
Abstract
OBJECTIVES Federated learning (FL) has gained popularity in clinical research in recent years to facilitate privacy-preserving collaboration. Structured data, one of the most prevalent forms of clinical data, has experienced significant growth in volume concurrently, notably with the widespread adoption of electronic health records in clinical practice. This review examines FL applications on structured medical data, identifies contemporary limitations, and discusses potential innovations. MATERIALS AND METHODS We searched 5 databases, SCOPUS, MEDLINE, Web of Science, Embase, and CINAHL, to identify articles that applied FL to structured medical data and reported results following the PRISMA guidelines. Each selected publication was evaluated from 3 primary perspectives, including data quality, modeling strategies, and FL frameworks. RESULTS Out of the 1193 papers screened, 34 met the inclusion criteria, with each article consisting of one or more studies that used FL to handle structured clinical/medical data. Of these, 24 utilized data acquired from electronic health records, with clinical predictions and association studies being the most common clinical research tasks that FL was applied to. Only one article exclusively explored the vertical FL setting, while the remaining 33 explored the horizontal FL setting, with only 14 discussing comparisons between single-site (local) and FL (global) analysis. CONCLUSIONS The existing FL applications on structured medical data lack sufficient evaluations of clinically meaningful benefits, particularly when compared to single-site analyses. Therefore, it is crucial for future FL applications to prioritize clinical motivations and develop designs and methodologies that can effectively support and aid clinical practice and research.
Collapse
Affiliation(s)
- Siqi Li
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Pinyan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Gustavo G Nascimento
- National Dental Research Institute Singapore, National Dental Centre Singapore, Singapore 168938, Singapore
- Oral Health Academic Clinical Programme, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Xinru Wang
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Fabio Renato Manzolli Leite
- National Dental Research Institute Singapore, National Dental Centre Singapore, Singapore 168938, Singapore
- Oral Health Academic Clinical Programme, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore 169857, Singapore
- Department of Statistics and Data Science, National University of Singapore, Singapore 117546, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC 27708, United States
| | - Chuan Hong
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC 27708, United States
| | - Yilin Ning
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Feng Xie
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Zhen Ling Teo
- Singapore National Eye Centre, Singapore, Singapore Eye Research Institute, Singapore 168751, Singapore
| | - Daniel Shu Wei Ting
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
- Singapore National Eye Centre, Singapore, Singapore Eye Research Institute, Singapore 168751, Singapore
| | - Hamed Haddadi
- Department of Computing, Imperial College London, London SW7 2AZ, England, United Kingdom
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore 169857, Singapore
- Department of Emergency Medicine, Singapore General Hospital, Singapore 169608, Singapore
| | - Marco Aurélio Peres
- National Dental Research Institute Singapore, National Dental Centre Singapore, Singapore 168938, Singapore
- Oral Health Academic Clinical Programme, Duke-NUS Medical School, Singapore 169857, Singapore
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Nan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore 169857, Singapore
- Institute of Data Science, National University of Singapore, Singapore 117602, Singapore
| |
Collapse
|
6
|
Saberi P, Stoner MCD, McCuistian CL, Balaban C, Ming K, Wagner D, Chakraborty B, Smith L, Sukhija-Cohen A, Neilands TB, Gruber VA, Johnson MO. iVY: protocol for a randomised clinical trial to test the effect of a technology-based intervention to improve virological suppression among young adults with HIV in the USA. BMJ Open 2023; 13:e077676. [PMID: 37802624 PMCID: PMC10565330 DOI: 10.1136/bmjopen-2023-077676] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Accepted: 09/19/2023] [Indexed: 10/10/2023] Open
Abstract
INTRODUCTION Young adults with HIV (YWH) experience worse clinical outcomes than adults and have high rates of substance use (SU) and mental illness that impact their engagement in care and adherence to antiretroviral therapy (ART). The intervention for Virologic Suppression in Youth (iVY) aims to address treatment engagement/adherence, mental health (MH) and SU in a tailored manner using a differentiated care approach that is youth friendly. Findings will provide information about the impact of iVY on HIV virological suppression, MH and SU among YWH who are disproportionately impacted by HIV and at elevated risk for poor health outcomes. METHODS AND ANALYSIS The iVY study will test the effect of a technology-based intervention with differing levels of resource requirements (ie, financial and personnel time) in a randomised clinical trial with an adaptive treatment strategy among 200 YWH (18-29 years old). The primary outcome is HIV virological suppression measured via dried blood spot. This piloted and protocolised intervention combines: (1) brief weekly sessions with a counsellor via a video-chat platform (video-counselling) to discuss MH, SU, HIV care engagement/adherence and other barriers to care; and (2) a mobile health app to address barriers such as ART forgetfulness, and social isolation. iVY has the potential to address important, distinct and changing barriers to HIV care engagement (eg, MH, SU) to increase virological suppression among YWH at elevated risk for poor health outcomes. ETHICS AND DISSEMINATION This study and its protocols have been approved by the University of California, San Francisco Institutional Review Board. Study staff will work with a Youth Advisory Panel to disseminate results to YWH, participants and the academic community. TRIAL REGISTRATION NUMBER NCT05877729.
Collapse
Affiliation(s)
- Parya Saberi
- Center for AIDS Prevention Studies, University of California San Francisco, San Francisco, California, USA
| | - Marie C D Stoner
- Women's Global Health Imperative, RTI International, Berkeley, California, USA
| | - Caravella L McCuistian
- Department of Psychiatry and Behavioral Sciences, University of California San Francisco, San Francisco, CA, USA
| | - Celeste Balaban
- Center for AIDS Prevention Studies, University of California San Francisco, San Francisco, California, USA
| | - Kristin Ming
- Center for AIDS Prevention Studies, University of California San Francisco, San Francisco, California, USA
| | - Danielle Wagner
- Women's Global Health Imperative, RTI International, Berkeley, California, USA
| | - Bibhas Chakraborty
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, NC, USA
- Centre for Quantitative Medicine and Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore
- Department of Statistics and Data Science, National University of Singapore, Singapore
| | - Louis Smith
- Center for AIDS Prevention Studies, University of California San Francisco, San Francisco, California, USA
| | | | - Torsten B Neilands
- Center for AIDS Prevention Studies, University of California San Francisco, San Francisco, California, USA
| | - Valerie A Gruber
- Department of Psychiatry and Behavioral Sciences, University of California San Francisco, San Francisco, CA, USA
| | - Mallory O Johnson
- Center for AIDS Prevention Studies, University of California San Francisco, San Francisco, California, USA
| |
Collapse
|
7
|
Mitra S, Kroeger CM, Xu J, Avery L, Masedunskas A, Cassidy S, Wang T, Hunyor I, Wilcox I, Huang R, Chakraborty B, Fontana L. Testing the Effects of App-Based Motivational Messages on Physical Activity and Resting Heart Rate Through Smartphone App Compliance in Patients With Vulnerable Coronary Artery Plaques: Protocol for a Microrandomized Trial. JMIR Res Protoc 2023; 12:e46082. [PMID: 37782531 PMCID: PMC10580140 DOI: 10.2196/46082] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2023] [Revised: 06/29/2023] [Accepted: 07/24/2023] [Indexed: 10/03/2023] Open
Abstract
BACKGROUND Achieving the weekly physical activity recommendations of at least 150-300 minutes of moderate-intensity or 75-150 minutes of vigorous-intensity aerobic exercise is important for reducing cardiometabolic risk, but evidence shows that most people struggle to meet these goals, particularly in the mid to long term. OBJECTIVE The Messages Improving Resting Heart Health (MIRTH) study aims to determine if (1) sending daily motivational messages through a research app is effective in improving motivation and in promoting adherence to physical activity recommendations in men and women with coronary heart disease randomized to a 12-month intensive lifestyle intervention, and (2) the time of the day when the message is delivered impacts compliance with exercise training. METHODS We will conduct a single-center, microrandomized trial. Participants will be randomized daily to either receive or not receive motivational messages over two 90-day periods at the beginning (phase 1: months 4-6) and at the end (phase 2: months 10-12) of the Lifestyle Vulnerable Plaque Study. Wrist-worn devices (Fitbit Inspire 2) and Bluetooth pairing with smartphones will be used to passively collect data for proximal (ie, physical activity duration, steps walked, and heart rate within 180 minutes of receiving messages) and distal (ie, change values for resting heart rate and total steps walked within and across both phases 1 and 2 of the trial) outcomes. Participants will be recruited from a large academic cardiology office practice (Central Sydney Cardiology) and the Royal Prince Alfred Hospital Departments of Cardiology and Radiology. All clinical investigations will be undertaken at the Charles Perkins Centre Royal Prince Alfred clinic. Individuals aged 18-80 years (n=58) with stable coronary heart disease who have low attenuation plaques based on a coronary computed tomography angiography within the past 3 months and have been randomized to an intensive lifestyle intervention program will be included in MIRTH. RESULTS The Lifestyle Vulnerable Plaque Study was funded in 2020 and started enrolling participants in February 2022. Recruitment for MIRTH commenced in November 2022. As of September 2023, 2 participants were enrolled in the MIRTH study and provided baseline data. CONCLUSIONS This MIRTH microrandomized trial will represent the single most detailed and integrated analysis of the effects of a comprehensive lifestyle intervention delivered through a customized mobile health app on smart devices on time-based motivational messaging for patients with coronary heart disease. This study will also help inform future studies optimizing for just-in-time adaptive interventions. TRIAL REGISTRATION Australian New Zealand Clinical Trials Registry ACTRN12622000731796; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=382861. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) DERR1-10.2196/46082.
Collapse
Affiliation(s)
- Sayan Mitra
- Central Clinical School, Faculty of Medicine and Health, The University of Sydney, Camperdown, Australia
- Charles Perkins Centre, The University of Sydney, Camperdown, Australia
| | - Cynthia M Kroeger
- Central Clinical School, Faculty of Medicine and Health, The University of Sydney, Camperdown, Australia
- Charles Perkins Centre, The University of Sydney, Camperdown, Australia
| | - Jing Xu
- Office of Education, Duke-National University of Singapore Medical School, Singapore, Singapore
- Program in Health Services and Systems Research, Duke-National University of Singapore Medical School, Singapore, Singapore
- Centre for Quantitative Medicine, Duke-National University of Singapore Medical School, Singapore, Singapore
| | - Leah Avery
- School of Health & Life Sciences, Teesside University, Tees Valley, England, United Kingdom
| | - Andrius Masedunskas
- Central Clinical School, Faculty of Medicine and Health, The University of Sydney, Camperdown, Australia
- Charles Perkins Centre, The University of Sydney, Camperdown, Australia
| | - Sophie Cassidy
- Central Clinical School, Faculty of Medicine and Health, The University of Sydney, Camperdown, Australia
- Charles Perkins Centre, The University of Sydney, Camperdown, Australia
| | - Tian Wang
- Central Clinical School, Faculty of Medicine and Health, The University of Sydney, Camperdown, Australia
- Charles Perkins Centre, The University of Sydney, Camperdown, Australia
| | - Imre Hunyor
- Central Clinical School, Faculty of Medicine and Health, The University of Sydney, Camperdown, Australia
- Department of Cardiology, Royal Prince Alfred Hospital, Sydney, Australia
- Central Sydney Cardiology, Royal Prince Alfred Medical Centre, Sydney, Australia
| | - Ian Wilcox
- Central Clinical School, Faculty of Medicine and Health, The University of Sydney, Camperdown, Australia
- Department of Cardiology, Royal Prince Alfred Hospital, Sydney, Australia
- Central Sydney Cardiology, Royal Prince Alfred Medical Centre, Sydney, Australia
| | - Robin Huang
- School of Computer Science, The University of Sydney, Darlington, Australia
| | - Bibhas Chakraborty
- Program in Health Services and Systems Research, Duke-National University of Singapore Medical School, Singapore, Singapore
- Centre for Quantitative Medicine, Duke-National University of Singapore Medical School, Singapore, Singapore
- Department of Statistics and Data Science, National University of Singapore, Singapore, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States
| | - Luigi Fontana
- Central Clinical School, Faculty of Medicine and Health, The University of Sydney, Camperdown, Australia
- Charles Perkins Centre, The University of Sydney, Camperdown, Australia
- Department of Endocrinology, Royal Prince Alfred Hospital, Sydney, Australia
- Department of Clinical and Experimental Sciences, Brescia University, Brescia, Italy
| |
Collapse
|
8
|
Li S, Ning Y, Ong MEH, Chakraborty B, Hong C, Xie F, Yuan H, Liu M, Buckland DM, Chen Y, Liu N. FedScore: A privacy-preserving framework for federated scoring system development. J Biomed Inform 2023; 146:104485. [PMID: 37660960 DOI: 10.1016/j.jbi.2023.104485] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Revised: 08/08/2023] [Accepted: 08/31/2023] [Indexed: 09/05/2023]
Abstract
OBJECTIVE We propose FedScore, a privacy-preserving federated learning framework for scoring system generation across multiple sites to facilitate cross-institutional collaborations. MATERIALS AND METHODS The FedScore framework includes five modules: federated variable ranking, federated variable transformation, federated score derivation, federated model selection and federated model evaluation. To illustrate usage and assess FedScore's performance, we built a hypothetical global scoring system for mortality prediction within 30 days after a visit to an emergency department using 10 simulated sites divided from a tertiary hospital in Singapore. We employed a pre-existing score generator to construct 10 local scoring systems independently at each site and we also developed a scoring system using centralized data for comparison. RESULTS We compared the acquired FedScore model's performance with that of other scoring models using the receiver operating characteristic (ROC) analysis. The FedScore model achieved an average area under the curve (AUC) value of 0.763 across all sites, with a standard deviation (SD) of 0.020. We also calculated the average AUC values and SDs for each local model, and the FedScore model showed promising accuracy and stability with a high average AUC value which was closest to the one of the pooled model and SD which was lower than that of most local models. CONCLUSION This study demonstrates that FedScore is a privacy-preserving scoring system generator with potentially good generalizability.
Collapse
Affiliation(s)
- Siqi Li
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Yilin Ning
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore; Health Services Research Centre, Singapore Health Services, Singapore, Singapore; Department of Emergency Medicine, Singapore General Hospital, Singapore, Singapore
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore; Department of Statistics and Data Science, National University of Singapore, Singapore, Singapore; Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| | - Chuan Hong
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| | - Feng Xie
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| | - Han Yuan
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Mingxuan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Daniel M Buckland
- Department of Emergency Medicine, Duke University School of Medicine, Durham, NC, USA
| | - Yong Chen
- Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, PA, USA
| | - Nan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore; Institute of Data Science, National University of Singapore, Singapore, Singapore.
| |
Collapse
|
9
|
Xu J, Yan X, Figueroa C, Williams JJ, Chakraborty B. A flexible micro-randomized trial design and sample size considerations. Stat Methods Med Res 2023; 32:1766-1783. [PMID: 37491804 DOI: 10.1177/09622802231188513] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/27/2023]
Abstract
Technological advancements have made it possible to deliver mobile health interventions to individuals. A novel framework that has emerged from such advancements is the just-in-time adaptive intervention, which aims to suggest the right support to the individuals when their needs arise. The micro-randomized trial design has been proposed recently to test the proximal effects of the components of these just-in-time adaptive interventions. However, the extant micro-randomized trial framework only considers components with a fixed number of categories added at the beginning of the study. We propose a more flexible micro-randomized trial design which allows addition of more categories to the components during the study. Note that the number and timing of the categories added during the study need to be fixed initially. The proposed design is motivated by collaboration on the Diabetes and Mental Health Adaptive Notification Tracking and Evaluation study, which learns to deliver effective text messages to encourage physical activity among patients with diabetes and depression. We developed a new test statistic and the corresponding sample size calculator for the flexible micro-randomized trial using an approach similar to the generalized estimating equation for longitudinal data. Simulation studies were conducted to evaluate the sample size calculators and an R shiny application for the calculators was developed.
Collapse
Affiliation(s)
- Jing Xu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| | - Xiaoxi Yan
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
| | - Caroline Figueroa
- Faculty of Technology, Policy and Management, Delft University of Technology, The Netherlands
- School of Social Welfare, University of California, Berkeley, USA
| | - Joseph Jay Williams
- Department of Computer Science, University of Toronto, ON, Canada
- Department of Statistical Sciences, University of Toronto, ON, Canada
- Department of Psychology, University of Toronto, ON, Canada
- Vector Institute for Artificial Intelligence Faculty Affiliate, University of Toronto, ON, Canada
- Department of Mechanical and Industrial Engineering, University of Toronto, ON, Canada
- Department of Economics, University of Toronto, ON, Canada
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
- Department of Statistics and Data Science, National University of Singapore, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| |
Collapse
|
10
|
Nisa CF, Yan X, Chakraborty B, Leander P, Bélanger JJ. COVID-19 may have increased global support for universal health coverage: multi-country observational study. Front Public Health 2023; 11:1213037. [PMID: 37693708 PMCID: PMC10486985 DOI: 10.3389/fpubh.2023.1213037] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/27/2023] [Accepted: 08/04/2023] [Indexed: 09/12/2023] Open
Abstract
Introduction The multiple risks generated by the COVID-19 pandemic intensified the debate about healthcare access and coverage. Whether the burden of disease caused by the coronavirus outbreak changed public opinion about healthcare provision remains unclear. In this study, it was specifically examined if the pandemic changed support for governmental intervention in healthcare as a proxy to support for universal health coverage (UHC). It also examined which psychological factors related to the socioeconomic interdependence exposed by the pandemic may be associated with a potential change. Methods Online survey data was collected over 18 months (from March 2020 to August 2021) across 73 countries, containing various social attitudes and risk perceptions related to COVID-19. This was a convenience sample composed of voluntary participants (N = 3,176; age 18 years and above). Results The results show that support for government intervention in healthcare increased across geographical regions, age groups, and gender groups (an average increase of 39%), more than the support for government intervention in other social welfare issues. Factors related to socioeconomic interdependence predicted increased support for government intervention in healthcare, namely, social solidarity (ß = 0.14, p < 0.0001), and risk to economic livelihood (ß = 0.09, p < 0.0001). Trust in the government to deal with COVID-19 decreased over time, and this negative trajectory predicted a demand for better future government intervention in healthcare (ß = -0.10, p = 0.0003). Conclusion The COVID-19 pandemic may have been a potential turning point in the global public support for UHC, as evidenced by a higher level of consensus that governments should be guarantors of healthcare.
Collapse
Affiliation(s)
- Claudia F. Nisa
- Division of Social Sciences, Duke Kunshan University, Kunshan, China
- Global Health Research Center, Duke Kunshan University, Kunshan, China
- New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
| | - Xiaoxi Yan
- Duke-NUS Medical School, Singapore, Singapore
| | | | - Pontus Leander
- College of Liberal Arts and Sciences, Wayne State University, Detroit, MI, United States
| | - Jocelyn J. Bélanger
- New York University Abu Dhabi, Abu Dhabi, United Arab Emirates
- Carnegie-Mellon University Qatar, Qatar Education City, Doha, Qatar
| |
Collapse
|
11
|
Liu M, Li S, Yuan H, Ong MEH, Ning Y, Xie F, Saffari SE, Shang Y, Volovici V, Chakraborty B, Liu N. Handling missing values in healthcare data: A systematic review of deep learning-based imputation techniques. Artif Intell Med 2023; 142:102587. [PMID: 37316097 DOI: 10.1016/j.artmed.2023.102587] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Revised: 04/08/2023] [Accepted: 05/16/2023] [Indexed: 06/16/2023]
Abstract
OBJECTIVE The proper handling of missing values is critical to delivering reliable estimates and decisions, especially in high-stakes fields such as clinical research. In response to the increasing diversity and complexity of data, many researchers have developed deep learning (DL)-based imputation techniques. We conducted a systematic review to evaluate the use of these techniques, with a particular focus on the types of data, intending to assist healthcare researchers from various disciplines in dealing with missing data. MATERIALS AND METHODS We searched five databases (MEDLINE, Web of Science, Embase, CINAHL, and Scopus) for articles published prior to February 8, 2023 that described the use of DL-based models for imputation. We examined selected articles from four perspectives: data types, model backbones (i.e., main architectures), imputation strategies, and comparisons with non-DL-based methods. Based on data types, we created an evidence map to illustrate the adoption of DL models. RESULTS Out of 1822 articles, a total of 111 were included, of which tabular static data (29%, 32/111) and temporal data (40%, 44/111) were the most frequently investigated. Our findings revealed a discernible pattern in the choice of model backbones and data types, for example, the dominance of autoencoder and recurrent neural networks for tabular temporal data. The discrepancy in imputation strategy usage among data types was also observed. The "integrated" imputation strategy, which solves the imputation task simultaneously with downstream tasks, was most popular for tabular temporal data (52%, 23/44) and multi-modal data (56%, 5/9). Moreover, DL-based imputation methods yielded a higher level of imputation accuracy than non-DL methods in most studies. CONCLUSION The DL-based imputation models are a family of techniques, with diverse network structures. Their designation in healthcare is usually tailored to data types with different characteristics. Although DL-based imputation models may not be superior to conventional approaches across all datasets, it is highly possible for them to achieve satisfactory results for a particular data type or dataset. There are, however, still issues with regard to portability, interpretability, and fairness associated with current DL-based imputation models.
Collapse
Affiliation(s)
- Mingxuan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
| | - Siqi Li
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
| | - Han Yuan
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore; Department of Emergency Medicine, Singapore General Hospital, Singapore
| | - Yilin Ning
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
| | - Feng Xie
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore
| | - Seyed Ehsan Saffari
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore
| | - Yuqing Shang
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
| | - Victor Volovici
- Department of Neurosurgery, Erasmus MC University Medical Center, Rotterdam, the Netherlands
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore; Department of Statistics and Data Science, National University of Singapore, Singapore; Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| | - Nan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore; SingHealth AI Office, Singapore Health Services, Singapore; Institute of Data Science, National University of Singapore, Singapore.
| |
Collapse
|
12
|
Yap J, J Dziak J, Maiti R, Lynch K, McKay JR, Chakraborty B, Nahum-Shani I. Sample size estimation for comparing dynamic treatment regimens in a SMART: A Monte Carlo-based approach and case study with longitudinal overdispersed count outcomes. Stat Methods Med Res 2023; 32:1267-1283. [PMID: 37167008 PMCID: PMC10520220 DOI: 10.1177/09622802231167435] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/12/2023]
Abstract
Dynamic treatment regimens (DTRs), also known as treatment algorithms or adaptive interventions, play an increasingly important role in many health domains. DTRs are motivated to address the unique and changing needs of individuals by delivering the type of treatment needed, when needed, while minimizing unnecessary treatment. Practically, a DTR is a sequence of decision rules that specify, for each of several points in time, how available information about the individual's status and progress should be used in practice to decide which treatment (e.g. type or intensity) to deliver. The sequential multiple assignment randomized trial (SMART) is an experimental design widely used to empirically inform the development of DTRs. Sample size planning resources for SMARTs have been developed for continuous, binary, and survival outcomes. However, an important gap exists in sample size estimation methodology for SMARTs with longitudinal count outcomes. Furthermore, in many health domains, count data are overdispersed-having variance greater than their mean. We propose a Monte Carlo-based approach to sample size estimation applicable to many types of longitudinal outcomes and provide a case study with longitudinal overdispersed count outcomes. A SMART for engaging alcohol and cocaine-dependent patients in treatment is used as motivation.
Collapse
Affiliation(s)
- Jamie Yap
- Institute for Social Research, University of Michigan, Ann Arbor, MI, USA
| | - John J Dziak
- Institute for Health Research and Policy, University of Illinois Chicago, Chicago, IL, USA
| | - Raju Maiti
- Economic Research Unit, Indian Statistical Institute, Kolkata, West Bengal, India
| | - Kevin Lynch
- Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - James R McKay
- Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
- Department of Statistics and Data Science, National University of Singapore, Singapore
- Department of Statistics and Bioinformatics, Duke University, Durnham, NC, USA
| | - Inbal Nahum-Shani
- Institute for Social Research, University of Michigan, Ann Arbor, MI, USA
| |
Collapse
|
13
|
Leong U, Chakraborty B. Participant Engagement in Microrandomized Trials of mHealth Interventions: Scoping Review. JMIR Mhealth Uhealth 2023; 11:e44685. [PMID: 37213178 DOI: 10.2196/44685] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Revised: 02/20/2023] [Accepted: 03/31/2023] [Indexed: 05/23/2023] Open
Abstract
BACKGROUND Microrandomized trials (MRTs) have emerged as the gold standard for the development and evaluation of multicomponent, adaptive mobile health (mHealth) interventions. However, not much is known about the state of participant engagement measurement in MRTs of mHealth interventions. OBJECTIVE In this scoping review, we aimed to quantify the proportion of existing or planned MRTs of mHealth interventions to date that have assessed (or have planned to assess) engagement. In addition, for the trials that have explicitly assessed (or have planned to assess) engagement, we aimed to investigate how engagement has been operationalized and to identify the factors that have been studied as determinants of engagement in MRTs of mHealth interventions. METHODS We conducted a broad search for MRTs of mHealth interventions in 5 databases and manually searched preprint servers and trial registries. Study characteristics of each included evidence source were extracted. We coded and categorized these data to identify how engagement has been operationalized and which determinants, moderators, and covariates have been assessed in existing MRTs. RESULTS Our database and manual search yielded 22 eligible evidence sources. Most of these studies (14/22, 64%) were designed to evaluate the effects of intervention components. The median sample size of the included MRTs was 110.5. At least 1 explicit measure of engagement was included in 91% (20/22) of the included MRTs. We found that objective measures such as system usage data (16/20, 80%) and sensor data (7/20, 35%) are the most common methods of measuring engagement. All studies included at least 1 measure of the physical facet of engagement, but the affective and cognitive facets of engagement have largely been neglected (only measured by 1 study each). Most studies measured engagement with the mHealth intervention (Little e) and not with the health behavior of interest (Big E). Only 6 (30%) of the 20 studies that measured engagement assessed the determinants of engagement in MRTs of mHealth interventions; notification-related variables were the most common determinants of engagement assessed (4/6, 67% studies). Of the 6 studies, 3 (50%) examined the moderators of participant engagement-2 studies investigated time-related moderators exclusively, and 1 study planned to investigate a comprehensive set of physiological and psychosocial moderators in addition to time-related moderators. CONCLUSIONS Although the measurement of participant engagement in MRTs of mHealth interventions is prevalent, there is a need for future trials to diversify the measurement of engagement. There is also a need for researchers to address the lack of attention to how engagement is determined and moderated. We hope that by mapping the state of engagement measurement in existing MRTs of mHealth interventions, this review will encourage researchers to pay more attention to these issues when planning for engagement measurement in future trials.
Collapse
Affiliation(s)
- Utek Leong
- Department of Psychology, National University of Singapore, Singapore, Singapore
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine and Program in Health Services and Systems Research, Duke-NUS Medical School, National University of Singapore, Singapore, Singapore
- Department of Statistics and Data Science, National University of Singapore, Singapore, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States
| |
Collapse
|
14
|
Xie F, Ning Y, Liu M, Li S, Saffari SE, Yuan H, Volovici V, Ting DSW, Goldstein BA, Ong MEH, Vaughan R, Chakraborty B, Liu N. A universal AutoScore framework to develop interpretable scoring systems for predicting common types of clinical outcomes. STAR Protoc 2023; 4:102302. [PMID: 37178115 DOI: 10.1016/j.xpro.2023.102302] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2023] [Revised: 03/13/2023] [Accepted: 04/21/2023] [Indexed: 05/15/2023] Open
Abstract
The AutoScore framework can automatically generate data-driven clinical scores in various clinical applications. Here, we present a protocol for developing clinical scoring systems for binary, survival, and ordinal outcomes using the open-source AutoScore package. We describe steps for package installation, detailed data processing and checking, and variable ranking. We then explain how to iterate through steps for variable selection, score generation, fine-tuning, and evaluation to generate understandable and explainable scoring systems using data-driven evidence and clinical knowledge. For complete details on the use and execution of this protocol, please refer to Xie et al. (2020),1 Xie et al. (2022)2, Saffari et al. (2022)3 and the online tutorial https://nliulab.github.io/AutoScore/.
Collapse
Affiliation(s)
- Feng Xie
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Yilin Ning
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Mingxuan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Siqi Li
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Seyed Ehsan Saffari
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Han Yuan
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Victor Volovici
- Department of Neurosurgery, Erasmus MC University Medical Center, 3015 GD Rotterdam, the Netherlands; Department of Public Health, Erasmus MC, 3015 GD Rotterdam, the Netherlands
| | - Daniel Shu Wei Ting
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore; Singapore Eye Research Institute, Singapore National Eye Centre, Singapore 168751, Singapore; SingHealth AI Office, Singapore Health Services, Singapore 168582, Singapore
| | - Benjamin Alan Goldstein
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore 169857, Singapore; Department of Biostatistics and Bioinformatics, Duke University, Durham, NC 27710, USA
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore 169857, Singapore; Health Services Research Centre, Singapore Health Services, Singapore 169856, Singapore; Department of Emergency Medicine, Singapore General Hospital, Singapore 169608, Singapore
| | - Roger Vaughan
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore 169857, Singapore; Department of Biostatistics and Bioinformatics, Duke University, Durham, NC 27710, USA; Department of Statistics and Data Science, National University of Singapore, Singapore 117546, Singapore
| | - Nan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore 169857, Singapore; Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore 169857, Singapore; SingHealth AI Office, Singapore Health Services, Singapore 168582, Singapore; Institute of Data Science, National University of Singapore, Singapore 117602, Singapore.
| |
Collapse
|
15
|
Ghosh P, Yan X, Chakraborty B. A novel approach to assess dynamic treatment regimes embedded in a SMART with an ordinal outcome. Stat Med 2023; 42:1096-1111. [PMID: 36726310 DOI: 10.1002/sim.9659] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2022] [Revised: 10/21/2022] [Accepted: 01/04/2023] [Indexed: 02/03/2023]
Abstract
Sequential multiple assignment randomized trials (SMARTs) are used to construct data-driven optimal intervention strategies for subjects based on their intervention and covariate histories in different branches of health and behavioral sciences where a sequence of interventions is given to a participant. Sequential intervention strategies are often called dynamic treatment regimes (DTR). In the existing literature, the majority of the analysis methodologies for SMART data assume a continuous primary outcome. However, ordinal outcomes are also quite common in clinical practice. In this work, first, we introduce the notion of generalized odds ratio ( G O R $$ GOR $$ ) to compare two DTRs embedded in a SMART with an ordinal outcome and discuss some combinatorial properties of this measure. Next, we propose a likelihood-based approach to estimate G O R $$ GOR $$ from SMART data, and derive the asymptotic properties of its estimate. We discuss alternative ways to estimate G O R $$ GOR $$ using concordant-discordant pairs and two-sample U $$ U $$ -statistic. We derive the required sample size formula for designing SMARTs with ordinal outcomes based on G O R $$ GOR $$ . A simulation study shows the performance of the estimated G O R $$ GOR $$ in terms of the estimated power corresponding to the derived sample size. The methodology is applied to analyze data from the SMART+ study, conducted in the UK, to improve carbohydrate periodization behavior in athletes using a menu planner mobile application, Hexis Performance. A freely available Shiny web app using R is provided to make the proposed methodology accessible to other researchers and practitioners.
Collapse
Affiliation(s)
- Palash Ghosh
- Department of Mathematics, Indian Institute of Technology Guwahati, Assam, India.,Jyoti and Bhupat Mehta School of Health Sciences and Technology, Indian Institute of Technology Guwahati, Assam, India.,Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Xiaoxi Yan
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore.,Department of Statistics and Data Science, National University of Singapore, Singapore, Singapore.,Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina
| |
Collapse
|
16
|
Maiti R, Li J, Das P, Liu X, Feng L, Hausenloy DJ, Chakraborty B. A distribution-free smoothed combination method to improve discrimination accuracy in multi-category classification. Stat Methods Med Res 2023; 32:242-266. [PMID: 36384309 DOI: 10.1177/09622802221137742] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Results from multiple diagnostic tests are combined in many ways to improve the overall diagnostic accuracy. For binary classification, maximization of the empirical estimate of the area under the receiver operating characteristic curve has widely been used to produce an optimal linear combination of multiple biomarkers. However, in the presence of a large number of biomarkers, this method proves to be computationally expensive and difficult to implement since it involves maximization of a discontinuous, non-smooth function for which gradient-based methods cannot be used directly. The complexity of this problem further increases when the classification problem becomes multi-category. In this article, we develop a linear combination method that maximizes a smooth approximation of the empirical Hyper-volume Under Manifolds for the multi-category outcome. We approximate HUM by replacing the indicator function with the sigmoid function and normal cumulative distribution function. With such smooth approximations, efficient gradient-based algorithms are employed to obtain better solutions with less computing time. We show that under some regularity conditions, the proposed method yields consistent estimates of the coefficient parameters. We derive the asymptotic normality of the coefficient estimates. A simulation study is performed to study the effectiveness of our proposed method as compared to other existing methods. The method is illustrated using two real medical data sets.
Collapse
Affiliation(s)
- Raju Maiti
- Economic Research Unit, Indian Statistical Institute Kolkata, Kolkata, India
| | - Jialiang Li
- Department of Statistics and Data Science, National University of Singapore, Singapore, Singapore
| | - Priyam Das
- Department of Biomedical Informatics, 1811Harvard Medical School, Boston, MA, USA
| | - Xueqing Liu
- Centre for Quantitative Medicine, 121579Duke-NUS Medical School, Singapore, Singapore
| | - Lei Feng
- Department of Psychological Medicine, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Derek J Hausenloy
- Cardiovascular and Metabolic Disorders Program, 121579Duke-NUS Medical School, Singapore, Singapore.,National Heart Research Institute Singapore, National Heart Centre, Singapore, Singapore.,Yong Loo Lin School of Medicine, National University Singapore, Singapore, Singapore.,The Hatter Cardiovascular Institute, University College London, London, UK.,Cardiovascular Research Center, College of Medical and Health Sciences, Asia University, Taichung
| | - Bibhas Chakraborty
- Department of Statistics and Data Science, National University of Singapore, Singapore, Singapore.,Centre for Quantitative Medicine, 121579Duke-NUS Medical School, Singapore, Singapore.,Department of Biostatistics and Bioinformatics, Duke University, USA
| |
Collapse
|
17
|
Wang X, Chakraborty B. The Sequential Multiple Assignment Randomized Trial for Controlling Infectious Diseases: A Review of Recent Developments. Am J Public Health 2023; 113:49-59. [PMID: 36516390 PMCID: PMC9755933 DOI: 10.2105/ajph.2022.307135] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
Infectious diseases have posed severe threats to public health across the world. Effective prevention and control of infectious diseases in the long term requires adapting interventions based on epidemiological evidence. The sequential multiple assignment randomized trial (SMART) is a multistage randomized trial that can provide valid evidence of when and how to adapt interventions for controlling infectious diseases based on evolving epidemiological evidence. We review recent developments in SMARTs to bring wider attention to the potential benefits of employing SMARTs in constructing effective adaptive interventions for controlling infectious diseases and other threats to public health. We discuss 2 example SMARTs for infectious diseases and summarize recent developments in SMARTs from the varied aspects of design, analysis, cost, and ethics. Public health investigators are encouraged to familiarize themselves with the related materials we discuss and collaborate with experts in SMARTs to translate the methodological developments into preeminent public health research. (Am J Public Health. 2023;113(1):49-59. https://doi.org/10.2105/AJPH.2022.307135).
Collapse
Affiliation(s)
- Xinru Wang
- Xinru Wang and Bibhas Chakraborty are with the Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore. Bibhas Chakraborty is also with the Department of Statistics and Data Science, National University of Singapore, Singapore
| | - Bibhas Chakraborty
- Xinru Wang and Bibhas Chakraborty are with the Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore. Bibhas Chakraborty is also with the Department of Statistics and Data Science, National University of Singapore, Singapore
| |
Collapse
|
18
|
Liu X, Deliu N, Chakraborty B. Microrandomized Trials: Developing Just-in-Time Adaptive Interventions for Better Public Health. Am J Public Health 2023; 113:60-69. [PMID: 36413704 PMCID: PMC9755932 DOI: 10.2105/ajph.2022.307150] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
Just-in-time adaptive interventions (JITAIs) represent an intervention design that adapts the provision and type of support over time to an individual's changing status and contexts, intending to deliver the right support on the right occasion. As a novel strategy for delivering mobile health interventions, JITAIs have the potential to improve access to quality care in underserved communities and, thus, alleviate health disparities, a significant public health concern. Valid experimental designs and analysis methods are required to inform the development of JITAIs. Here, we briefly review the cutting-edge design of microrandomized trials (MRTs), covering both the classical MRT design and its outcome-adaptive counterpart. Associated statistical challenges related to the design and analysis of MRTs are also discussed. Two case studies are provided to illustrate the aforementioned concepts and designs throughout the article. We hope our work leads to better design and application of JITAIs, advancing public health research and practice. (Am J Public Health. 2023;113(1):60-69. https://doi.org/10.2105/AJPH.2022.307150).
Collapse
Affiliation(s)
- Xueqing Liu
- Xueqing Liu is with the Centre for Quantitative Medicine, Duke-National University of Singapore (NUS) Medical School, Singapore. Nina Deliu is with the Medical Research Council Biostatistics Unit, University of Cambridge, UK, and the Department of Methods and Models for Economics, Territory and Finance, Sapienza University of Rome, Italy. Bibhas Chakraborty is with the Centre for Quantitative Medicine and Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore; the Department of Statistics and Data Science, NUS, Singapore; and the Department of Biostatistics and Bioinformatics, Duke University, Durham, NC
| | - Nina Deliu
- Xueqing Liu is with the Centre for Quantitative Medicine, Duke-National University of Singapore (NUS) Medical School, Singapore. Nina Deliu is with the Medical Research Council Biostatistics Unit, University of Cambridge, UK, and the Department of Methods and Models for Economics, Territory and Finance, Sapienza University of Rome, Italy. Bibhas Chakraborty is with the Centre for Quantitative Medicine and Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore; the Department of Statistics and Data Science, NUS, Singapore; and the Department of Biostatistics and Bioinformatics, Duke University, Durham, NC
| | - Bibhas Chakraborty
- Xueqing Liu is with the Centre for Quantitative Medicine, Duke-National University of Singapore (NUS) Medical School, Singapore. Nina Deliu is with the Medical Research Council Biostatistics Unit, University of Cambridge, UK, and the Department of Methods and Models for Economics, Territory and Finance, Sapienza University of Rome, Italy. Bibhas Chakraborty is with the Centre for Quantitative Medicine and Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore; the Department of Statistics and Data Science, NUS, Singapore; and the Department of Biostatistics and Bioinformatics, Duke University, Durham, NC
| |
Collapse
|
19
|
Sung SC, Lim L, Lim SH, Finkelstein EA, Chin SLH, Annathurai A, Chakraborty B, Strauman TJ, Pollack MH, Ong MEH. Protocol for a multi-site randomized controlled trial of a stepped-care intervention for emergency department patients with panic-related anxiety. BMC Psychiatry 2022; 22:795. [PMID: 36527018 PMCID: PMC9756520 DOI: 10.1186/s12888-022-04387-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Accepted: 11/11/2022] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Approximately 40% of Emergency Department (ED) patients with chest pain meet diagnostic criteria for panic-related anxiety, but only 1-2% are correctly diagnosed and appropriately managed in the ED. A stepped-care model, which focuses on providing evidence-based interventions in a resource-efficient manner, is the state-of-the art for treating panic disorder patients in medical settings such as primary care. Stepped-care has yet to be tested in the ED setting, which is the first point of contact with the healthcare system for most patients with panic symptoms. METHODS This multi-site randomized controlled trial (RCT) aims to evaluate the clinical, patient-centred, and economic effectiveness of a stepped-care intervention in a sample of 212 patients with panic-related anxiety presenting to the ED of Singapore's largest public healthcare group. Participants will be randomly assigned to either: 1) an enhanced care arm consisting of a stepped-care intervention for panic-related anxiety; or 2) a control arm consisting of screening for panic attacks and panic disorder. Screening will be followed by baseline assessments and blocked randomization in a 1:1 ratio. Masked follow-up assessments will be conducted at 1, 3, 6, and 12 months. Clinical outcomes will be panic symptom severity and rates of panic disorder. Patient-centred outcomes will be health-related quality of life, daily functioning, psychiatric comorbidity, and health services utilization. Economic effectiveness outcomes will be the incremental cost-effectiveness ratio of the stepped-care intervention relative to screening alone. DISCUSSION This trial will examine the impact of early intervention for patients with panic-related anxiety in the ED setting. The results will be used to propose a clinically-meaningful and cost-effective model of care for ED patients with panic-related anxiety. TRIAL REGISTRATION ClinicalTrials.gov NCT03632356. Retrospectively registered 15 August 2018.
Collapse
Affiliation(s)
- Sharon C. Sung
- grid.428397.30000 0004 0385 0924Duke-NUS Medical School Singapore, 8 College Road, Singapore, 169857 Singapore
| | - Leslie Lim
- grid.163555.10000 0000 9486 5048Singapore General Hospital, Outram Road, Singapore, 169608 Singapore
| | - Swee Han Lim
- grid.163555.10000 0000 9486 5048Singapore General Hospital, Outram Road, Singapore, 169608 Singapore
| | - Eric A. Finkelstein
- grid.428397.30000 0004 0385 0924Duke-NUS Medical School Singapore, 8 College Road, Singapore, 169857 Singapore
| | - Steven Lim Hoon Chin
- grid.413815.a0000 0004 0469 9373Changi General Hospital, 2 Simei Street 3, Singapore, 529889 Singapore
| | - Annitha Annathurai
- grid.508163.90000 0004 7665 4668Sengkang General Hospital, 110 Sengkang E Way, Singapore, 544886 Singapore
| | - Bibhas Chakraborty
- grid.428397.30000 0004 0385 0924Duke-NUS Medical School Singapore, 8 College Road, Singapore, 169857 Singapore ,grid.4280.e0000 0001 2180 6431National University of Singapore, 6 Science Drive 2, Singapore, 117546 Singapore ,grid.26009.3d0000 0004 1936 7961Duke University, 2424 Erwin Road, Suite 1102, Durham, NC 27710 USA
| | - Timothy J. Strauman
- grid.189509.c0000000100241216Duke University Medical Center, 10 Duke Medicine Cir, Durham, NC 27710 USA
| | - Mark H. Pollack
- grid.240684.c0000 0001 0705 3621Rush University Medical Center, 1645 W. Jackson Blvd, Suite 400, Chicago, IL 60612 USA ,grid.476678.c0000 0004 5913 664XSage Therapeutics, 215 First Street, Cambridge, MA 02142 USA
| | - Marcus Eng Hock Ong
- grid.428397.30000 0004 0385 0924Duke-NUS Medical School Singapore, 8 College Road, Singapore, 169857 Singapore ,grid.163555.10000 0000 9486 5048Singapore General Hospital, Outram Road, Singapore, 169608 Singapore
| |
Collapse
|
20
|
Saffari SE, Ning Y, Xie F, Chakraborty B, Volovici V, Vaughan R, Ong MEH, Liu N. AutoScore-Ordinal: an interpretable machine learning framework for generating scoring models for ordinal outcomes. BMC Med Res Methodol 2022; 22:286. [PMID: 36333672 PMCID: PMC9636613 DOI: 10.1186/s12874-022-01770-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2022] [Accepted: 10/25/2022] [Indexed: 11/06/2022] Open
Abstract
Background Risk prediction models are useful tools in clinical decision-making which help with risk stratification and resource allocations and may lead to a better health care for patients. AutoScore is a machine learning–based automatic clinical score generator for binary outcomes. This study aims to expand the AutoScore framework to provide a tool for interpretable risk prediction for ordinal outcomes. Methods The AutoScore-Ordinal framework is generated using the same 6 modules of the original AutoScore algorithm including variable ranking, variable transformation, score derivation (from proportional odds models), model selection, score fine-tuning, and model evaluation. To illustrate the AutoScore-Ordinal performance, the method was conducted on electronic health records data from the emergency department at Singapore General Hospital over 2008 to 2017. The model was trained on 70% of the data, validated on 10% and tested on the remaining 20%. Results This study included 445,989 inpatient cases, where the distribution of the ordinal outcome was 80.7% alive without 30-day readmission, 12.5% alive with 30-day readmission, and 6.8% died inpatient or by day 30 post discharge. Two point-based risk prediction models were developed using two sets of 8 predictor variables identified by the flexible variable selection procedure. The two models indicated reasonably good performance measured by mean area under the receiver operating characteristic curve (0.758 and 0.793) and generalized c-index (0.737 and 0.760), which were comparable to alternative models. Conclusion AutoScore-Ordinal provides an automated and easy-to-use framework for development and validation of risk prediction models for ordinal outcomes, which can systematically identify potential predictors from high-dimensional data.
Collapse
|
21
|
Das P, De D, Maiti R, Kamal M, Hutcheson KA, Fuller CD, Chakraborty B, Peterson CB. Estimating the optimal linear combination of predictors using spherically constrained optimization. BMC Bioinformatics 2022; 23:436. [PMID: 36261805 PMCID: PMC9583504 DOI: 10.1186/s12859-022-04953-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2022] [Accepted: 09/19/2022] [Indexed: 01/01/2023] Open
Abstract
BACKGROUND In the context of a binary classification problem, the optimal linear combination of continuous predictors can be estimated by maximizing the area under the receiver operating characteristic curve. For ordinal responses, the optimal predictor combination can similarly be obtained by maximization of the hypervolume under the manifold (HUM). Since the empirical HUM is discontinuous, non-differentiable, and possibly multi-modal, solving this maximization problem requires a global optimization technique. Estimation of the optimal coefficient vector using existing global optimization techniques is computationally expensive, becoming prohibitive as the number of predictors and the number of outcome categories increases. RESULTS We propose an efficient derivative-free black-box optimization technique based on pattern search to solve this problem, which we refer to as Spherically Constrained Optimization Routine (SCOR). Through extensive simulation studies, we demonstrate that the proposed method achieves better performance than existing methods including the step-down algorithm. Finally, we illustrate the proposed method to predict the severity of swallowing difficulty after radiation therapy for oropharyngeal cancer based on radiation dose to various structures in the head and neck. CONCLUSIONS Our proposed method addresses an important challenge in combining multiple biomarkers to predict an ordinal outcome. This problem is particularly relevant to medical research, where it may be of interest to diagnose a disease with various stages of progression or a toxicity with multiple grades of severity. We provide the implementation of our proposed SCOR method as an R package, available online at https://CRAN.R-project.org/package=SCOR .
Collapse
Affiliation(s)
- Priyam Das
- grid.38142.3c000000041936754XDepartment of Biomedical Informatics, Harvard Medical School, Boston, MA USA
| | - Debsurya De
- grid.39953.350000 0001 2157 0617Indian Statistical Institute, Kolkata, India
| | - Raju Maiti
- grid.428397.30000 0004 0385 0924Centre for Quantitative Medicine, Duke-National University of Singapore Medical School, Singapore, Singapore
| | - Mona Kamal
- grid.240145.60000 0001 2291 4776Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Katherine A. Hutcheson
- grid.240145.60000 0001 2291 4776Department of Head and Neck Surgery, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Clifton D. Fuller
- grid.240145.60000 0001 2291 4776Department of Radiation Oncology, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| | - Bibhas Chakraborty
- grid.428397.30000 0004 0385 0924Centre for Quantitative Medicine, Duke-National University of Singapore Medical School, Singapore, Singapore ,grid.4280.e0000 0001 2180 6431Department of Statistics and Applied Probability, National University of Singapore, Singapore, Singapore ,grid.26009.3d0000 0004 1936 7961Department of Biostatistics and Bioinformatics, Duke University, Durham, NC USA
| | - Christine B. Peterson
- grid.240145.60000 0001 2291 4776Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston, TX USA
| |
Collapse
|
22
|
Jha PK, Kumar A, Kushwaha SK, Chakraborty B. Investigation of Thermal and Hydraulic Regime of CDCP for Higher Throughput in SAIL Plant. Coke Chem 2022. [DOI: 10.3103/s1068364x22700193] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/09/2023]
|
23
|
Ghosh B, Sahoo BK, Jha PK, Kushwaha SK, Chakraborty B, Manjhi KK. Understanding the Impact of Coal Blend Properties on the Coke Strength. Coke Chem 2022. [DOI: 10.3103/s1068364x22070043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
24
|
Ning Y, Li S, Ong MEH, Xie F, Chakraborty B, Ting DSW, Liu N. A novel interpretable machine learning system to generate clinical risk scores: An application for predicting early mortality or unplanned readmission in a retrospective cohort study. PLOS Digit Health 2022; 1:e0000062. [PMID: 36812536 PMCID: PMC9931273 DOI: 10.1371/journal.pdig.0000062] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 02/08/2022] [Accepted: 05/10/2022] [Indexed: 01/19/2023]
Abstract
Risk scores are widely used for clinical decision making and commonly generated from logistic regression models. Machine-learning-based methods may work well for identifying important predictors to create parsimonious scores, but such 'black box' variable selection limits interpretability, and variable importance evaluated from a single model can be biased. We propose a robust and interpretable variable selection approach using the recently developed Shapley variable importance cloud (ShapleyVIC) that accounts for variability in variable importance across models. Our approach evaluates and visualizes overall variable contributions for in-depth inference and transparent variable selection, and filters out non-significant contributors to simplify model building steps. We derive an ensemble variable ranking from variable contributions across models, which is easily integrated with an automated and modularized risk score generator, AutoScore, for convenient implementation. In a study of early death or unplanned readmission after hospital discharge, ShapleyVIC selected 6 variables from 41 candidates to create a well-performing risk score, which had similar performance to a 16-variable model from machine-learning-based ranking. Our work contributes to the recent emphasis on interpretability of prediction models for high-stakes decision making, providing a disciplined solution to detailed assessment of variable importance and transparent development of parsimonious clinical risk scores.
Collapse
Affiliation(s)
- Yilin Ning
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Siqi Li
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore,Health Services Research Centre, Singapore Health Services, Singapore, Singapore,Department of Emergency Medicine, Singapore General Hospital, Singapore, Singapore
| | - Feng Xie
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore,Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore,Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore,Department of Statistics and Data Science, National University of Singapore, Singapore, Singapore,Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States of America
| | - Daniel Shu Wei Ting
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore,Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore,SingHealth AI Health Program, Singapore Health Services, Singapore, Singapore
| | - Nan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore,Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore,Health Services Research Centre, Singapore Health Services, Singapore, Singapore,SingHealth AI Health Program, Singapore Health Services, Singapore, Singapore,Institute of Data Science, National University of Singapore, Singapore, Singapore,* E-mail:
| |
Collapse
|
25
|
Ang Y, Li S, Ong MEH, Xie F, Teo SH, Choong L, Koniman R, Chakraborty B, Ho AFW, Liu N. Development and validation of an interpretable clinical score for early identification of acute kidney injury at the emergency department. Sci Rep 2022; 12:7111. [PMID: 35501411 PMCID: PMC9061747 DOI: 10.1038/s41598-022-11129-4] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2021] [Accepted: 04/12/2022] [Indexed: 12/24/2022] Open
Abstract
Acute kidney injury (AKI) in hospitalised patients is a common syndrome associated with poorer patient outcomes. Clinical risk scores can be used for the early identification of patients at risk of AKI. We conducted a retrospective study using electronic health records of Singapore General Hospital emergency department patients who were admitted from 2008 to 2016. The primary outcome was inpatient AKI of any stage within 7 days of admission based on the Kidney Disease Improving Global Outcome (KDIGO) 2012 guidelines. A machine learning-based framework AutoScore was used to generate clinical scores from the study sample which was randomly divided into training, validation and testing cohorts. Model performance was evaluated using area under the curve (AUC). Among the 119,468 admissions, 10,693 (9.0%) developed AKI. 8491 were stage 1 (79.4%), 906 stage 2 (8.5%) and 1296 stage 3 (12.1%). The AKI Risk Score (AKI-RiSc) was a summation of the integer scores of 6 variables: serum creatinine, serum bicarbonate, pulse, systolic blood pressure, diastolic blood pressure, and age. AUC of AKI-RiSc was 0.730 (95% CI 0.714–0.747), outperforming an existing AKI Prediction Score model which achieved AUC of 0.665 (95% CI 0.646–0.679) on the testing cohort. At a cut-off of 4 points, AKI-RiSc had a sensitivity of 82.6% and specificity of 46.7%. AKI-RiSc is a simple clinical score that can be easily implemented on the ground for early identification of AKI and potentially be applied in international settings.
Collapse
|
26
|
Yuan H, Xie F, Eng Hock Ong M, Ning Y, Lucas Chee M, Ehsan Saffari S, Rizal Abdullah H, Alan Goldstein B, Chakraborty B, Liu N. AutoScore-Imbalance: An Interpretable Machine Learning Tool for Development of Clinical Scores with Rare Events Data. J Biomed Inform 2022; 129:104072. [PMID: 35421602 DOI: 10.1016/j.jbi.2022.104072] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/14/2021] [Revised: 03/10/2022] [Accepted: 04/07/2022] [Indexed: 02/06/2023]
Abstract
BACKGROUND Medical decision-making impacts both individual and public health. Clinical scores are commonly used among various decision-making models to determine the degree of disease deterioration at the bedside. AutoScore was proposed as a useful clinical score generator based on machine learning and a generalized linear model. However, its current framework still leaves room for improvement when addressing unbalanced data of rare events. METHODS Using machine intelligence approaches, we developed AutoScore-Imbalance, which comprises three components: training dataset optimization, sample weight optimization, and adjusted AutoScore. Baseline techniques for performance comparison included the original AutoScore, full logistic regression, stepwise logistic regression, least absolute shrinkage and selection operator (LASSO), full random forest, and random forest with a reduced number of variables. These models were evaluated based on their area under the curve (AUC) in the receiver operating characteristic analysis and balanced accuracy (i.e., mean value of sensitivity and specificity). By utilizing a publicly accessible dataset from Beth Israel Deaconess Medical Center, we assessed the proposed model and baseline approaches to predict inpatient mortality. RESULTS AutoScore-Imbalance outperformed baselines in terms of AUC and balanced accuracy. The nine-variable AutoScore-Imbalance sub-model achieved the highest AUC of 0.786 (0.732-0.839), while the eleven-variable original AutoScore obtained an AUC of 0.723 (0.663-0.783), and the logistic regression with 21 variables obtained an AUC of 0.743 (0.685-0.800). The AutoScore-Imbalance sub-model (using a down-sampling algorithm) yielded an AUC of 0.771 (0.718-0.823) with only five variables, demonstrating a good balance between performance and variable sparsity. Furthermore, AutoScore-Imbalance obtained the highest balanced accuracy of 0.757 (0.702-0.805), compared to 0.698 (0.643-0.753) by the original AutoScore and the maximum of 0.720 (0.664-0.769) by other baseline models. CONCLUSIONS We have developed an interpretable tool to handle clinical data imbalance, presented its structure, and demonstrated its superiority over baselines. The AutoScore-Imbalance tool can be applied to highly unbalanced datasets to gain further insight into rare medical events and facilitate real-world clinical decision-making.
Collapse
Affiliation(s)
- Han Yuan
- Duke-NUS Medical School, National University of Singapore, Singapore
| | - Feng Xie
- Duke-NUS Medical School, National University of Singapore, Singapore
| | - Marcus Eng Hock Ong
- Duke-NUS Medical School, National University of Singapore, Singapore; Department of Emergency Medicine, Singapore General Hospital, Singapore; Health Services Research Centre, Singapore Health Services, Singapore
| | - Yilin Ning
- Duke-NUS Medical School, National University of Singapore, Singapore
| | - Marcel Lucas Chee
- Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia
| | | | - Hairil Rizal Abdullah
- Duke-NUS Medical School, National University of Singapore, Singapore; Department of Anaesthesiology, Singapore General Hospital, Singapore
| | - Benjamin Alan Goldstein
- Duke-NUS Medical School, National University of Singapore, Singapore; Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States
| | - Bibhas Chakraborty
- Duke-NUS Medical School, National University of Singapore, Singapore; Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States; Department of Statistics and Data Science, National University of Singapore, Singapore
| | - Nan Liu
- Duke-NUS Medical School, National University of Singapore, Singapore; Health Services Research Centre, Singapore Health Services, Singapore; Institute of Data Science, National University of Singapore, Singapore.
| |
Collapse
|
27
|
Yan X, Dunne DM, Impey SG, Cunniffe B, Lefevre CE, Mazorra R, Morton JP, Tod D, Close GL, Murphy R, Chakraborty B. A pilot sequential multiple assignment randomized trial (SMART) protocol for developing an adaptive coaching intervention around a mobile application for athletes to improve carbohydrate periodization behavior. Contemp Clin Trials Commun 2022; 26:100899. [PMID: 35198794 PMCID: PMC8844798 DOI: 10.1016/j.conctc.2022.100899] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2021] [Revised: 12/23/2021] [Accepted: 01/30/2022] [Indexed: 11/14/2022] Open
Abstract
Background It has recently been identified that manipulating carbohydrate availability around exercise activity can enhance training-induced metabolic adaptations. Despite this approach being accepted in the athletic populations, athletes do not systematically follow the guidelines. Digital environments appear to allow nutritionists to deliver this intervention at scale, reducing expensive human coaching time. Yet, digitally delivered dietary behavior change interventions for athletes and the coaching strategy to support them are still novel concepts within sports nutrition. Methods/design We aim to recruit 900 athletes across the UK. 500 athletes will be recruited to test the feasibility of a novel menu planner mobile application with coaching for 6 weeks. 250 athletes with pre-existing nutritionist support will also be recruited as control. We will then conduct a 4-week pilot sequential multiple assignment randomized trial (SMART) with an additional 150 athletes. In the SMART, athletes will be given the application and additional coaching according to their engagement responses. The primary outcomes are the mobile application and coach uptake, retention, engagement, and success in attaining carbohydrate periodization behavior. Secondary outcomes are changes in goal, weight, carbohydrate periodization self-efficacy, and beliefs about consequences. Due to the high attrition nature of digital interventions, all quantitative analyses will be carried out based on both the intention-to-treat and per-protocol principles. Discussion This study will be the first to investigate improving carbohydrate periodization using a digital approach and tailored coaching strategies under this context. Foundational evidence from this study will provide insights into the feasibility of the digital approach.
Collapse
|
28
|
Ning Y, Ong MEH, Chakraborty B, Goldstein BA, Ting DSW, Vaughan R, Liu N. Shapley variable importance cloud for interpretable machine learning. Patterns 2022; 3:100452. [PMID: 35465224 PMCID: PMC9023900 DOI: 10.1016/j.patter.2022.100452] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/19/2021] [Revised: 12/28/2021] [Accepted: 01/27/2022] [Indexed: 12/02/2022]
Abstract
Interpretable machine learning has been focusing on explaining final models that optimize performance. The state-of-the-art Shapley additive explanations (SHAP) locally explains the variable impact on individual predictions and has recently been extended to provide global assessments across the dataset. Our work further extends “global” assessments to a set of models that are “good enough” and are practically as relevant as the final model to a prediction task. The resulting Shapley variable importance cloud consists of Shapley-based importance measures from each good model and pools information across models to provide an overall importance measure, with uncertainty explicitly quantified to support formal statistical inference. We developed visualizations to highlight the uncertainty and to illustrate its implications to practical inference. Building on a common theoretical basis, our method seamlessly complements the widely adopted SHAP assessments of a single final model to avoid biased inference, which we demonstrate in two experiments using recidivism prediction data and clinical data. Comprehensive global variable importance assessments beyond final (optimal) models Integrates with SHAP to complement current interpretable machine learning research ShapleyVIC quantifies uncertainty in variable importance for rigorous assessments ShapleyVIC visualizes uncertainty to explore good models with specific properties
With the wide use of machine learning models in decision making, various explanation methods have been developed to help researchers understand how each variable contributes to predictions. However, the current explanation approach focuses on explaining the final (often best performing) models, ignoring the fact that in practice, researchers are willing to consider models that are “good enough” and are easier to understand and/or implement. We propose the Shapley variable importance cloud to address this practical need by extending the current explanation approach to a set of “good models,” which pools information across models to derive a more reliable measure for overall variable importance. Moreover, we analyze and visualize the uncertainty of variable importance across models, which enables rigorous statistical assessments and helps discover alternative models with preferrable properties.
Collapse
Affiliation(s)
- Yilin Ning
- Centre for Quantitative Medicine, Duke-NUS Medical School, 8 College Road, Singapore 169857, Singapore
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, Singapore 169857, Singapore
- Health Services Research Centre, Singapore Health Services, 20 College Road, Singapore 169856, Singapore
- Department of Emergency Medicine, Singapore General Hospital, 1 Hospital Crescent Outram Road, Singapore 169608, Singapore
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, 8 College Road, Singapore 169857, Singapore
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, Singapore 169857, Singapore
- Department of Statistics and Data Science, National University of Singapore, 6 Science Drive 2, Singapore 117546, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, 2424 Erwin Road, Durham, NC 27710, USA
| | - Benjamin Alan Goldstein
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, Singapore 169857, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, 2424 Erwin Road, Durham, NC 27710, USA
| | - Daniel Shu Wei Ting
- Centre for Quantitative Medicine, Duke-NUS Medical School, 8 College Road, Singapore 169857, Singapore
- Singapore Eye Research Institute, Singapore National Eye Centre, 11 Third Hospital Avenue, Singapore 168751, Singapore
- SingHealth AI Health Program, Singapore Health Services, 10 Hospital Boulevard, Singapore 168582, Singapore
| | - Roger Vaughan
- Centre for Quantitative Medicine, Duke-NUS Medical School, 8 College Road, Singapore 169857, Singapore
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, Singapore 169857, Singapore
| | - Nan Liu
- Centre for Quantitative Medicine, Duke-NUS Medical School, 8 College Road, Singapore 169857, Singapore
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, Singapore 169857, Singapore
- Health Services Research Centre, Singapore Health Services, 20 College Road, Singapore 169856, Singapore
- SingHealth AI Health Program, Singapore Health Services, 10 Hospital Boulevard, Singapore 168582, Singapore
- Institute of Data Science, National University of Singapore, 3 Research Link, Singapore 117602, Singapore
- Corresponding author
| |
Collapse
|
29
|
Liu N, Xie F, Siddiqui FJ, Ho AFW, Chakraborty B, Nadarajan GD, Tan KBK, Ong MEH. Leveraging Large-Scale Electronic Health Records and Interpretable Machine Learning for Clinical Decision Making at the Emergency Department: Protocol for System Development and Validation. JMIR Res Protoc 2022; 11:e34201. [PMID: 35333179 PMCID: PMC9492092 DOI: 10.2196/34201] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2021] [Revised: 11/29/2021] [Accepted: 11/30/2021] [Indexed: 12/23/2022] Open
Abstract
Background There is a growing demand globally for emergency department (ED) services. An increase in ED visits has resulted in overcrowding and longer waiting times. The triage process plays a crucial role in assessing and stratifying patients’ risks and ensuring that the critically ill promptly receive appropriate priority and emergency treatment. A substantial amount of research has been conducted on the use of machine learning tools to construct triage and risk prediction models; however, the black box nature of these models has limited their clinical application and interpretation. Objective In this study, we plan to develop an innovative, dynamic, and interpretable System for Emergency Risk Triage (SERT) for risk stratification in the ED by leveraging large-scale electronic health records (EHRs) and machine learning. Methods To achieve this objective, we will conduct a retrospective, single-center study based on a large, longitudinal data set obtained from the EHRs of the largest tertiary hospital in Singapore. Study outcomes include adverse events experienced by patients, such as the need for an intensive care unit and inpatient death. With preidentified candidate variables drawn from expert opinions and relevant literature, we will apply an interpretable machine learning–based AutoScore to develop 3 SERT scores. These 3 scores can be used at different times in the ED, that is, on arrival, during ED stay, and at admission. Furthermore, we will compare our novel SERT scores with established clinical scores and previously described black box machine learning models as baselines. Receiver operating characteristic analysis will be conducted on the testing cohorts for performance evaluation. Results The study is currently being conducted. The extracted data indicate approximately 1.8 million ED visits by over 810,000 unique patients. Modelling results are expected to be published in 2022. Conclusions The SERT scoring system proposed in this study will be unique and innovative because of its dynamic nature and modelling transparency. If successfully validated, our proposed solution will establish a standard for data processing and modelling by taking advantage of large-scale EHRs and interpretable machine learning tools. International Registered Report Identifier (IRRID) DERR1-10.2196/34201
Collapse
Affiliation(s)
- Nan Liu
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore.,Institute of Data Science, National University of Singapore, Singapore, Singapore.,SingHealth AI Health Program, Singapore Health Services, Singapore, Singapore.,Health Service Research Centre, Singapore Health Services, Singapore, Singapore
| | - Feng Xie
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| | - Fahad Javaid Siddiqui
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| | - Andrew Fu Wah Ho
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore.,Department of Emergency Medicine, Singapore General Hospital, Singapore, Singapore
| | - Bibhas Chakraborty
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore.,Department of Statistics and Data Science, National University of Singapore, Singapore, Singapore.,Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States
| | | | | | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore.,Health Service Research Centre, Singapore Health Services, Singapore, Singapore.,Department of Emergency Medicine, Singapore General Hospital, Singapore, Singapore
| |
Collapse
|
30
|
Sahoo BK, Ghosh B, Jha PK, Pankaj PK, Kushwaha SK, Chakraborty B, Manjhi KK, Pradhan N. Effect of Different Size Fractions on Coal Properties. Coke Chem 2022. [DOI: 10.3103/s1068364x21080068] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
31
|
Chakraborty B, Tirkey S, Mishra S, Jha PK, Pankaj PK, Kumar A, Ghosh B, Sahoo BK. Implementation of an Integrated System for Coke Oven Battery Health Monitoring at Rourkela Steel Plant. Coke Chem 2022. [DOI: 10.3103/s1068364x21050021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
32
|
Xie F, Liu N, Yan L, Ning Y, Lim KK, Gong C, Kwan YH, Ho AFW, Low LL, Chakraborty B, Ong MEH. Development and validation of an interpretable machine learning scoring tool for estimating time to emergency readmissions. EClinicalMedicine 2022; 45:101315. [PMID: 35284804 PMCID: PMC8904223 DOI: 10.1016/j.eclinm.2022.101315] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/23/2021] [Revised: 01/22/2022] [Accepted: 02/07/2022] [Indexed: 01/06/2023] Open
Abstract
BACKGROUND Emergency readmission poses an additional burden on both patients and healthcare systems. Risk stratification is the first step of transitional care interventions targeted at reducing readmission. To accurately predict the short- and intermediate-term risks of readmission and provide information for further temporal risk stratification, we developed and validated an interpretable machine learning risk scoring system. METHODS In this retrospective study, all emergency admission episodes from January 1st 2009 to December 31st 2016 at a tertiary hospital in Singapore were assessed. The primary outcome was time to emergency readmission within 90 days post discharge. The Score for Emergency ReAdmission Prediction (SERAP) tool was derived via an interpretable machine learning-based system for time-to-event outcomes. SERAP is six-variable survival score, and takes the number of emergency admissions last year, age, history of malignancy, history of renal diseases, serum creatinine level, and serum albumin level during index admission into consideration. FINDINGS A total of 293,589 ED admission episodes were finally included in the whole cohort. Among them, 203,748 episodes were included in the training cohort, 50,937 episodes in the validation cohort, and 38,904 in the testing cohort. Readmission within 90 days was documented in 80,213 (27.3%) episodes, with a median time to emergency readmission of 22 days (Interquartile range: 8-47). For different time points, the readmission rates observed in the whole cohort were 6.7% at 7 days, 10.6% at 14 days, 13.6% at 21 days, 16.4% at 30 days, and 23.0% at 60 days. In the testing cohort, the SERAP achieved an integrated area under the curve of 0.737 (95% confidence interval: 0.730-0.743). For a specific 30-day readmission prediction, SERAP outperformed the LACE index (Length of stay, Acuity of admission, Charlson comorbidity index, and Emergency department visits in past six months) and the HOSPITAL score (Hemoglobin at discharge, discharge from an Oncology service, Sodium level at discharge, Procedure during the index admission, Index Type of admission, number of Admissions during the last 12 months, and Length of stay). Besides 30-day readmission, SERAP can predict readmission rates at any time point during the 90-day period. INTERPRETATION Better performance in risk prediction was achieved by the SERAP than other existing scores, and accurate information about time to emergency readmission was generated for further temporal risk stratification and clinical decision-making. In the future, external validation studies are needed to evaluate the SERAP at different settings and assess their real-world performance. FUNDING This study was supported by the Singapore National Medical Research Council under the PULSES Center Grant, and Duke-NUS Medical School.
Collapse
Affiliation(s)
- Feng Xie
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, 169857, Singapore
| | - Nan Liu
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, 169857, Singapore
- Health Services Research Centre, Singapore Health Services, Singapore
- Institute of Data Science, National University of Singapore, Singapore
- Corresponding author at: Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, 169857, Singapore.
| | - Linxuan Yan
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, 169857, Singapore
| | - Yilin Ning
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, 169857, Singapore
| | - Ka Keat Lim
- School of Population Health and Environmental Sciences, Faculty of Life Sciences and Medicine, King's College London, London, United Kingdom
- National Institute for Health Research (NIHR) Biomedical Research Centre, Guy's and St Thomas' NHS Foundation Trust and King's College London, London, United Kingdom
| | - Changlin Gong
- Department of Internal Medicine, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China
| | - Yu Heng Kwan
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, 169857, Singapore
| | - Andrew Fu Wah Ho
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, 169857, Singapore
- Department of Emergency Medicine, Singapore General Hospital, Singapore
| | - Lian Leng Low
- Department of Family Medicine and Continuing Care, Singapore General Hospital, Singapore
- Department of Post-Acute and Continuing Care, Outram Community Hospital, Singapore
- SingHealth Duke-NUS Family Medicine Academic Clinical Program, Duke-NUS Medical School, Singapore
| | - Bibhas Chakraborty
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, 169857, Singapore
- Department of Statistics and Data Science, National University of Singapore, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, 8 College Road, 169857, Singapore
- Health Services Research Centre, Singapore Health Services, Singapore
- Department of Emergency Medicine, Singapore General Hospital, Singapore
| |
Collapse
|
33
|
Shin S, Chakraborty B, Yan X, van Dam RM, Finkelstein EA. Evaluation of Combinations of Nudging, Pricing, and Labeling Strategies to Improve Diet Quality: A Virtual Grocery Store Experiment Employing a Multiphase Optimization Strategy. Ann Behav Med 2022; 56:933-945. [PMID: 35195704 DOI: 10.1093/abm/kaab115] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
BACKGROUND Several intervention strategies have been shown to improve diet quality. However, there is limited evidence on the increase in effectiveness that may be achieved through select combinations of these strategies. PURPOSE This study aimed to identify an effective multicomponent intervention to improve diet quality of a grocery basket by applying a Multiphase Optimization Strategy framework and testing various combinations of four promising strategies using a fully functional web-based grocery store: (i) front-of-pack food labels and real-time feedback of the healthiness of the shoppers' grocery basket, (ii) a tax, (iii) ordering products by a nutritional quality score, and (iv) healthier substitute offers. METHODS We conducted a hypothetical shopping study (N = 756) with a randomized full factorial design (16 conditions) to estimate main and interaction effects of the four interventions. RESULTS The "food labels & real-time feedback" and "ordering" strategies had significantly positive main effects on overall diet quality of the shopping basket (both at p < .001). We found no effects on diet quality for the "tax" and "healthier substitute offers." None of the two-way interaction effects for different strategies on overall diet quality and nutrients were significant. CONCLUSIONS Having "food labels & real-time feedback" and "ordering" simultaneously seemed to be more effective at improving diet quality, compared to having only one of these interventions. These results suggest that a combination of food labels with real-time feedback and ordering interventions can be part of a promising multicomponent strategy to improve diet quality in online shopping platforms. TRIAL REGISTRATION ClinicalTrials.gov NCT04632212.
Collapse
Affiliation(s)
- Soye Shin
- Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore
| | - Bibhas Chakraborty
- Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore.,Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore.,Department of Statistics and Applied Probability, National University of Singapore, Singapore.,Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| | - Xiaoxi Yan
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
| | - Rob M van Dam
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore
| | - Eric A Finkelstein
- Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore.,Saw Swee Hock School of Public Health, National University of Singapore, Singapore
| |
Collapse
|
34
|
Xie F, Yuan H, Ning Y, Ong MEH, Feng M, Hsu W, Chakraborty B, Liu N. Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies. J Biomed Inform 2021; 126:103980. [PMID: 34974189 DOI: 10.1016/j.jbi.2021.103980] [Citation(s) in RCA: 24] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2021] [Revised: 11/07/2021] [Accepted: 12/20/2021] [Indexed: 12/21/2022]
Abstract
OBJECTIVE Temporal electronic health records (EHRs) contain a wealth of information for secondary uses, such as clinical events prediction and chronic disease management. However, challenges exist for temporal data representation. We therefore sought to identify these challenges and evaluate novel methodologies for addressing them through a systematic examination of deep learning solutions. METHODS We searched five databases (PubMed, Embase, the Institute of Electrical and Electronics Engineers [IEEE] Xplore Digital Library, the Association for Computing Machinery [ACM] Digital Library, and Web of Science) complemented with hand-searching in several prestigious computer science conference proceedings. We sought articles that reported deep learning methodologies on temporal data representation in structured EHR data from January 1, 2010, to August 30, 2020. We summarized and analyzed the selected articles from three perspectives: nature of time series, methodology, and model implementation. RESULTS We included 98 articles related to temporal data representation using deep learning. Four major challenges were identified, including data irregularity, heterogeneity, sparsity, and model opacity. We then studied how deep learning techniques were applied to address these challenges. Finally, we discuss some open challenges arising from deep learning. CONCLUSION Temporal EHR data present several major challenges for clinical prediction modeling and data utilization. To some extent, current deep learning solutions can address these challenges. Future studies may consider designing comprehensive and integrated solutions. Moreover, researchers should incorporate clinical domain knowledge into study designs and enhance model interpretability to facilitate clinical implementation.
Collapse
Affiliation(s)
- Feng Xie
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore; Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
| | - Han Yuan
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
| | - Yilin Ning
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore; Department of Emergency Medicine, Singapore General Hospital, Singapore
| | - Mengling Feng
- Saw Swee Hock School of Public Health, National University of Singapore, Singapore
| | - Wynne Hsu
- School of Computing, National University of Singapore, Singapore; Institute of Data Science, National University of Singapore, Singapore
| | - Bibhas Chakraborty
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore; Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore; Department of Statistics and Data Science, National University of Singapore, Singapore; Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States
| | - Nan Liu
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore; Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore; Institute of Data Science, National University of Singapore, Singapore; SingHealth AI Health Program, Singapore Health Services, Singapore.
| |
Collapse
|
35
|
Xie F, Ning Y, Yuan H, Goldstein BA, Ong MEH, Liu N, Chakraborty B. AutoScore-Survival: Developing interpretable machine learning-based time-to-event scores with right-censored survival data. J Biomed Inform 2021; 125:103959. [PMID: 34826628 DOI: 10.1016/j.jbi.2021.103959] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Revised: 11/18/2021] [Accepted: 11/19/2021] [Indexed: 12/12/2022]
Abstract
BACKGROUND Scoring systems are highly interpretable and widely used to evaluate time-to-event outcomes in healthcare research. However, existing time-to-event scores are predominantly created ad-hoc using a few manually selected variables based on clinician's knowledge, suggesting an unmet need for a robust and efficient generic score-generating method. METHODS AutoScore was previously developed as an interpretable machine learning score generator, integrating both machine learning and point-based scores in the strong discriminability and accessibility. We have further extended it to the time-to-event outcomes and developed AutoScore-Survival, for generating time-to-event scores with right-censored survival data. Random survival forest provided an efficient solution for selecting variables, and Cox regression was used for score weighting. We implemented our proposed method as an R package. We illustrated our method in a study of 90-day survival prediction for patients in intensive care units and compared its performance with other survival models, the random survival forest, and two traditional clinical scores. RESULTS The AutoScore-Survival-derived scoring system was more parsimonious than survival models built using traditional variable selection methods (e.g., penalized likelihood approach and stepwise variable selection), and its performance was comparable to survival models using the same set of variables. Although AutoScore-Survival achieved a comparable integrated area under the curve of 0.782 (95% CI: 0.767-0.794), the integer-valued time-to-event scores generated are favorable in clinical applications because they are easier to compute and interpret. CONCLUSIONS Our proposed AutoScore-Survival provides a robust and easy-to-use machine learning-based clinical score generator to studies of time-to-event outcomes. It gives a systematic guideline to facilitate the future development of time-to-event scores for clinical applications.
Collapse
Affiliation(s)
- Feng Xie
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore; Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Yilin Ning
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Han Yuan
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
| | - Benjamin Alan Goldstein
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore; Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore; Department of Emergency Medicine, Singapore General Hospital, Singapore, Singapore
| | - Nan Liu
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore; Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore; Institute of Data Science, National University of Singapore, Singapore, Singapore; Health Services Research Centre, Singapore Health Services, Singapore, Singapore.
| | - Bibhas Chakraborty
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore; Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore; Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States; Department of Statistics and Data Science, National University of Singapore, Singapore, Singapore
| |
Collapse
|
36
|
Aguilera A, Hernandez-Ramos R, Haro-Ramos AY, Boone CE, Luo TC, Xu J, Chakraborty B, Karr C, Darrow S, Figueroa CA. A Text Messaging Intervention (StayWell at Home) to Counteract Depression and Anxiety During COVID-19 Social Distancing: Pre-Post Study. JMIR Ment Health 2021; 8:e25298. [PMID: 34543230 PMCID: PMC8562416 DOI: 10.2196/25298] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/07/2021] [Revised: 07/07/2021] [Accepted: 09/13/2021] [Indexed: 02/06/2023] Open
Abstract
BACKGROUND Social distancing and stay-at-home orders are critical interventions to slow down person-to-person transmission of COVID-19. While these societal changes help contain the pandemic, they also have unintended negative consequences, including anxiety and depression. We developed StayWell, a daily skills-based SMS text messaging program, to mitigate COVID-19-related depression and anxiety symptoms among people who speak English and Spanish in the United States. OBJECTIVE This paper describes the changes in StayWell participants' anxiety and depression levels after 60 days of exposure to skills-based SMS text messages. METHODS We used self-administered, empirically supported web-based questionnaires to assess the demographic and clinical characteristics of StayWell participants. Anxiety and depression were measured using the 2-item Generalized Anxiety Disorder (GAD-2) scale and the 8-item Patient Health Questionnaire-8 (PHQ-8) scale at baseline and 60-day timepoints. We used 2-tailed paired t tests to detect changes in PHQ-8 and GAD-2 scores from baseline to follow-up measured 60 days later. RESULTS The analytic sample includes 193 participants who completed both the baseline and 60-day exit questionnaires. At the 60-day time point, there were significant reductions in both PHQ-8 and GAD-2 scores from baseline. We found an average reduction of -1.72 (95% CI -2.35 to -1.09) in PHQ-8 scores and -0.48 (95% CI -0.71 to -0.25) in GAD-2 scores. These improvements translated to an 18.5% and 17.2% reduction in mean PHQ-8 and GAD-2 scores, respectively. CONCLUSIONS StayWell is an accessible, low-intensity population-level mental health intervention. Participation in StayWell focused on COVID-19 mental health coping skills and was related to improved depression and anxiety symptoms. In addition to improvements in outcomes, we found high levels of engagement during the 60-day intervention period. Text messaging interventions could serve as an important public health tool for disseminating strategies to manage mental health. TRIAL REGISTRATION ClinicalTrials.gov NCT04473599; https://clinicaltrials.gov/ct2/show/NCT04473599. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) RR2-10.2196/23592.
Collapse
Affiliation(s)
- Adrian Aguilera
- School of Social Welfare, University of California, Berkeley, Berkeley, CA, United States.,Department of Psychiatry and Behavioral Sciences, University of California San Francisco, San Francisco, CA, United States
| | - Rosa Hernandez-Ramos
- School of Social Welfare, University of California, Berkeley, Berkeley, CA, United States
| | - Alein Y Haro-Ramos
- School of Public Health, University of California, Berkeley, Berkeley, CA, United States
| | - Claire Elizabeth Boone
- School of Public Health, University of California, Berkeley, Berkeley, CA, United States
| | - Tiffany Christina Luo
- School of Social Welfare, University of California, Berkeley, Berkeley, CA, United States
| | - Jing Xu
- Data Science Programme, Division of Science and Technology, Beijing Normal University-Hong Kong Baptist University United International College, Zhuhai, China.,Center for Quantitative Medicine, Duke National University of Singapore, Singapore, Singapore
| | - Bibhas Chakraborty
- Center for Quantitative Medicine, Duke National University of Singapore, Singapore, Singapore.,Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States.,Department of Statistics and Applied Probability, National University of Singapore, Singapore, Singapore
| | - Chris Karr
- Audacious Software, Chicago, IL, United States
| | - Sabrina Darrow
- Department of Psychiatry and Behavioral Sciences, University of California San Francisco, San Francisco, CA, United States
| | | |
Collapse
|
37
|
Yan X, Matchar DB, Sivapragasam N, Ansah JP, Goel A, Chakraborty B. Sequential Multiple Assignment Randomized Trial (SMART) to identify optimal sequences of telemedicine interventions for improving initiation of insulin therapy: A simulation study. BMC Med Res Methodol 2021; 21:200. [PMID: 34592951 PMCID: PMC8481760 DOI: 10.1186/s12874-021-01395-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Accepted: 09/08/2021] [Indexed: 01/06/2023] Open
Abstract
BACKGROUND To examine the value of a Sequential Multiple Assignment Randomized Trial (SMART) design compared to a conventional randomized control trial (RCT) for telemedicine strategies to support titration of insulin therapy for Type 2 Diabetes Mellitus (T2DM) patients new to insulin. METHODS Microsimulation models were created in R using a synthetic sample based on primary data from 63 subjects enrolled in a pilot study of a smartphone application (App), Diabetes Pal compared to a nurse-based telemedicine strategy (Nurse). For comparability, the SMART and an RCT design were constructed to allow comparison of four (embedded) adaptive interventions (AIs). RESULTS In the base case scenario, the SMART has similar overall mean expected HbA1c and cost per subject compared with RCT, for sample size of n = 100 over 10,000 simulations. SMART has lower (better) standard deviations of the mean expected HbA1c per AI, and higher efficiency of choosing the correct AI across various sample sizes. The differences between SMART and RCT become apparent as sample size decreases. For both trial designs, the threshold value at which a subject was deemed to have been responsive at an intermediate point in the trial had an optimal choice (i.e., the sensitivity curve had a U-shape). SMART design dominates the RCT, in the overall mean HbA1c (lower value) when the threshold value is close to optimal. CONCLUSIONS SMART is suited to evaluating the efficacy of different sequences of treatment options, in addition to the advantage of providing information on optimal treatment sequences.
Collapse
Affiliation(s)
- Xiaoxi Yan
- Centre for Quantitative Medicine. Duke-NUS Medical School, 8 College Road, Singapore, 169857 Singapore
| | - David B. Matchar
- Health Services and Systems Research Department, Duke-NUS Medical School, 8 College Road, Singapore, 169857 Singapore
- Department of Medicine, Duke University Medical Center, Durham, North Carolina USA
| | - Nirmali Sivapragasam
- Health Services and Systems Research Department, Duke-NUS Medical School, 8 College Road, Singapore, 169857 Singapore
| | - John P. Ansah
- Health Services and Systems Research Department, Duke-NUS Medical School, 8 College Road, Singapore, 169857 Singapore
| | - Aastha Goel
- Health Services and Systems Research Department, Duke-NUS Medical School, 8 College Road, Singapore, 169857 Singapore
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine. Duke-NUS Medical School, 8 College Road, Singapore, 169857 Singapore
- Department of Statistics and Applied Probability, Faculty of Science, National University of Singapore, Singapore, 117546 Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina USA
| |
Collapse
|
38
|
Sinha K, Chakraborty B, Chaudhury SS, Chaudhuri CR, Chattopadhyay SK, Das Mukhopadhyay C. Selective, Ultra-sensitive and Rapid Detection of Serotonin by Optimized ZnO Nanorod FET Biosensor. IEEE Trans Nanobioscience 2021; 21:65-74. [PMID: 34516379 DOI: 10.1109/tnb.2021.3112534] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
BACKGROUND Fluctuation in serotonin (5-HT) level is an essential manifestation of several neurological disorders. In view of such importance, it is necessary to monitor the levels of 5-HT with good sensitivity, selectivity, affordability and low response time. Zinc oxide (ZnO) based field effect transistors (FET) with attributes like minimized noise levels and large on-off ratio are regarded as emerging high performance biosensor platforms. However, their response is significantly non-linear and there has been no appreciable endeavor for improving the non-linearity. METHOD In this paper, we have introduced embedded gate electrode encompassing the channel of the FET which improves the uniformity in electric field line distribution through the electrolyte and proportionately enhances the capture of target biomolecule at ultra low concentrations, thereby increasing the linearity. Further, we have incorporated the optimized parameters of ZnO nanorods reported previously, for rapid and selective detection of 5-HT. RESULTS It has been observed that the fabricated ZnO FET biosensor lowers the detection limit down to 0.1fM which is at least one order of magnitude lower than the existing reports. The sensor also has wide linear range from 0.1fM to 1nM with a detection time of about 20 minutes. CONCLUSION The proposed zinc oxide nanorod-based sensor can be used as an excellent tool for future diagnosis of neurological disorders.
Collapse
|
39
|
Xie F, Ong MEH, Liew JNMH, Tan KBK, Ho AFW, Nadarajan GD, Low LL, Kwan YH, Goldstein BA, Matchar DB, Chakraborty B, Liu N. Development and Assessment of an Interpretable Machine Learning Triage Tool for Estimating Mortality After Emergency Admissions. JAMA Netw Open 2021; 4:e2118467. [PMID: 34448870 PMCID: PMC8397930 DOI: 10.1001/jamanetworkopen.2021.18467] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/15/2022] Open
Abstract
IMPORTANCE Triage in the emergency department (ED) is a complex clinical judgment based on the tacit understanding of the patient's likelihood of survival, availability of medical resources, and local practices. Although a scoring tool could be valuable in risk stratification, currently available scores have demonstrated limitations. OBJECTIVES To develop an interpretable machine learning tool based on a parsimonious list of variables available at ED triage; provide a simple, early, and accurate estimate of patients' risk of death; and evaluate the tool's predictive accuracy compared with several established clinical scores. DESIGN, SETTING, AND PARTICIPANTS This single-site, retrospective cohort study assessed all ED patients between January 1, 2009, and December 31, 2016, who were subsequently admitted to a tertiary hospital in Singapore. The Score for Emergency Risk Prediction (SERP) tool was derived using a machine learning framework. To estimate mortality outcomes after emergency admissions, SERP was compared with several triage systems, including Patient Acuity Category Scale, Modified Early Warning Score, National Early Warning Score, Cardiac Arrest Risk Triage, Rapid Acute Physiology Score, and Rapid Emergency Medicine Score. The initial analyses were completed in October 2020, and additional analyses were conducted in May 2021. MAIN OUTCOMES AND MEASURES Three SERP scores, namely SERP-2d, SERP-7d, and SERP-30d, were developed using the primary outcomes of interest of 2-, 7-, and 30-day mortality, respectively. Secondary outcomes included 3-day mortality and inpatient mortality. The SERP's predictive power was measured using the area under the curve in the receiver operating characteristic analysis. RESULTS The study included 224 666 ED episodes in the model training cohort (mean [SD] patient age, 63.60 [16.90] years; 113 426 [50.5%] female), 56 167 episodes in the validation cohort (mean [SD] patient age, 63.58 [16.87] years; 28 427 [50.6%] female), and 42 676 episodes in the testing cohort (mean [SD] patient age, 64.85 [16.80] years; 21 556 [50.5%] female). The mortality rates in the training cohort were 0.8% at 2 days, 2.2% at 7 days, and 5.9% at 30 days. In the testing cohort, the areas under the curve of SERP-30d were 0.821 (95% CI, 0.796-0.847) for 2-day mortality, 0.826 (95% CI, 0.811-0.841) for 7-day mortality, and 0.823 (95% CI, 0.814-0.832) for 30-day mortality and outperformed several benchmark scores. CONCLUSIONS AND RELEVANCE In this retrospective cohort study, SERP had better prediction performance than existing triage scores while maintaining easy implementation and ease of ascertainment in the ED. It has the potential to be widely applied and validated in different circumstances and health care settings.
Collapse
Affiliation(s)
- Feng Xie
- Programme in Health Services and Systems Research, Duke–National University of Singapore Medical School, Singapore
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke–National University of Singapore Medical School, Singapore
- Department of Emergency Medicine, Singapore General Hospital, Singapore
| | | | | | - Andrew Fu Wah Ho
- Programme in Health Services and Systems Research, Duke–National University of Singapore Medical School, Singapore
- Department of Emergency Medicine, Singapore General Hospital, Singapore
| | | | - Lian Leng Low
- Programme in Health Services and Systems Research, Duke–National University of Singapore Medical School, Singapore
- Department of Family Medicine and Continuing Care, Singapore General Hospital, Singapore
| | - Yu Heng Kwan
- Programme in Health Services and Systems Research, Duke–National University of Singapore Medical School, Singapore
- Department of Pharmacy, Faculty of Science, National University of Singapore, Singapore
| | - Benjamin Alan Goldstein
- Programme in Health Services and Systems Research, Duke–National University of Singapore Medical School, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina
| | - David Bruce Matchar
- Programme in Health Services and Systems Research, Duke–National University of Singapore Medical School, Singapore
- Duke University Medical Center, Duke University, Durham, North Carolina
| | - Bibhas Chakraborty
- Programme in Health Services and Systems Research, Duke–National University of Singapore Medical School, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina
- Department of Statistics and Data Science, National University of Singapore, Singapore
| | - Nan Liu
- Programme in Health Services and Systems Research, Duke–National University of Singapore Medical School, Singapore
- Health Service Research Centre, Singapore Health Services, Singapore
- Institute of Data Science, National University of Singapore, Singapore
| |
Collapse
|
40
|
Das N, Damodaran V, Chakraborty B, Roychaudhuri C. Experiment and FEM Analysis of Silica Nanoparticle-Based Impedance Immunosensor for Sensitivity Enhancement. IEEE Trans Nanobioscience 2021; 20:247-255. [PMID: 33690122 DOI: 10.1109/tnb.2021.3064677] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [MESH Headings] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
This article investigates the impact of incorporating silica nanoparticles of varying diameters in label free impedance immunosensor. It has been observed that even if the surface area improvement has been adjusted to be similar for all the diameters, the sensitivity is enhanced by five times at a particular diameter of 100 nm due to the optimum combination of intersection with electric field lines and surface convexity. This study has enabled the detection of 0.1 fM Hep-B surface antigen with a reliable sensitivity of around 75%. Further, it has been observed that the SNR corresponding to 0.1 fM is 20 dB only for 100 nm particle. This SNR is comparable to a recent report on Hep-B virus detection but the limit of detection in the proposed sensor is lowered by more than three orders of magnitude.
Collapse
|
41
|
Figueroa CA, Aguilera A, Chakraborty B, Modiri A, Aggarwal J, Deliu N, Sarkar U, Jay Williams J, Lyles CR. Adaptive learning algorithms to optimize mobile applications for behavioral health: guidelines for design decisions. J Am Med Inform Assoc 2021; 28:1225-1234. [PMID: 33657217 PMCID: PMC8200266 DOI: 10.1093/jamia/ocab001] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2020] [Accepted: 01/07/2021] [Indexed: 01/16/2023] Open
Abstract
OBJECTIVE Providing behavioral health interventions via smartphones allows these interventions to be adapted to the changing behavior, preferences, and needs of individuals. This can be achieved through reinforcement learning (RL), a sub-area of machine learning. However, many challenges could affect the effectiveness of these algorithms in the real world. We provide guidelines for decision-making. MATERIALS AND METHODS Using thematic analysis, we describe challenges, considerations, and solutions for algorithm design decisions in a collaboration between health services researchers, clinicians, and data scientists. We use the design process of an RL algorithm for a mobile health study "DIAMANTE" for increasing physical activity in underserved patients with diabetes and depression. Over the 1.5-year project, we kept track of the research process using collaborative cloud Google Documents, Whatsapp messenger, and video teleconferencing. We discussed, categorized, and coded critical challenges. We grouped challenges to create thematic topic process domains. RESULTS Nine challenges emerged, which we divided into 3 major themes: 1. Choosing the model for decision-making, including appropriate contextual and reward variables; 2. Data handling/collection, such as how to deal with missing or incorrect data in real-time; 3. Weighing the algorithm performance vs effectiveness/implementation in real-world settings. CONCLUSION The creation of effective behavioral health interventions does not depend only on final algorithm performance. Many decisions in the real world are necessary to formulate the design of problem parameters to which an algorithm is applied. Researchers must document and evaulate these considerations and decisions before and during the intervention period, to increase transparency, accountability, and reproducibility. TRIAL REGISTRATION clinicaltrials.gov, NCT03490253.
Collapse
Affiliation(s)
- Caroline A Figueroa
- School of Social Welfare, University of California Berkeley, Berkeley, California, USA
| | - Adrian Aguilera
- School of Social Welfare, University of California Berkeley, Berkeley, California, USA
- UCSF Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, California, USA
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-National University of Singapore Medical School, Singapore, Singapore
- Department of Statistics and Applied Probability, National University of Singapore, Singapore, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, USA
| | - Arghavan Modiri
- Department of Computer Science, University of Toronto, Toronto, Canada
| | - Jai Aggarwal
- Department of Computer Science, University of Toronto, Toronto, Canada
| | - Nina Deliu
- Department of Computer Science, University of Toronto, Toronto, Canada
- Department of Statistical Sciences, Sapienza University of Rome, Rome, Italy
| | - Urmimala Sarkar
- UCSF Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, California, USA
| | | | - Courtney R Lyles
- UCSF Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, California, USA
| |
Collapse
|
42
|
Tan YK, Teo P, Saffari SE, Xin X, Chakraborty B, Ng CT, Thumboo J. A musculoskeletal ultrasound program as an intervention to improve disease modifying anti-rheumatic drugs adherence in rheumatoid arthritis: a randomized controlled trial. Scand J Rheumatol 2021; 51:1-9. [PMID: 34107851 DOI: 10.1080/03009742.2021.1901416] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
Objectives: To evaluate the effect of a musculoskeletal ultrasound programme (MUSP) applying real-time ultrasonography with reinforcement of findings by a rheumatologist on improving disease-modifying anti-rheumatic drugs (DMARDs) adherence in rheumatoid arthritis (RA).Method: Eligible RA patients with low adherence score (< 6) on the 8-item Morisky Medication Adherence Scale (MMAS-8) were randomized to either an intervention group (receiving MUSP at baseline) or a control group (no MUSP), and followed up for 6 months. Adherence measures (patient-reported and pharmacy dispensing records) and clinical efficacy data were collected. The MUSP's feasibility and acceptability were assessed.Results: Among 132 recruited RA patients, six without baseline visits were excluded; therefore, 126 patients were analysed (62 intervention and 64 control). The primary outcome (proportion of patients with 1 month MMAS-8 score < 6) was significantly smaller (p = 0.019) in the intervention (35.48%) than the control group (56.25%). However, 3 and 6 month adherence and clinical efficacy outcomes were not significantly different between the two groups (all p > 0.05). All 62 patients completed the MUSP (mean time taken, 9.2 min), with the majority reporting moderately/very much improved understanding of their joint condition (71%) and the importance of regularly taking their RA medication(s) (79%). Most patients (90.3%) would recommend the MUSP to another RA patient.Conclusions: The MUSP improved RA patients' DMARDs adherence in the short term and was feasible and well accepted by patients. Future studies could evaluate whether repeated feedback using MUSP could help to sustain the improvement in DMARD adherence in RA patients, and whether this may be clinically impactful and cost-effective.
Collapse
Affiliation(s)
- Y K Tan
- Department of Rheumatology and Immunology, Singapore General Hospital, Singapore.,Duke-NUS Medical School, Singapore.,Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - Pse Teo
- Health Services Research Unit, Singapore General Hospital, Singapore
| | - S E Saffari
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore
| | - X Xin
- Health Services Research Unit, Singapore General Hospital, Singapore
| | - B Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore.,Department of Statistics and Applied Probability, National University of Singapore, Singapore.,Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| | - C T Ng
- Department of Rheumatology and Immunology, Singapore General Hospital, Singapore.,Duke-NUS Medical School, Singapore.,Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| | - J Thumboo
- Department of Rheumatology and Immunology, Singapore General Hospital, Singapore.,Duke-NUS Medical School, Singapore.,Yong Loo Lin School of Medicine, National University of Singapore, Singapore
| |
Collapse
|
43
|
Figueroa CA, Deliu N, Chakraborty B, Modiri A, Xu J, Aggarwal J, Jay Williams J, Lyles C, Aguilera A. Daily Motivational Text Messages to Promote Physical Activity in University Students: Results From a Microrandomized Trial. Ann Behav Med 2021; 56:212-218. [PMID: 33871015 DOI: 10.1093/abm/kaab028] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/17/2022] Open
Abstract
BACKGROUND Low physical activity is an important risk factor for common physical and mental disorders. Physical activity interventions delivered via smartphones can help users maintain and increase physical activity, but outcomes have been mixed. PURPOSE Here we assessed the effects of sending daily motivational and feedback text messages in a microrandomized clinical trial on changes in physical activity from one day to the next in a student population. METHODS We included 93 participants who used a physical activity app, "DIAMANTE" for a period of 6 weeks. Every day, their phone pedometer passively tracked participants' steps. They were microrandomized to receive different types of motivational messages, based on a cognitive-behavioral framework, and feedback on their steps. We used generalized estimation equation models to test the effectiveness of feedback and motivational messages on changes in steps from one day to the next. RESULTS Sending any versus no text message initially resulted in an increase in daily steps (729 steps, p = .012), but this effect decreased over time. A multivariate analysis evaluating each text message category separately showed that the initial positive effect was driven by the motivational messages though the effect was small and trend-wise significant (717 steps; p = .083), but not the feedback messages (-276 steps, p = .4). CONCLUSION Sending motivational physical activity text messages based on a cognitive-behavioral framework may have a positive effect on increasing steps, but this decreases with time. Further work is needed to examine using personalization and contextualization to improve the efficacy of text-messaging interventions on physical activity outcomes. CLINICALTRIALS.GOV IDENTIFIER NCT04440553.
Collapse
Affiliation(s)
| | - Nina Deliu
- Department of Statistical Sciences, Sapienza University of Rome, Rome, Italy
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine and Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore.,Department of Statistics and Applied Probability, National University of Singapore, Singapore, Singapore.,Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, USA
| | - Arghavan Modiri
- Department of Computer Science, University of Toronto, Toronto, ON, Canada
| | - Jing Xu
- Centre for Quantitative Medicine and Program in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore.,Data Science Program, Division of Science and Technology, Beijing Normal University and Hong Kong Baptist University-United International College, Zhuhai, Guangdong, China
| | - Jai Aggarwal
- Department of Computer Science, University of Toronto, Toronto, ON, Canada
| | | | - Courtney Lyles
- UCSF Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, CA, USA
| | - Adrian Aguilera
- School of Social Welfare, University of California, Berkeley, CA, USA.,UCSF Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, CA, USA
| |
Collapse
|
44
|
Mahar RK, McGuinness MB, Chakraborty B, Carlin JB, IJzerman MJ, Simpson JA. A scoping review of studies using observational data to optimise dynamic treatment regimens. BMC Med Res Methodol 2021; 21:39. [PMID: 33618655 PMCID: PMC7898728 DOI: 10.1186/s12874-021-01211-2] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Accepted: 01/19/2021] [Indexed: 11/16/2022] Open
Abstract
BACKGROUND Dynamic treatment regimens (DTRs) formalise the multi-stage and dynamic decision problems that clinicians often face when treating chronic or progressive medical conditions. Compared to randomised controlled trials, using observational data to optimise DTRs may allow a wider range of treatments to be evaluated at a lower cost. This review aimed to provide an overview of how DTRs are optimised with observational data in practice. METHODS Using the PubMed database, a scoping review of studies in which DTRs were optimised using observational data was performed in October 2020. Data extracted from eligible articles included target medical condition, source and type of data, statistical methods, and translational relevance of the included studies. RESULTS From 209 PubMed abstracts, 37 full-text articles were identified, and a further 26 were screened from the reference lists, totalling 63 articles for inclusion in a narrative data synthesis. Observational DTR models are a recent development and their application has been concentrated in a few medical areas, primarily HIV/AIDS (27, 43%), followed by cancer (8, 13%), and diabetes (6, 10%). There was substantial variation in the scope, intent, complexity, and quality between the included studies. Statistical methods that were used included inverse-probability weighting (26, 41%), the parametric G-formula (16, 25%), Q-learning (10, 16%), G-estimation (4, 6%), targeted maximum likelihood/minimum loss-based estimation (4, 6%), regret regression (3, 5%), and other less common approaches (10, 16%). Notably, studies that were primarily intended to address real-world clinical questions (18, 29%) tended to use inverse-probability weighting and the parametric G-formula, relatively well-established methods, along with a large amount of data. Studies focused on methodological developments (45, 71%) tended to be more complicated and included a demonstrative real-world application only. CONCLUSIONS As chronic and progressive conditions become more common, the need will grow for personalised treatments and methods to estimate the effects of DTRs. Observational DTR studies will be necessary, but so far their use to inform clinical practice has been limited. Focusing on simple DTRs, collecting large and rich clinical datasets, and fostering tight partnerships between content experts and data analysts may result in more clinically relevant observational DTR studies.
Collapse
Affiliation(s)
- Robert K Mahar
- Biostatistics Unit, Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, University of Melbourne, Parkville, Victoria, Australia.
- Cancer Health Services Research Unit, University of Melbourne Centre for Cancer Research and Centre for Health Policy, Melbourne School of Population and Global Health, University of Melbourne, Parkville, Victoria, Australia.
- Victorian Comprehensive Cancer Centre, Parkville, Victoria, Australia.
| | - Myra B McGuinness
- Biostatistics Unit, Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, University of Melbourne, Parkville, Victoria, Australia
- Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, Melbourne, Victoria, Australia
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-NUS Medical School, Singapore, Singapore
- Department of Statistics and Applied Probability, Faculty of Science, National University of Singapore, Singapore, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, USA
| | - John B Carlin
- Biostatistics Unit, Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, University of Melbourne, Parkville, Victoria, Australia
- Clinical Epidemiology and Biostatistics Unit, Murdoch Children's Research Institute, Parkville, Victoria, Australia
| | - Maarten J IJzerman
- Cancer Health Services Research Unit, University of Melbourne Centre for Cancer Research and Centre for Health Policy, Melbourne School of Population and Global Health, University of Melbourne, Parkville, Victoria, Australia
- Victorian Comprehensive Cancer Centre, Parkville, Victoria, Australia
- Peter MacCallum Cancer Centre, Parkville, Victoria, Australia
| | - Julie A Simpson
- Biostatistics Unit, Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, University of Melbourne, Parkville, Victoria, Australia
| |
Collapse
|
45
|
Chakraborty B, Das A, Mandal N, Samanta N, Das N, Chaudhuri CR. Label free, electric field mediated ultrasensitive electrochemical point-of-care device for CEA detection. Sci Rep 2021; 11:2962. [PMID: 33536505 PMCID: PMC7859218 DOI: 10.1038/s41598-021-82580-y] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2020] [Accepted: 01/20/2021] [Indexed: 01/09/2023] Open
Abstract
Developing point-of-care (PoC) diagnostic platforms for carcinoembryonic antigen detection is essential. However, thefew implementations of transferring the signal amplification strategies in electrochemical sensing on paper-based platforms are not satisfactory in terms of detection limit (LOD). In the quest for pushing down LOD, majority of the research has been targeted towards development of improved nanostructured substrates for entrapping more analyte molecules and augmenting the electron transfer rate to the working electrode. But, such approaches have reached saturation. This paper focuses on enhancing the mass transport of the analyte towards the sensor surface through the application of an electric field, in graphene-ZnO nanorods heterostructure. These hybrid nanostructures have been deposited on flexible polyethylene terephthalate substrates with screen printed electrodes for PoC application. The ZnO nanorods have been functionalized with aptamers and the working sensor has been integrated with smartphone interfaced indigenously developed low cost potentiostat. The performance of the system, requiring only 50 µl analyte has been evaluated using electrochemical impedance spectroscopy and validated against commercially available ELISA kit. Limit of detection of 1 fg/ml in human serum with 6.5% coefficient of variation has been demonstrated, which is more than three orders of magnitude lower than the existing attempts on PoC device.
Collapse
Affiliation(s)
- B Chakraborty
- Department of Electronics and Telecommunication Engineering, Indian Institute of Engineering Science and Technology, Shibpur, West Bengal, 711103, India
| | - A Das
- Department of Electronics and Telecommunication Engineering, Indian Institute of Engineering Science and Technology, Shibpur, West Bengal, 711103, India
| | - N Mandal
- School of Electrical Sciences, Indian Institute of Technology Goa, Ponda, 403401, Goa, India
| | - N Samanta
- Department of Electronics and Communication Engineering, Techno India University, Sector V, Kolkata, 700091, West Bengal, India
| | - N Das
- Department of Electronics and Communication Engineering, KL University, Green Fields, Vaddeswaram, Andhra Pradesh, 522502, India
| | - C Roy Chaudhuri
- Department of Electronics and Telecommunication Engineering, Indian Institute of Engineering Science and Technology, Shibpur, West Bengal, 711103, India.
| |
Collapse
|
46
|
Figueroa CA, Hernandez-Ramos R, Boone CE, Gómez-Pathak L, Yip V, Luo T, Sierra V, Xu J, Chakraborty B, Darrow S, Aguilera A. A Text Messaging Intervention for Coping With Social Distancing During COVID-19 (StayWell at Home): Protocol for a Randomized Controlled Trial. JMIR Res Protoc 2021; 10:e23592. [PMID: 33370721 PMCID: PMC7813560 DOI: 10.2196/23592] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2020] [Revised: 09/24/2020] [Accepted: 11/10/2020] [Indexed: 01/23/2023] Open
Abstract
BACKGROUND Social distancing is a crucial intervention to slow down person-to-person transmission of COVID-19. However, social distancing has negative consequences, including increases in depression and anxiety. Digital interventions, such as text messaging, can provide accessible support on a population-wide scale. We developed text messages in English and Spanish to help individuals manage their depressive mood and anxiety during the COVID-19 pandemic. OBJECTIVE In a two-arm randomized controlled trial, we aim to examine the effect of our 60-day text messaging intervention. Additionally, we aim to assess whether the use of machine learning to adapt the messaging frequency and content improves the effectiveness of the intervention. Finally, we will examine the differences in daily mood ratings between the message categories and time windows. METHODS The messages were designed within two different categories: behavioral activation and coping skills. Participants will be randomized into (1) a random messaging arm, where message category and timing will be chosen with equal probabilities, and (2) a reinforcement learning arm, with a learned decision mechanism for choosing the messages. Participants in both arms will receive one message per day within three different time windows and will be asked to provide their mood rating 3 hours later. We will compare self-reported daily mood ratings; self-reported depression, using the 8-item Patient Health Questionnaire; and self-reported anxiety, using the 7-item Generalized Anxiety Disorder scale at baseline and at intervention completion. RESULTS The Committee for the Protection of Human Subjects at the University of California Berkeley approved this study in April 2020 (No. 2020-04-13162). Data collection began in April 2020 and will run to April 2021. As of August 24, 2020, we have enrolled 229 participants. We plan to submit manuscripts describing the main results of the trial and results from the microrandomized trial for publication in peer-reviewed journals and for presentations at national and international scientific meetings. CONCLUSIONS Results will contribute to our knowledge of effective psychological tools to alleviate the negative effects of social distancing and the benefit of using machine learning to personalize digital mental health interventions. TRIAL REGISTRATION ClinicalTrials.gov NCT04473599; https://clinicaltrials.gov/ct2/show/NCT04473599. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) DERR1-10.2196/23592.
Collapse
Affiliation(s)
| | - Rosa Hernandez-Ramos
- School of Social Welfare, University of California Berkeley, Berkeley, CA, United States
| | | | - Laura Gómez-Pathak
- School of Social Welfare, University of California Berkeley, Berkeley, CA, United States
| | - Vivian Yip
- School of Social Welfare, University of California Berkeley, Berkeley, CA, United States
| | - Tiffany Luo
- School of Social Welfare, University of California Berkeley, Berkeley, CA, United States
| | - Valentín Sierra
- School of Social Welfare, University of California Berkeley, Berkeley, CA, United States
| | - Jing Xu
- Centre for Quantitative Medicine, Duke-National University of Singapore Medical School, Singapore, Singapore
- Data Science Program, Division of Science and Technology, Beijing Normal University-Hong Kong Baptist University United International College, Zhuhai, Guangdong, China
| | - Bibhas Chakraborty
- Centre for Quantitative Medicine, Duke-National University of Singapore Medical School, Singapore, Singapore
- Department of Statistics and Applied Probability, National University of Singapore, Singapore, Singapore
- Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, United States
| | - Sabrina Darrow
- Department of Psychiatry and Behavioral Sciences, University of California San Francisco, San Francisco, CA, United States
- Zuckerberg San Francisco General Hospital, San Francisco, CA, United States
| | - Adrian Aguilera
- School of Social Welfare, University of California Berkeley, Berkeley, CA, United States
- Department of Psychiatry and Behavioral Sciences, University of California San Francisco, San Francisco, CA, United States
- Zuckerberg San Francisco General Hospital, San Francisco, CA, United States
| |
Collapse
|
47
|
|
48
|
Cvijic M, Bezy S, Petrescu A, Santos P, Orlowska M, Chakraborty B, Duchenne J, Pedrosa J, Vanassche T, Van Cleemput J, Dhooge J, Voigt J. Differentiation of hypertensive heart disease and hypertrophic cardiomyopathy with myocardial stiffness measurements: a shear wave imaging study using ultra-high frame rate echocardiography. Eur Heart J 2020. [DOI: 10.1093/ehjci/ehaa946.0087] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
Abstract
Background
Recently, cardiac shear wave (SW) elastography, based on high frame rate (HFR) echocardiography, has been proposed as new non-invasive technique for assessing myocardial stiffness. As myocardial stiffness increases with increasing wall stress, differences in measured operating myocardial stiffness do not necessarily reflect differences in intrinsic myocardial properties, but can also be caused by mere changes in loading or chamber geometry. This complicates myocardial stiffness interpretation for different types of pathologic hypertrophy.
Purpose
To explore the relationship between myocardial stiffness and underlying pathological substrates for cardiac hypertrophy.
Methods
We included 20 patients with hypertension (HT) and myocardial remodelling (59±14 years, 75% male), 20 patients with hypertrophic cardiomyopathy (HCM) (59±16 years, 60% male) and 20 healthy controls (56±14 years, 75% male). Left ventricular (LV) parasternal long axis views were acquired with an experimental HFR scanner at 1293±362 frames per seconds. Propagation velocity of SW occurring after mitral valve closure in the interventricular septum (IVS) served as measure of operating myocardial stiffness (Figure A). To compare myocardial stiffness among hearts with differing loading conditions and chamber geometry, SW velocities were normalized to end-diastolic wall stress, estimated at IVS from regional wall thickness, longitudinal and circumferential regional radii of curvature, and non-invasively estimated LV end-diastolic pressure (EDP).
Results
SW velocities differed significantly between groups (p<0.001). The controls had the lowest SW velocities (4.02±0.97 m/s), whereas values between HT and HCM group were comparable (6.46±0.99 m/s vs. 7.00±2.10 m/s; p=0.738). Considering end-diastolic wall stress, HCM patients had the same SW velocity at lower wall stress compared to HT (Figure B), indicating higher myocardial stiffness in the HCM group. SW velocities normalized for wall stress indicated significantly different myocardial stiffness among all groups (p<0.001) (Figure C). In a multiple linear regression model, the underlying pathological substrate independently influenced SW velocity (beta 1.37, 95% CI (0.78–1.96); p<0.001), while wall stress did not significantly affect its value (p=0.479).
Conclusions
Our study demonstrated that SW elastography can detect differences in myocardial stiffness in hypertensive heart and hypertrophic cardiomyopathy. Additionally, our results suggest that SW velocity is dominated by underlying myocardial tissue properties. We hypothesize that differential changes in cardiomyocytes and/or the extracellular matrix contribute to the differential myocardial stiffening in different pathologic entities of LV hypertrophy. Thus, SW elastography could provide useful novel diagnostic information in the evaluation of LV hypertrophy.
Figure A, B, C
Funding Acknowledgement
Type of funding source: None
Collapse
Affiliation(s)
- M Cvijic
- University Hospitals (UZ) Leuven, Department of Cardiovascular Diseases, Leuven, Belgium
| | - S Bezy
- University Hospitals (UZ) Leuven, Department of Cardiovascular Diseases, Leuven, Belgium
| | - A Petrescu
- University Hospitals (UZ) Leuven, Department of Cardiovascular Diseases, Leuven, Belgium
| | - P Santos
- KU Leuven, Department of Cardiovascular Sciences, Leuven, Belgium
| | - M Orlowska
- KU Leuven, Department of Cardiovascular Sciences, Leuven, Belgium
| | - B Chakraborty
- KU Leuven, Department of Cardiovascular Sciences, Leuven, Belgium
| | - J Duchenne
- University Hospitals (UZ) Leuven, Department of Cardiovascular Diseases, Leuven, Belgium
| | - J Pedrosa
- KU Leuven, Department of Cardiovascular Sciences, Leuven, Belgium
| | - T Vanassche
- University Hospitals (UZ) Leuven, Department of Cardiovascular Diseases, Leuven, Belgium
| | - J Van Cleemput
- University Hospitals (UZ) Leuven, Department of Cardiovascular Diseases, Leuven, Belgium
| | - J Dhooge
- KU Leuven, Department of Cardiovascular Sciences, Leuven, Belgium
| | - J.U Voigt
- University Hospitals (UZ) Leuven, Department of Cardiovascular Diseases, Leuven, Belgium
| |
Collapse
|
49
|
Chakraborty B. Statistical Remedies for Medical Researchers
Peter F. Thall Springer, 2020, xi + 291 pages, £ 79.99/$109.99, hardcover ISBN: 978‐3‐030‐43713‐8. Int Stat Rev 2020. [DOI: 10.1111/insr.12419] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Affiliation(s)
- Bibhas Chakraborty
- Center for Quantitative Medicine, Duke‐NUS Medical School 8 College Road 169857 Singapore
- Department of Statistics and Applied Probability National University of Singapore Singapore
- Department of Biostatistics and Bioinformatics Duke University Durham North Carolina USA
| |
Collapse
|
50
|
Xie F, Chakraborty B, Ong MEH, Goldstein BA, Liu N. AutoScore: A Machine Learning-Based Automatic Clinical Score Generator and Its Application to Mortality Prediction Using Electronic Health Records. JMIR Med Inform 2020; 8:e21798. [PMID: 33084589 PMCID: PMC7641783 DOI: 10.2196/21798] [Citation(s) in RCA: 49] [Impact Index Per Article: 12.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2020] [Revised: 07/25/2020] [Accepted: 07/27/2020] [Indexed: 01/01/2023] Open
Abstract
BACKGROUND Risk scores can be useful in clinical risk stratification and accurate allocations of medical resources, helping health providers improve patient care. Point-based scores are more understandable and explainable than other complex models and are now widely used in clinical decision making. However, the development of the risk scoring model is nontrivial and has not yet been systematically presented, with few studies investigating methods of clinical score generation using electronic health records. OBJECTIVE This study aims to propose AutoScore, a machine learning-based automatic clinical score generator consisting of 6 modules for developing interpretable point-based scores. Future users can employ the AutoScore framework to create clinical scores effortlessly in various clinical applications. METHODS We proposed the AutoScore framework comprising 6 modules that included variable ranking, variable transformation, score derivation, model selection, score fine-tuning, and model evaluation. To demonstrate the performance of AutoScore, we used data from the Beth Israel Deaconess Medical Center to build a scoring model for mortality prediction and then compared the data with other baseline models using the receiver operating characteristic analysis. A software package in R 3.5.3 (R Foundation) was also developed to demonstrate the implementation of AutoScore. RESULTS Implemented on the data set with 44,918 individual admission episodes of intensive care, the AutoScore-created scoring models performed comparably well as other standard methods (ie, logistic regression, stepwise regression, least absolute shrinkage and selection operator, and random forest) in terms of predictive accuracy and model calibration but required fewer predictors and presented high interpretability and accessibility. The nine-variable, AutoScore-created, point-based scoring model achieved an area under the curve (AUC) of 0.780 (95% CI 0.764-0.798), whereas the model of logistic regression with 24 variables had an AUC of 0.778 (95% CI 0.760-0.795). Moreover, the AutoScore framework also drives the clinical research continuum and automation with its integration of all necessary modules. CONCLUSIONS We developed an easy-to-use, machine learning-based automatic clinical score generator, AutoScore; systematically presented its structure; and demonstrated its superiority (predictive performance and interpretability) over other conventional methods using a benchmark database. AutoScore will emerge as a potential scoring tool in various medical applications.
Collapse
Affiliation(s)
- Feng Xie
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
| | - Bibhas Chakraborty
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
- Department of Statistics and Applied Probability, National University of Singapore, Singapore, Singapore
- Department of Biostatistics & Bioinformatics, Duke University, Durham, NC, United States
| | - Marcus Eng Hock Ong
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
- Department of Emergency Medicine, Singapore General Hospital, Singapore, Singapore
- Health Services Research Centre, Singapore Health Services, Singapore, Singapore
| | - Benjamin Alan Goldstein
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
- Department of Biostatistics & Bioinformatics, Duke University, Durham, NC, United States
| | - Nan Liu
- Programme in Health Services and Systems Research, Duke-NUS Medical School, Singapore, Singapore
- Health Services Research Centre, Singapore Health Services, Singapore, Singapore
- Institute of Data Science, National University of Singapore, Singapore, Singapore
| |
Collapse
|