1
|
Liu Y, Zhang X, Cao W, Cui W, Tan T, Peng Y, Huang J, Lei Z, Shen J, Zheng J. Bootstrapping BI-RADS classification using large language models and transformers in breast magnetic resonance imaging reports. Vis Comput Ind Biomed Art 2025; 8:8. [PMID: 40178668 PMCID: PMC11968601 DOI: 10.1186/s42492-025-00189-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2024] [Accepted: 02/26/2025] [Indexed: 04/05/2025] Open
Abstract
Breast cancer is one of the most common malignancies among women globally. Magnetic resonance imaging (MRI), as the final non-invasive diagnostic tool before biopsy, provides detailed free-text reports that support clinical decision-making. Therefore, the effective utilization of the information in MRI reports to make reliable decisions is crucial for patient care. This study proposes a novel method for BI-RADS classification using breast MRI reports. Large language models are employed to transform free-text reports into structured reports. Specifically, missing category information (MCI) that is absent in the free-text reports is supplemented by assigning default values to the missing categories in the structured reports. To ensure data privacy, a locally deployed Qwen-Chat model is employed. Furthermore, to enhance the domain-specific adaptability, a knowledge-driven prompt is designed. The Qwen-7B-Chat model is fine-tuned specifically for structuring breast MRI reports. To prevent information loss and enable comprehensive learning of all report details, a fusion strategy is introduced, combining free-text and structured reports to train the classification model. Experimental results show that the proposed BI-RADS classification method outperforms existing report classification methods across multiple evaluation metrics. Furthermore, an external test set from a different hospital is used to validate the robustness of the proposed approach. The proposed structured method surpasses GPT-4o in terms of performance. Ablation experiments confirm that the knowledge-driven prompt, MCI, and the fusion strategy are crucial to the model's performance.
Collapse
Affiliation(s)
- Yuxin Liu
- School of Biomedical Engineering (Suzhou), University of Science and Technology of China, Division of Life Sciences and Medicine, Hefei, 230026, Anhui, China
- Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, Jiangsu, China
| | - Xiang Zhang
- Department of Radiology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou, 510120, Guangdong, China
- Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Medical Research Center, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou, 510120, Guangdong, China
| | - Weiwei Cao
- School of Biomedical Engineering (Suzhou), University of Science and Technology of China, Division of Life Sciences and Medicine, Hefei, 230026, Anhui, China
- Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, Jiangsu, China
| | - Wenju Cui
- School of Biomedical Engineering (Suzhou), University of Science and Technology of China, Division of Life Sciences and Medicine, Hefei, 230026, Anhui, China
- Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, Jiangsu, China
- Shandong Laboratory of Advanced Biomaterials and Medical Devices in Weihai, Shandong University, Weihai, 264200, Shandong, China
| | - Tao Tan
- Faculty of Applied Sciences, Macao Polytechnic University, Macao, China
| | - Yuqin Peng
- Department of Radiology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou, 510120, Guangdong, China
- Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Medical Research Center, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou, 510120, Guangdong, China
| | - Jiayi Huang
- Department of Radiology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou, 510120, Guangdong, China
- Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Medical Research Center, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou, 510120, Guangdong, China
| | - Zhen Lei
- Institute of Automation, Chinese Academy of Sciences, Beijing, 100190, China
| | - Jun Shen
- Department of Radiology, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou, 510120, Guangdong, China.
- Guangdong Provincial Key Laboratory of Malignant Tumor Epigenetics and Gene Regulation, Medical Research Center, Sun Yat-Sen Memorial Hospital, Sun Yat-Sen University, Guangzhou, 510120, Guangdong, China.
| | - Jian Zheng
- School of Biomedical Engineering (Suzhou), University of Science and Technology of China, Division of Life Sciences and Medicine, Hefei, 230026, Anhui, China.
- Medical Imaging Department, Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, 215163, Jiangsu, China.
- Shandong Laboratory of Advanced Biomaterials and Medical Devices in Weihai, Shandong University, Weihai, 264200, Shandong, China.
| |
Collapse
|
2
|
Atlas SJ, Tosteson ANA, Wright A, Orav EJ, Burdick TE, Zhao W, Hort SJ, Wint AJ, Smith RE, Chang FY, Aman DG, Thillaiyapillai M, Diamond CJ, Zhou L, Haas JS. A Multilevel Primary Care Intervention to Improve Follow-Up of Overdue Abnormal Cancer Screening Test Results: A Cluster Randomized Clinical Trial. JAMA 2023; 330:1348-1358. [PMID: 37815566 PMCID: PMC10565610 DOI: 10.1001/jama.2023.18755] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/19/2023] [Accepted: 08/31/2023] [Indexed: 10/11/2023]
Abstract
Importance Realizing the benefits of cancer screening requires testing of eligible individuals and processes to ensure follow-up of abnormal results. Objective To test interventions to improve timely follow-up of overdue abnormal breast, cervical, colorectal, and lung cancer screening results. Design, Setting, and Participants Pragmatic, cluster randomized clinical trial conducted at 44 primary care practices within 3 health networks in the US enrolling patients with at least 1 abnormal cancer screening test result not yet followed up between August 24, 2020, and December 13, 2021. Intervention Automated algorithms developed using data from electronic health records (EHRs) recommended follow-up actions and times for abnormal screening results. Primary care practices were randomized in a 1:1:1:1 ratio to (1) usual care, (2) EHR reminders, (3) EHR reminders and outreach (a patient letter was sent at week 2 and a phone call at week 4), or (4) EHR reminders, outreach, and navigation (a patient letter was sent at week 2 and a navigator outreach phone call at week 4). Patients, physicians, and practices were unblinded to treatment assignment. Main Outcomes and Measures The primary outcome was completion of recommended follow-up within 120 days of study enrollment. The secondary outcomes included completion of recommended follow-up within 240 days of enrollment and completion of recommended follow-up within 120 days and 240 days for specific cancer types and levels of risk. Results Among 11 980 patients (median age, 60 years [IQR, 52-69 years]; 64.8% were women; 83.3% were White; and 15.4% were insured through Medicaid) with an abnormal cancer screening test result for colorectal cancer (8245 patients [69%]), cervical cancer (2596 patients [22%]), breast cancer (1005 patients [8%]), or lung cancer (134 patients [1%]) and abnormal test results categorized as low risk (6082 patients [51%]), medium risk (3712 patients [31%]), or high risk (2186 patients [18%]), the adjusted proportion who completed recommended follow-up within 120 days was 31.4% in the EHR reminders, outreach, and navigation group (n = 3455), 31.0% in the EHR reminders and outreach group (n = 2569), 22.7% in the EHR reminders group (n = 3254), and 22.9% in the usual care group (n = 2702) (adjusted absolute difference for comparison of EHR reminders, outreach, and navigation group vs usual care, 8.5% [95% CI, 4.8%-12.0%], P < .001). The secondary outcomes showed similar results for completion of recommended follow-up within 240 days and by subgroups for cancer type and level of risk for the abnormal screening result. Conclusions and Relevance A multilevel primary care intervention that included EHR reminders and patient outreach with or without patient navigation improved timely follow-up of overdue abnormal cancer screening test results for breast, cervical, colorectal, and lung cancer. Trial Registration ClinicalTrials.gov Identifier: NCT03979495.
Collapse
Affiliation(s)
- Steven J. Atlas
- Division of General Internal Medicine, Massachusetts General Hospital and Harvard Medical School, Boston
| | - Anna N. A. Tosteson
- Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine, Dartmouth College, Lebanon, New Hampshire
- Dartmouth Cancer Center, Dartmouth Health and Geisel School of Medicine, Dartmouth College, Lebanon, New Hampshire
- Department of Community and Family Medicine, Dartmouth Health, Lebanon, New Hampshire
- Department of Medicine, Dartmouth Health, Lebanon, New Hampshire
| | - Adam Wright
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee
| | - E. John Orav
- Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Boston, Massachusetts
| | - Timothy E. Burdick
- Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine, Dartmouth College, Lebanon, New Hampshire
- Department of Community and Family Medicine, Dartmouth Health, Lebanon, New Hampshire
- SYNERGY Research Informatics, Dartmouth Health, Lebanon, New Hampshire
- Department of Biomedical Data Science, Dartmouth Health, Lebanon, New Hampshire
| | - Wenyan Zhao
- Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine, Dartmouth College, Lebanon, New Hampshire
| | - Shoshana J. Hort
- Department of Medicine, Dartmouth Health, Lebanon, New Hampshire
- SYNERGY Research Informatics, Dartmouth Health, Lebanon, New Hampshire
| | - Amy J. Wint
- Division of General Internal Medicine, Massachusetts General Hospital and Harvard Medical School, Boston
| | - Rebecca E. Smith
- Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine, Dartmouth College, Lebanon, New Hampshire
- Department of Community and Family Medicine, Dartmouth Health, Lebanon, New Hampshire
| | - Frank Y. Chang
- Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Boston, Massachusetts
| | - David G. Aman
- Research Computing, Dartmouth College, Lebanon, New Hampshire
| | | | - Courtney J. Diamond
- Department of Biomedical Informatics, Irving Medical Center, Columbia University, New York, New York
| | - Li Zhou
- Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Boston, Massachusetts
| | - Jennifer S. Haas
- Division of General Internal Medicine, Massachusetts General Hospital and Harvard Medical School, Boston
| |
Collapse
|
3
|
Saha A, Burns L, Kulkarni AM. A scoping review of natural language processing of radiology reports in breast cancer. Front Oncol 2023; 13:1160167. [PMID: 37124523 PMCID: PMC10130381 DOI: 10.3389/fonc.2023.1160167] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/06/2023] [Accepted: 03/28/2023] [Indexed: 05/02/2023] Open
Abstract
Various natural language processing (NLP) algorithms have been applied in the literature to analyze radiology reports pertaining to the diagnosis and subsequent care of cancer patients. Applications of this technology include cohort selection for clinical trials, population of large-scale data registries, and quality improvement in radiology workflows including mammography screening. This scoping review is the first to examine such applications in the specific context of breast cancer. Out of 210 identified articles initially, 44 met our inclusion criteria for this review. Extracted data elements included both clinical and technical details of studies that developed or evaluated NLP algorithms applied to free-text radiology reports of breast cancer. Our review illustrates an emphasis on applications in diagnostic and screening processes over treatment or therapeutic applications and describes growth in deep learning and transfer learning approaches in recent years, although rule-based approaches continue to be useful. Furthermore, we observe increased efforts in code and software sharing but not with data sharing.
Collapse
Affiliation(s)
- Ashirbani Saha
- Department of Oncology, McMaster University, Hamilton, ON, Canada
- Hamilton Health Sciences and McMaster University, Escarpment Cancer Research Institute, Hamilton, ON, Canada
| | - Levi Burns
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada
| | | |
Collapse
|