Mining electronic health records: towards better research applications and clinical care

Leveraging undecided cases in chart-reviewed phenotypes to enhance EHR-based association studies

OBJECTIVES

In electronic health record (EHR)-based association studies, phenotyping algorithms efficiently classify patient clinical outcomes into binary categories but are susceptible to misclassification errors. The gold standard, manual chart review, involves clinicians determining the true disease status based on their assessment of health records. These clinicians-labeled phenotypes are labor-intensive and typically limited to a small subset of patients, potentially introducing a third "undecided" category when phenotypes are indeterminate. We aim to effectively integrate the algorithm-derived and chart-reviewed outcomes when both are available in EHR-based association studies.

MATERIAL AND METHODS

We propose an augmented estimation method that combines the binary algorithm-derived phenotypes for the entire cohort with the trinary chart-reviewed phenotypes for a small, selected subset. Additionally, a cost-effective outcome-dependent sampling strategy is used to address the rare disease scenarios. The proposed trinary chart-reviewed phenotype integrated cost-effective augmented estimation (TriCA) was evaluated across a wide range of simulation settings and real-world applications, including using EHR data on Alzheimer's disease and related dementias (ADRD) from the OneFlorida + Clinical Research Network, and using cohort data on second breast cancer events (SBCE) from the Kaiser Permanente Washington.

RESULTS

Compared to estimation based on random sampling, our augmented method improved mean square error by up to 28.3% in simulation studies; compared to estimation using only trinary chart-reviewed phenotypes, our method improved efficiency by up to 33.3% in ADRD data and 50.8% in SBCE data.

DISCUSSION

Our simulation studies and real-world applications demonstrate that, compared to existing methods, the proposed method provides unbiased estimates with higher statistical efficiency.

CONCLUSION

The proposed method effectively combined binary algorithm-derived phenotypes for the whole cohort with trinary chart-reviewed outcomes for a limited validation set, making it applicable to a broader range of applications and enhancing risk factor identification in EHR-based association studies.

Development of a natural language processing algorithm to extract social determinants of health from clinician notes

Disparities in access to the organ transplant waitlist are well-documented, but research into modifiable factors has been limited due to a lack of access to organized prewaitlisting data. This study aimed to develop a natural language processing (NLP) algorithm to extract social determinants of health (SDOH) from free-text notes and quantify the association of SDOH with access to the transplant waitlist. We collected 261 802 clinician notes from 11 111 adults referred for kidney or liver transplants between 2016 and 2022 at the Duke University Health System. An SDOH ontology and a rule-based NLP algorithm were created to extract and organize terms. Education, transportation, and age were the most frequent terms identified. Negative sentiment and refer were the most negatively associated features with listing in both kidney and liver transplant patients. Income and employment for the kidney, and judgment and positive sentiment for liver were the most positively associated features with the listing. This study suggests that the integration of NLP tools into the transplant clinical workflow could help improve collection and organization of SDOH and inform center-level efforts at resource allocation, potentially improving access to the transplant waitlist and posttransplant outcomes.

Robust temporal knowledge inference via pathway snapshots with liquid neural network

Static graphs play a pivotal role in modeling and analyzing biological and biomedical data. However, many real-world scenarios-such as disease progression and drug pharmacokinetic processes-exhibit dynamic behaviors. Consequently, static graph methods often struggle to robustly address new environments characterized by complex and previously unseen relationship changes. Here, we propose a method for constructing temporal knowledge inference agents tailored to disease pathways, enabling effective relation reasoning beyond their training environment under complex shifts. To achieve this, we developed an imitation learning framework using liquid neural networks, a class of continuous-time neural models inspired by the brain function that are causal and adaptable to changing conditions. Our findings indicate that liquid agents can distill the essential tasks from knowledge graph inputs while accounting temporal evolution, thereby enabling the transfer of temporal skills to novel time nodes. Compared to state-of-the-art deep reinforcement learning agents, experiments demonstrate that temporal robustness in decision-making emerges uniquely in liquid networks.

ISPO: An Integrated Ontology of Symptom Phenotypes for Semantic Integration of Traditional Chinese Medical Data

Symptom phenotypes are crucial for diagnosing and treating various disease conditions. However, the diversity of symptom terminologies poses a significant challenge to analyzing and sharing of symptom-related medical data, particularly in the field of traditional Chinese medicine (TCM). This study aims to construct an Integrated Symptom Phenotype Ontology (ISPO) to support data mining of Chinese electronic medical records (EMRs) and real-world studies in the TCM field.We manually annotated and extracted symptom terms from 21 classical TCM textbooks and 78,696 inpatient EMRs, and integrated them with five publicly available symptom-related biomedical vocabularies. Through a human-machine collaborative approach for terminology editing and ontology development, including term screening, semantic mapping, and concept classification, we constructed a high-quality symptom ontology that integrates both TCM and Western medical terminology.ISPO provides 3,147 concepts, 23,475 terms, and 23,363 hierarchical relationships. Compared with international symptom-related ontologies such as the Symptom Ontology, ISPO offers significant improvements in the number of terms and synonymous relationships. Furthermore, evaluation across three independent curated clinical datasets demonstrated that ISPO achieved over 90% coverage of symptom terms, highlighting its strong clinical usability and completeness.ISPO represents the first clinical ontology globally dedicated to the systematic representation of symptoms. It integrates symptom terminologies from historical and contemporary sources, encompassing both TCM and Western medicine, thereby enhancing semantic interoperability across heterogeneous medical data sources and clinical decision support systems in TCM.

Enhancing medical coding efficiency through domain-specific fine-tuned large language models

Medical coding is essential for healthcare operations yet remains predominantly manual, error-prone (up to 20%), and costly (up to $18.2 billion annually). Although large language models (LLMs) have shown promise in natural language processing, their application to medical coding has produced limited accuracy. In this study, we evaluated whether fine-tuning LLMs with specialized ICD-10 knowledge can automate code generation across clinical documentation. We adopted a two-phase approach: initial fine-tuning using 74,260 ICD-10 code-description pairs, followed by enhanced training to address linguistic and lexical variations. Evaluations using a proprietary model (GPT-4o mini) on a cloud platform and an open-source model (Llama) on local GPUs demonstrated that initial fine-tuning increased exact matching from <1% to 97%, while enhanced fine-tuning further improved performance in complex scenarios, with real-world clinical notes achieving 69.20% exact match and 87.16% category match. These findings indicate that domain-specific fine-tuned LLMs can reduce manual burdens and improve reliability.

NLP for Analyzing Electronic Health Records and Clinical Notes in Cancer Research: A Review

This review examines the application of natural language processing (NLP) techniques in cancer research using electronic health records (EHRs) and clinical notes. It addresses gaps in existing literature by providing a broader perspective than previous studies focused on specific cancer types or applications. A comprehensive literature search in the Scopus database identified 94 relevant studies published between 2019 and 2024. The analysis revealed a growing trend in NLP applications for cancer research, with information extraction (47 studies) and text classification (40 studies) emerging as predominant NLP tasks, followed by named entity recognition (7 studies). Among cancer types, breast, lung, and colorectal cancers were found to be the most studied. A significant shift from rule-based and traditional machine learning approaches to advanced deep learning techniques and transformer-based models was observed. It was found that dataset sizes used in existing studies varied widely, ranging from small, manually annotated datasets to large-scale EHRs. The review highlighted key challenges, including the limited generalizability of proposed solutions and the need for improved integration into clinical workflows. While NLP techniques show significant potential in analyzing EHRs and clinical notes for cancer research, future work should focus on improving model generalizability, enhancing robustness in handling complex clinical language, and expanding applications to understudied cancer types. The integration of NLP tools into palliative medicine and addressing ethical considerations remain crucial for utilizing the full potential of NLP in enhancing cancer diagnosis, treatment, and patient outcomes. This review provides valuable insights into the current state and future directions of NLP applications in cancer research.

Comparing neural language models for medical concept representation and patient trajectory prediction

Effective representation of medical concepts is crucial for secondary analyses of electronic health records. Neural language models have shown promise in automatically deriving medical concept representations from clinical data. However, the comparative performance of different language models for creating these empirical representations, and the extent to which they encode medical semantics, has not been extensively studied. This study aims to address this gap by evaluating the effectiveness of three popular language models - word2vec, fastText, and GloVe - in creating medical concept embeddings that capture their semantic meaning. By using a large dataset of digital health records, we created patient trajectories and used them to train the language models. We then assessed the ability of the learned embeddings to encode semantics through an explicit comparison with biomedical terminologies, and implicitly by predicting patient outcomes and trajectories with different levels of available information. Our qualitative analysis shows that empirical clusters of embeddings learned by fastText exhibit the highest similarity with theoretical clustering patterns obtained from biomedical terminologies, with a similarity score between empirical and theoretical clusters of 0.88, 0.80, and 0.92 for diagnosis, procedure, and medication codes, respectively. Conversely, for outcome prediction, word2vec and GloVe tend to outperform fastText, with the former achieving AUROC as high as 0.78, 0.62, and 0.85 for length-of-stay, readmission, and mortality prediction, respectively. In predicting medical codes in patient trajectories, GloVe achieves the highest performance for diagnosis and medication codes (AUPRC of 0.45 and of 0.81, respectively) at the highest level of the semantic hierarchy, while fastText outperforms the other models for procedure codes (AUPRC of 0.66). Our study demonstrates that subword information is crucial for learning medical concept representations, but global embedding vectors are better suited for more high-level downstream tasks, such as trajectory prediction. Thus, these models can be harnessed to learn representations that convey clinical meaning, and our insights highlight the potential of using machine learning techniques to semantically encode medical data.

Optimal Surrogate-Assisted Sampling for Cost-Efficient Validation of Electronic Health Record Outcomes

Electronic Health Record (EHR) databases are an increasingly valuable resource for observational studies. However, misclassification of EHR-derived outcomes due to imperfect phenotyping leads to bias, inflated type I error, and reduced power in risk-factor association studies. On the other hand, manual chart review to validate outcomes is both cost-prohibitive and time-consuming, and a randomly selected validation sample may not yield sufficient cases to support precise model estimation when the disease is rare. Sampling procedures have been developed for maximizing computational and statistical efficiency in settings where the true disease status is known. However, less work has been done in measurement constrained settings, particularly when an informative surrogate outcome is available. Motivated by this gap, we propose an Optimal Subsampling strategy with Surrogate-Assisted Two-step procedure (OSSAT) to guide cost-effective chart review in measurement constrained settings. The sampling weight in OSSAT leverages information contained in the potentially misclassified phenotype and covariates to prioritize observations most informative for the model of interest. We compare our proposed weight with existing approaches through simulations under various covariate distributions, differential misclassification rates and degrees of surrogate accuracy. We then apply our proposed weighting schemes to a study of risk factors for second breast cancer events using a real EHR data set.

Modeling opioid overdose events recurrence with a covariate-adjusted triggering point process

Substance use disorder, particularly opioid-related, is a serious public health challenge in the U.S. Accurately predicting opioid overdose events and stratifying the risk of having such an event are critical for healthcare providers to deliver effective interventions in patients with opioid overdose. Despite a large body of literature investigating various risk factors for the prediction, the existing research to date has not explicitly investigated and quantitatively modeled how an individual's past opioid overdose events affect future occurrences. In this paper, we proposed a covariate-adjusted triggering point process to simultaneously model the effect of various risk factors on opioid overdose events and the triggering mechanism among opioid overdose events. The prediction performance was assessed by the U.S. state-wise Medicaid reimbursement claims data. Compared with commonly used prediction models, the proposed model achieved the lowest Mean Absolute Errors and Mean Absolute Percentage Errors on 30-, 60-, 90, 120-, 150-, and 180-day-ahead predictions. In addition, our results showed the statistical significance of considering the triggering mechanism for recurrent opioid overdose events prediction. On average, around 47% of the event recurrence were explained by the triggering mechanism.

Identifying progression subphenotypes of Alzheimer's disease from large-scale electronic health records with machine learning

OBJECTIVE

Identification of clinically meaningful subphenotypes of disease progression can enhance the understanding of disease heterogeneity and underlying pathophysiology. In this study, we propose a machine learning framework to identify subphenotypes of Alzheimer's disease progression based on longitudinal real-world patient records.

METHODS

The framework, dynaPhenoM, extracts coherent clinical topics across patient visits and employs a time-aware latent class analysis to characterize subphenotypes. We validated dynaPhenoM using three patient databases with a total of 3952 AD patients across the United States, demonstrating its effectiveness in revealing mild cognitive impairment (MCI) progression to AD.

RESULTS

Our study identified five subphenotypes associated with distinct organ systems for disease progression from MCI to AD, including common subtypes across cohorts-respiratory, musculoskeletal, cardiovascular, and endocrine/metabolic-as well as a cohort-specific digestive subtype.

CONCLUSION

Our study unravels the complexity and heterogeneity of the progression from MCI to AD. These findings highlight disease progression heterogeneity and can inform both diagnostic and therapeutic strategies, thereby advancing precision medicine for Alzheimer's disease.

Robust privacy amidst innovation with large language models through a critical assessment of the risks

OBJECTIVE

This study evaluates the integration of electronic health records (EHRs) and natural language processing (NLP) with large language models (LLMs) to enhance healthcare data management and patient care, focusing on using advanced language models to create secure, Health Insurance Portability and Accountability Act-compliant synthetic patient notes for global biomedical research.

MATERIALS AND METHODS

The study used de-identified and re-identified versions of the MIMIC III dataset with GPT-3.5, GPT-4, and Mistral 7B to generate synthetic clinical notes. Text generation employed templates and keyword extraction for contextually relevant notes, with One-shot generation for comparison. Privacy was assessed by analyzing protected health information (PHI) occurrence and co-occurrence, while utility was evaluated by training an ICD-9 coder using synthetic notes. Text quality was measured using ROUGE (Recall-Oriented Understudy for Gisting Evaluation) and cosine similarity metrics to compare synthetic notes with source notes for semantic similarity.

RESULTS

The analysis of PHI occurrence and text utility via the ICD-9 coding task showed that the keyword-based method had low risk and good performance. One-shot generation exhibited the highest PHI exposure and PHI co-occurrence, particularly in geographic location and date categories. The Normalized One-shot method achieved the highest classification accuracy. Re-identified data consistently outperformed de-identified data.

DISCUSSION

Privacy analysis revealed a critical balance between data utility and privacy protection, influencing future data use and sharing.

CONCLUSION

This study shows that keyword-based methods can create synthetic clinical notes that protect privacy while retaining data usability, potentially improving clinical data sharing. The use of dummy PHIs to counter privacy attacks may offer better utility and privacy than traditional de-identification.

Learning-Based Models for Predicting IVIG Resistance and Coronary Artery Lesions in Kawasaki Disease: A Review of Technical Aspects and Study Features

Kawasaki disease (KD) is a common pediatric vasculitis, with coronary artery lesions (CALs) representing its most severe complication. Early identification of high-risk patients, including those with disease resistant to first-line treatments, is essential to guide personalized therapeutic approaches. Given the limited reliability of current scoring systems, there has been growing interest in the development of new prognostic models based on machine learning algorithms and artificial intelligence (AI). AI has the potential to revolutionize the management of KD by improving patient stratification and supporting more targeted treatment strategies. This narrative review examines recent applications of AI in stratifying patients with KD, with a particular focus on the ability of models to predict intravenous immunoglobulin resistance and the risk of CALs. We analyzed studies published between January 2019 and April 2024 that incorporated AI-based predictive models. In total, 21 papers met the inclusion criteria and were subject to technical and statistical review; 90% of these were conducted in patients from Asian hospitals. Most of the studies (18/21; 85.7%) were retrospective, and two-thirds included fewer than 1000 patients. Significant heterogeneity in study design and parameter selection was observed across the studies. Resistance to intravenous immunoglobulin emerged as a key factor in AI-based models for predicting CALs. Only five models demonstrated a sensitivity > 80%, and four studies provided access to the underlying algorithms and datasets. Challenges such as small sample sizes, class imbalance, and the need for multicenter validation currently limit the clinical applicability of machine-learning-based predictive models. The effectiveness of AI models is heavily influenced by the quantity and quality of data, labeling accuracy, and the completeness of the training datasets. Additionally, issues such as noise and missing data can negatively affect model performance and generalizability. These limitations highlight the need for rigorous validation and open access to model code to ensure transparency and reproducibility. Collaboration and data sharing will be essential for refining AI algorithms, improving patient stratification, and optimizing treatment strategies.

A Genetics-guided Integrative Framework for Drug Repurposing: Identifying Anti-hypertensive Drug Telmisartan for Type 2 Diabetes

Drug development is a long and costly process, and repurposing existing drugs for use toward a different disease or condition may serve as a cost-effective alternative. As drug targets with genetic support have a doubled success rate, genetics-informed drug repurposing holds promise in translating genetic findings into therapeutics. In this study, we developed a Genetics Informed Network-based Drug Repurposing via in silico Perturbation (GIN-DRIP) framework and applied the framework to repurpose drugs for type-2 diabetes (T2D). In GIN-DRIP for T2D, it integrates multi-level omics data to translate T2D GWAS signals into a genetics-informed network that simultaneously encodes gene importance scores and a directional effect (up/down) of risk genes for T2D; it then bases on the GIN to perform signature matching with drug perturbation experiments to identify drugs that can counteract the effect of T2D risk alleles. With this approach, we identified 3 high-confidence FDA-approved candidate drugs for T2D, and validated telmisartan, an anti-hypertensive drug, in our EHR data with over 3 million patients. We found that telmisartan users were associated with a reduced incidence of T2D compared to users of other anti-hypertensive drugs and non-users, supporting the therapeutic potential of telmisartan for T2D. Our framework can be applied to other diseases for translating GWAS findings to aid drug repurposing for complex diseases.

Exploring the implementation of a data trust committee: a qualitative evaluation of processes and practices

BACKGROUND

There's a significant demand to link and analyse administrative and routine local hospital data for health research to improve treatments and understand disease and diagnosis. Involving patients and members of the public in how data are accessed for service improvement is crucial for developing an acceptable, ethical and information governance-compliant whole system data linkage. A key challenge is ensuring sustainable and genuine public engagement that fosters trust in data use. This study evaluates the early implementation of a Data Trust Committee (DTC) at a London hospital, assessing its impact on research efficiency and the experiences of key stakeholders, including patients, staff and researchers.

METHODS

A rapid qualitative evaluation was conducted using semi-structured to assess the implementation and perceived impact of the DTC. Purposive sampling targeted DTC members (n = 8), attendees (n = 3), and researchers (n = 2). Thematic analysis, supported by RREAL sheets, identified key themes in stakeholders' experiences and perceptions.

RESULTS

Findings highlighted five key areas: (1) the programme theory, outlining the DTC's role in data governance and responsible data access; (2) varying stakeholder perceptions of the DTC's purpose and decision-making processes; (3) The DTC's impact on research oversight, data access and approval processes; (4) challenges related to role clarification and communication; (5) the perceived effectiveness of the DTC in enhancing data quality, research oversight and approval speed. While participants recognised the DTC's potential to enhance data quality and prioritising patient experiences, challenges related to the speed of applications, communication gaps, and technology barriers were identified.

CONCLUSION

The DTC played a pivotal role in reshaping research regulatory processes, and how this may benefit patients. However, balancing ethical risks with patient benefits remains an ongoing challenge. Addressing role clarity, communication strategies, and stakeholder engagement is essential for optimising future DTC implementation. Future research should expand to evaluate DTC models across diverse healthcare settings to enhance data sharing frameworks.

Digital Phenotyping of Rare Endocrine Diseases Across International Data Networks and the Effect of Granularity of Original Vocabulary

PURPOSE

Rare diseases occur in <50 per 100000 people and require lifelong management. However, essential epidemiological data on such diseases are lacking, and a consecutive monitoring system across time and regions remains to be established. Standardized digital phenotypes are required to leverage an international data network for research on rare endocrine diseases. We developed digital phenotypes for rare endocrine diseases using the observational medical outcome partnership common data model.

MATERIALS AND METHODS

Digital phenotypes of three rare endocrine diseases (medullary thyroid cancer, hypoparathyroidism, pheochromocytoma/paraganglioma) were validated across three databases that use different vocabularies: Severance Hospital's electronic health record from South Korea; IQVIA's United Kingdom (UK) database for general practitioners; and IQVIA's United States (US) hospital database for general hospitals. We estimated the performance of different digital phenotyping methods based on International Classification of Diseases (ICD)-10 in the UK and the US or systematized nomenclature of medicine clinical terms (SNOMED CT) in Korea.

RESULTS

The positive predictive value of digital phenotyping was higher using SNOMED CT-based phenotyping than ICD-10-based phenotyping for all three diseases in Korea (e.g., pheochromocytoma/paraganglioma: ICD-10, 58%-62%; SNOMED CT, 89%). Estimated incidence rates by digital phenotyping were as follows: medullary thyroid cancer, 0.34-2.07 (Korea), 0.13-0.30 (US); hypoparathyroidism, 0.40-1.20 (Korea), 0.59-1.01 (US), 0.00-1.78 (UK); and pheochromocytoma/paraganglioma, 0.95-1.67 (Korea), 0.35-0.77 (US), 0.00-0.49 (UK).

CONCLUSION

Our findings demonstrate the feasibility of developing digital phenotyping of rare endocrine diseases and highlight the importance of implementing SNOMED CT in routine clinical practice to provide granularity for research.

An in-depth review of AI-powered advancements in cancer drug discovery

The convergence of artificial intelligence (AI) and genomics is redefining cancer drug discovery by facilitating the development of personalized and effective therapies. This review examines the transformative role of AI technologies, including deep learning and advanced data analytics, in accelerating key stages of the drug discovery process: target identification, drug design, clinical trial optimization, and drug response prediction. Cutting-edge tools such as DrugnomeAI and PandaOmics have made substantial contributions to therapeutic target identification, while AI's predictive capabilities are driving personalized treatment strategies. Additionally, advancements like AlphaFold highlight AI's capacity to address intricate challenges in drug development. However, the field faces significant challenges, including the management of large-scale genomic datasets and ethical concerns surrounding AI deployment in healthcare. This review underscores the promise of data-centric AI approaches and emphasizes the necessity of continued innovation and interdisciplinary collaboration. Together, AI and genomics are charting a path toward more precise, efficient, and transformative cancer therapeutics.

Active learning for extracting rare adverse events from electronic health records: A study in pediatric cardiology

OBJECTIVE

Automate the extraction of adverse events from the text of electronic medical records of patients hospitalized for cardiac catheterization.

METHODS

We focused on events related to cardiac catheterization as defined by the NCDR-IMPACT registry. These events were extracted from the Necker Children's Hospital data warehouse. Electronic health records were pre-screened using regular expressions. The resulting datasets contained numerous false positives sentences that were annotated by a cardiologist using an active learning process. A deep learning text classifier was then trained on this active learning-annotated dataset to accurately identify patients who have suffered a serious adverse event.

RESULTS

The dataset included 2,980 patients. Regular expression based extraction of adverse events related to cardiac catheterization achieved a perfect recall. Due to the rarity of adverse events, the dataset obtained from this initial pre-screening step was imbalanced, containing a significant number of false positives. The active learning annotation enabled the acquisition of a representative dataset suitable for training a deep learning model. The deep learning text-classifier identified patients who underwent adverse events after cardiac catheterization with a recall of 0.78 and a specificity of 0.94.

CONCLUSION

Our model effectively identified patients who experienced adverse events related to cardiac catheterization using real clinical data. Enabled by an active learning annotation process, it shows promise for large language model applications in clinical research, especially for rare diseases with limited annotated databases. Our model's strength lies in its development by physicians for physicians, ensuring its relevance and applicability in clinical practice.

Post-discharge suicide prediction among US veterans using natural language processing-enriched social and behavioral determinants of health

Despite the established association between social and behavioral determinants of health (SBDH) and suicide risk, SBDHs from unstructured electronic health record notes for suicide prediction remain underutilized. This study investigates the impact of SBDH identified from both structured and unstructured data utilizing a natural language processing (NLP) system on suicide prediction at 7, 30, 90, and 180 days post-discharge. Using data from 2,987,006 US Veterans between 1 October 2009, and 30 September 2015, we designed a case-control study demonstrating that structured and NLP-extracted SBDH significantly enhance distinct prediction models' performance. For example, the random forest model improved its 180-day post-discharge prediction with an area under the receiver operating characteristic curve increase from 83.57% to 84.25% (95% CI = 0.63%-0.98%, p val < 0.001) and area under the precision-recall curve increase from 57.38% to 59.87% (95% CI = 3.86%-4.82%, p val < 0.001) after integrating NLP-extracted SBDH. These findings underscore the potential of NLP-extracted SBDH in advancing suicide prediction.

Externally validated and clinically useful machine learning algorithms to support patient-related decision-making in oncology: a scoping review

BACKGROUND

This scoping review systematically maps externally validated machine learning (ML)-based models in cancer patient care, quantifying their performance, and clinical utility, and examining relationships between models, cancer types, and clinical decisions. By synthesizing evidence, this study identifies, strengths, limitations, and areas requiring further research.

METHODS

The review followed the Joanna Briggs Institute's methodology, Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines, and the Population, Concept, and Context mnemonic. Searches were conducted across Embase, IEEE Xplore, PubMed, Scopus, and Web of Science (January 2014-September 2022), targeting English-language quantitative studies in Q1 journals (SciMago Journal and Country Ranking > 1) that used ML to evaluate clinical outcomes for human cancer patients with commonly available data. Eligible models required external validation, clinical utility assessment, and performance metric reporting. Studies involving genetics, synthetic patients, plants, or animals were excluded. Results were presented in tabular, graphical, and descriptive form.

RESULTS

From 4023 deduplicated abstracts and 636 full-text reviews, 56 studies (2018-2022) met the inclusion criteria, covering diverse cancer types and applications. Convolutional neural networks were most prevalent, demonstrating high performance, followed by gradient- and decision tree-based algorithms. Other algorithms, though underrepresented, showed promise. Lung and digestive system cancers were most frequently studied, focusing on diagnosis and outcome predictions. Most studies were retrospective and multi-institutional, primarily using image-based data, followed by text-based and hybrid approaches. Clinical utility assessments involved 499 clinicians and 12 tools, indicating improved clinician performance with AI assistance and superior performance to standard clinical systems.

DISCUSSION

Interest in ML-based clinical decision-making has grown in recent years alongside increased multi-institutional collaboration. However, small sample sizes likely impacted data quality and generalizability. Persistent challenges include limited international validation across ethnicities, inconsistent data sharing, disparities in validation metrics, and insufficient calibration reporting, hindering model comparison reliability.

CONCLUSION

Successful integration of ML in oncology decision-making requires standardized data and methodologies, larger sample sizes, greater transparency, and robust validation and clinical utility assessments.

OTHER

Financed by FCT-Fundação para a Ciência e a Tecnologia (Portugal, project LA/P/0063/2020, grant 2021.09040.BD) as part of CSS's Ph.D. This work was not registered.

Challenges and opportunities for digital twins in precision medicine from a complex systems perspective

Digital twins (DTs) in precision medicine are increasingly viable, propelled by extensive data collection and advancements in artificial intelligence (AI), alongside traditional biomedical methodologies. We argue that including mechanistic simulations that produce behavior based on explicitly defined biological hypotheses and multiscale mechanisms is beneficial. It enables the exploration of diverse therapeutic strategies and supports dynamic clinical decision-making through insights from network science, quantitative biology, and digital medicine.

Evaluation of machine learning-based regression techniques for prediction of diabetes levels fluctuations

Middle-Aged and Elderly people today face a variety of health problems as a result of their modern lifestyle, which includes increased work stress, less physical activity, and altered food habits. Because of Complications arising, diabetes has become one of the most frequent, severe, and fatal illnesses around the world. Therefore, inaccurate measurements of blood glucose levels can seriously damage vital organs. Several strategies for long-term glucose prediction have been proposed in the literature. Unfortunately, these methods require the patient to identify their daily activities, which can be error-prone, such as meal intake, insulin injection, and emotional aspects. This paper suggests using continuous glucose monitoring (CGM) of 14733 patients, with three assistance factors to predict blood glucose levels independently of other parameters, hence reducing the burden on the patients. To support this an Artificial Neural Network (ANN), Binary Decision Tree (BDT), Linear Regression (LR), Boosting Regression Tree Ensemble (BSTE), Linear Regression with Stochastic Gradient Descent (LRSGD), Stepwise (SW), Support Vector Machine (SVM), and Gaussian process regression (GPR) were investigated. The result indicated that The highest classification accuracy of (92.58%) has been achieved by BDT followed by BSTE (92.04%) and GPR (88.59%). The obtained average of root means square error (MSE) was 1.64, 1.67, 1.69, mg/dL for prediction horizon (PH) respectively to GPR, BSTE, and ANN.

Twenty-Five Years of Evolution and Hurdles in Electronic Health Records and Interoperability in Medical Research: Comprehensive Review

BACKGROUND

Electronic health records (EHRs) facilitate the accessibility and sharing of patient data among various health care providers, contributing to more coordinated and efficient care.

OBJECTIVE

This study aimed to summarize the evolution of secondary use of EHRs and their interoperability in medical research over the past 25 years.

METHODS

We conducted an extensive literature search in the PubMed, Scopus, and Web of Science databases using the keywords Electronic health record and Electronic medical record in the title or abstract and Medical research in all fields from 2000 to 2024. Specific terms were applied to different time periods.

RESULTS

The review yielded 2212 studies, all of which were then screened and processed in a structured manner. Of these 2212 studies, 2102 (93.03%) were included in the review analysis, of which 1079 (51.33%) studies were from 2000 to 2009, 582 (27.69%) were from 2010 to 2019, 251 (11.94%) were from 2020 to 2023, and 190 (9.04%) were from 2024.

CONCLUSIONS

The evolution of EHRs marks an important milestone in health care's journey toward integrating technology and medicine. From early documentation practices to the sophisticated use of artificial intelligence and big data analytics today, EHRs have become central to improving patient care, enhancing public health surveillance, and advancing medical research.

Whole slide images as non-fungible tokens: A decentralized approach to secure, scalable data storage and access

Background

Distributed ledger technology (DLT) enables the creation of tamper-resistant, decentralized, and secure digital ledgers. A non-fungible token (NFT) represents a record on-chain associated with a digital or physical asset, such as a whole-slide image (WSI). The InterPlanetary File System (IPFS) represents an off-chain network, hypermedia, and file sharing peer-to-peer protocol for storing and sharing data in a distributed file system. Today, we need cheaper, more efficient, highly scalable, and transparent solutions for WSI data storage and access of medical records and medical imaging data.

Methods

WSIs were created from non-human tissues and H&E-stained sections were scanned on a Philips Ultrafast WSI scanner at 40× magnification objective lens (1 μm/pixel). TIFF images were stored on IPFS, while NFTs were minted on the Ethereum blockchain network in ERC-1155 standard. WSI-NFTs were stored on MetaMask and OpenSea was used to display the WSI-NFT collection. Filebase storage application programing interface (API) were used to create dedicated gateways and content delivery networks (CDN).

Results

A total of 10 WSI-NFTs were minted on the Ethereum blockchain network, found on our collection "Whole Slide Images as Non-fungible Tokens Project" on Open Sea: https://opensea.io/collection/untitled-collection-126765644. WSI TIFF files ranged in size from 1.6 to 2.2 GB and were stored on IPFS and pinned on 3 separate nodes. Under optimal conditions, and using a dedicated CDN, WSI reached retrieved at speeds of over 10 mb/s, however, download speeds and WSI retrieval times varied significantly depending on the file and gateway used. Overall, the public IPFS gateway resulted in variably poorer WSI download retrieval performance compared to gateways provided by Filebase storage API.

Conclusion

Whole-slide images, as the most complex and substantial data files in healthcare, demand innovative solutions. In this technical report, we identify pitfalls in IPFS, and demonstrate proof-of-concept using a 3-layer architecture for scalable, decentralized storage, and access. Optimized through dedicated gateways and CDNs, which can be effectively applied to all medical data and imaging modalities across the healthcare sector. DLT and off-chain network solutions present numerous opportunities for advancements in clinical care, education, and research. Such approaches uphold the principles of equitable healthcare data ownership, security, and democratization, and are poised to drive significant innovation.

Mdpg: a novel multi-disease diagnosis prediction method based on patient knowledge graphs

Diagnosis prediction, a key factor in enhancing healthcare efficiency, remains a focal point in clinical decision support research. However, the time-series, sparse and multi-noise characteristics of electronic health record (EHR) data make it a great challenge. Existing methods commonly address these issues using RNNs and incorporating medical prior knowledge from medical knowledge bases, but they neglect the local spatial characteristics and spatial-temporal correlation of the data. Consequently, we propose MDPG, a diagnosis prediction model based on patient knowledge graphs. Initially, we represent the electronic visit records of patients as a patient-centered temporal knowledge graph, capturing the local spatial structure and temporal characteristics of the visit information. Subsequently, we design the spatial graph convolution block, temporal self-attention block, and spatial-temporal synchronous graph convolution block to capture the spatial, temporal, and spatial-temporal correlations embedded in them, respectively. Ultimately, we accomplish the prediction of patients' future states through multi-label classification. We conduct comprehensive experiments on two real-world datasets independently and evaluate the results using visit-level precision@k and code-level accuracy@k metrics. The experimental results demonstrate that MDPG outperforms all baseline models, yielding the best performance.

Artificial intelligence-aided data mining of medical records for cancer detection and screening

The application of artificial intelligence methods to electronic patient records paves the way for large-scale analysis of multimodal data. Such population-wide data describing deep phenotypes composed of thousands of features are now being leveraged to create data-driven algorithms, which in turn has led to improved methods for early cancer detection and screening. Remaining challenges include establishment of infrastructures for prospective testing of such methods, ways to assess biases given the data, and gathering of sufficiently large and diverse datasets that reflect disease heterogeneities across populations. This Review provides an overview of artificial intelligence methods designed to detect cancer early, including key aspects of concern (eg, the problem of data drift-when the underlying health-care data change over time), ethical aspects, and discrepancies between access to cancer screening in high-income countries versus low-income and middle-income countries.

Robust extraction of pneumonia-associated clinical states from electronic health records

Mining of electronic health records (EHR) promises to automate the identification of comprehensive disease phenotypes. However, the realization of this promise is hindered by the unavailability of generalizable ground-truth information, data incompleteness and heterogeneity, and the lack of generalization to multiple cohorts. We present here a data-driven approach to identify clinical states that we implement for 585 critical care patients with suspected pneumonia recruited by the SCRIPT study, which we compare to and integrate with 9,918 pneumonia patients from the MIMIC-IV dataset. We extract and curate from their structured EHRs a primary set of clinical features (53 and 59 features for SCRIPT and MIMIC-IV, respectively), including disease severity scores, vital signs, and so on, at various degrees of completeness. We aggregate irregular time series into daily frequency, resulting in 12,495 and 94,684 patient-day pairs for SCRIPT and MIMIC, respectively. We define a "common-sense" ground truth that we then use in a semisupervised pipeline to optimize choices for data preprocessing, and reduce the feature space to four principal components. We describe and validate an ensemble-based clustering method that enables us to robustly identify five clinical states, and use a Gaussian mixture model to quantify uncertainty in cluster assignment. Demonstrating the clinical relevance of the identified states, we find that three states are strongly associated with disease outcomes (dying vs. recovering), while the other two reflect disease etiology. The outcome associated clinical states provide significantly increased discrimination of mortality rates over standard approaches.

Deep Reinforcement Learning for personalized diagnostic decision pathways using Electronic Health Records: A comparative study on anemia and Systemic Lupus Erythematosus

BACKGROUND

Clinical diagnoses are typically made by following a series of steps recommended by guidelines that are authored by colleges of experts. Accordingly, guidelines play a crucial role in rationalizing clinical decisions. However, they suffer from limitations, as they are designed to cover the majority of the population and often fail to account for patients with uncommon conditions. Moreover, their updates are long and expensive, making them unsuitable for emerging diseases and new medical practices.

METHODS

Inspired by guidelines, we formulate the task of diagnosis as a sequential decision-making problem and study the use of Deep Reinforcement Learning (DRL) algorithms to learn the optimal sequence of actions to perform in order to obtain a correct diagnosis from Electronic Health Records (EHRs), which we name a diagnostic decision pathway. We apply DRL to synthetic yet realistic EHRs and develop two clinical use cases: Anemia diagnosis, where the decision pathways follow a decision tree schema, and Systemic Lupus Erythematosus (SLE) diagnosis, which follows a weighted criteria score. We particularly evaluate the robustness of our approaches to noise and missing data, as these frequently occur in EHRs.

RESULTS

In both use cases, even with imperfect data, our best DRL algorithms exhibit competitive performance compared to traditional classifiers, with the added advantage of progressively generating a pathway to the suggested diagnosis, which can both guide and explain the decision-making process.

CONCLUSION

DRL offers the opportunity to learn personalized decision pathways for diagnosis. Our two use cases illustrate the advantages of this approach: they generate step-by-step pathways that are explainable, and their performance is competitive when compared to state-of-the-art methods.

Medical-Knowledge-Based Graph Neural Network for Medication Combination Prediction

Medication combination prediction (MCP) can provide assistance for experts in the more thorough comprehension of complex mechanisms behind health and disease. Many recent studies focus on the patient representation from the historical medical records, but neglect the value of the medical knowledge, such as the prior knowledge and the medication knowledge. This article develops a medical-knowledge-based graph neural network (MK-GNN) model which incorporates the representation of patients and the medical knowledge into the neural network. More specifically, the features of patients are extracted from their medical records in different feature subspaces. Then these features are concatenated to obtain the feature representation of patients. The prior knowledge, which is calculated according to the mapping relationship between medications and diagnoses, provides heuristic medication features according to the diagnosis results. Such medication features can help the MK-GNN model learn optimal parameters. Moreover, the medication relationship in prescriptions is formulated as a drug network to integrate the medication knowledge into medication representation vectors. The results reveal the superior performance of the MK-GNN model compared with the state-of-the-art baselines on different evaluation metrics. The case study manifests the application potential of the MK-GNN model.

Advancing governance for digital transformation in health: insights from Georgia's experience

Enhancing digital health governance is critical to healthcare systems in low-income and middle-income countries. However, implementing governance-enhancing reforms in these countries is often challenging due to the multiplicity of external players and insufficient operational guidance that is accessible. Using data from desktop research, in-depth interviews, focus group discussions and three stakeholder workshops, this paper aims to provide insights into Georgia's experience in advancing digital health governance reforms. It reveals how Georgia has progressed on this path by unpacking the general term 'governance' into operational domains, where stakeholders and involved institutions could easily relate their institutional and personal roles and responsibilities with the specific function needed for digital health. Based on this work, the country delineated institutional responsibilities and passed the necessary regulations to establish better governance arrangements for digital health. The Georgia experience provides practical insights into the challenges faced and solutions found for advancing digital health governance in a middle-income country setting. The paper highlights the usefulness of operational definitions for the digital health governance domains that helped (a) increase awareness among stakeholders about the identified domains and their meaning, (b) discuss possible governance and institutional arrangements relevant to a country context, and (c) design the digital health governance architecture that the government decreed. Finally, the paper offers a broad description of domains in which the governance arrangements could be considered and used for other settings where relevant. The paper points to the need for a comprehensive taxonomy for governance domains to better guide digital health governance enhancements in low-middle-income country settings.

LLM-AIx: An open source pipeline for Information Extraction from unstructured medical text based on privacy preserving Large Language Models

In clinical science and practice, text data, such as clinical letters or procedure reports, is stored in an unstructured way. This type of data is not a quantifiable resource for any kind of quantitative investigations and any manual review or structured information retrieval is time-consuming and costly. The capabilities of Large Language Models (LLMs) mark a paradigm shift in natural language processing and offer new possibilities for structured Information Extraction (IE) from medical free text. This protocol describes a workflow for LLM based information extraction (LLM-AIx), enabling extraction of predefined entities from unstructured text using privacy preserving LLMs. By converting unstructured clinical text into structured data, LLM-AIx addresses a critical barrier in clinical research and practice, where the efficient extraction of information is essential for improving clinical decision-making, enhancing patient outcomes, and facilitating large-scale data analysis. The protocol consists of four main processing steps: 1) Problem definition and data preparation, 2) data preprocessing, 3) LLM-based IE and 4) output evaluation. LLM-AIx allows integration on local hospital hardware without the need of transferring any patient data to external servers. As example tasks, we applied LLM-AIx for the anonymization of fictitious clinical letters from patients with pulmonary embolism. Additionally, we extracted symptoms and laterality of the pulmonary embolism of these fictitious letters. We demonstrate troubleshooting for potential problems within the pipeline with an IE on a real-world dataset, 100 pathology reports from the Cancer Genome Atlas Program (TCGA), for TNM stage extraction. LLM-AIx can be executed without any programming knowledge via an easy-to-use interface and in no more than a few minutes or hours, depending on the LLM model selected.

Improving large language models for clinical named entity recognition via prompt engineering

IMPORTANCE

The study highlights the potential of large language models, specifically GPT-3.5 and GPT-4, in processing complex clinical data and extracting meaningful information with minimal training data. By developing and refining prompt-based strategies, we can significantly enhance the models' performance, making them viable tools for clinical NER tasks and possibly reducing the reliance on extensive annotated datasets.

OBJECTIVES

This study quantifies the capabilities of GPT-3.5 and GPT-4 for clinical named entity recognition (NER) tasks and proposes task-specific prompts to improve their performance.

MATERIALS AND METHODS

We evaluated these models on 2 clinical NER tasks: (1) to extract medical problems, treatments, and tests from clinical notes in the MTSamples corpus, following the 2010 i2b2 concept extraction shared task, and (2) to identify nervous system disorder-related adverse events from safety reports in the vaccine adverse event reporting system (VAERS). To improve the GPT models' performance, we developed a clinical task-specific prompt framework that includes (1) baseline prompts with task description and format specification, (2) annotation guideline-based prompts, (3) error analysis-based instructions, and (4) annotated samples for few-shot learning. We assessed each prompt's effectiveness and compared the models to BioClinicalBERT.

RESULTS

Using baseline prompts, GPT-3.5 and GPT-4 achieved relaxed F1 scores of 0.634, 0.804 for MTSamples and 0.301, 0.593 for VAERS. Additional prompt components consistently improved model performance. When all 4 components were used, GPT-3.5 and GPT-4 achieved relaxed F1 socres of 0.794, 0.861 for MTSamples and 0.676, 0.736 for VAERS, demonstrating the effectiveness of our prompt framework. Although these results trail BioClinicalBERT (F1 of 0.901 for the MTSamples dataset and 0.802 for the VAERS), it is very promising considering few training samples are needed.

DISCUSSION

The study's findings suggest a promising direction in leveraging LLMs for clinical NER tasks. However, while the performance of GPT models improved with task-specific prompts, there's a need for further development and refinement. LLMs like GPT-4 show potential in achieving close performance to state-of-the-art models like BioClinicalBERT, but they still require careful prompt engineering and understanding of task-specific knowledge. The study also underscores the importance of evaluation schemas that accurately reflect the capabilities and performance of LLMs in clinical settings.

CONCLUSION

While direct application of GPT models to clinical NER tasks falls short of optimal performance, our task-specific prompt framework, incorporating medical knowledge and training samples, significantly enhances GPT models' feasibility for potential clinical applications.

Effectiveness of PCSK9 inhibitors: A Target Trial Emulation framework based on Real-World Electronic Health Records

Low-Density Lipoprotein (LDL) cholesterol is one of the main target for cardiovascular (CV) prevention and therapy. In the last years, Proprotein Convertase Subtilisin-Kexin type 9 inhibitors (PCSK9-i) has emerged as a key therapeutic target to lower LDL and were introduced for prevention of CV events. Recently (June 2022) the Italian Medicines Agency (AIFA) modified the eligibility criteria for the use of PCSK9-i. We designed an observational study to estimate the prevalence of eligible subjects and evaluate the effectiveness of PCSK9-i applying a Target Trial Emulation (TTE) approach based on Electronic Health Records (EHR). Subjects meeting the eligibility criteria were identified from July 2017 (when PCSK9-i became available) to December 2020. Outcomes were all-cause death and the first hospitalization. Among eligible subjects, we identified those treated at date of the first prescription. Inverse Probability of Treatment Weights (IPTW) were estimated including demographic and clinical covariates, history of treatment with statins and the month/year eligibility date. Competing risk models on weighted cohorts were used to derive the Average Treatment Effect (ATE) and the Conditional Average Treatment Effect (CATE) in subgroups of interest. Out of 1976 eligible subjects, 161 (8%) received treatment with PCSK9-i. Treated individuals were slightly younger, predominantly male, had more severe CV conditions, and were more often treated with statin compared to the untreated subjects. The latter exhibited a higher prevalence of non-CV comorbidities. A significant absolute and relative risk reduction of death and a lower relative risk for the first hospitalization was observed. The risk reduction for death was confirmed in CATE analysis. PCSk9-i were prescribed to a minority of eligible subjects. Within the TTE framework, the analysis confirmed the association between PCSK9-i and lower risk of events, aligning with findings from randomized clinical trials (RCTs). In our study, PCSK9-i provided protection specifically against all-cause death, expanding upon the evidence from RCTs that had primarily focused on composite CV outcomes.

Patient-reported outcomes in lung cancer surgery: A narrative review

Lung cancer is a leading cause of cancer-related mortality worldwide, profoundly affecting patients' quality of life. Patient-reported outcomes (PROs) provide essential insights from the patients' perspective, a crucial aspect often overlooked by traditional clinical outcomes. This review synthesizes research on the role of PROs in lung cancer surgery to enhance patient care and outcomes. We conducted a comprehensive literature search across PubMed, Scopus, and Web of Science up to March 2024, using terms such as "lung cancer," "Patient Reported Outcome," "lobectomy," "segmentectomy," and "lung surgery." The criteria included original studies on lung cancer patients who underwent surgical treatment and reported on PROs. After screening and removing duplicates, reviews, non-English articles, and irrelevant studies, 36 research articles were selected, supported by an additional 53 publications, totaling 89 references. The findings highlight the utility of PROs in assessing post-surgical outcomes, informing clinical decisions, and facilitating patient-centered care. However, challenges in standardization, patient burden, and integration into clinical workflows remain, underscoring the need for further research and methodological refinement. PROs are indispensable for understanding the quality-of-life post-surgery and enhancing communication and decision-making in clinical practice. Their integration into routine care is vital for a holistic approach to lung cancer treatment, promising significant improvements in patient outcomes and quality of care.

Applications of Multimodal Artificial Intelligence in Non-Hodgkin Lymphoma B Cells

Given advancements in large-scale data and AI, integrating multimodal artificial intelligence into cancer research can enhance our understanding of tumor behavior by simultaneously processing diverse biomedical data types. In this review, we explore the potential of multimodal AI in comprehending B-cell non-Hodgkin lymphomas (B-NHLs). B-cell non-Hodgkin lymphomas (B-NHLs) represent a particular challenge in oncology due to tumor heterogeneity and the intricate ecosystem in which tumors develop. These complexities complicate diagnosis, prognosis, and therapy response, emphasizing the need to use sophisticated approaches to enhance personalized treatment strategies for better patient outcomes. Therefore, multimodal AI can be leveraged to synthesize critical information from available biomedical data such as clinical record, imaging, pathology and omics data, to picture the whole tumor. In this review, we first define various types of modalities, multimodal AI frameworks, and several applications in precision medicine. Then, we provide several examples of its usage in B-NHLs, for analyzing the complexity of the ecosystem, identifying immune biomarkers, optimizing therapy strategy, and its clinical applications. Lastly, we address the limitations and future directions of multimodal AI, highlighting the need to overcome these challenges for better clinical practice and application in healthcare.

Graph Artificial Intelligence in Medicine

In clinical artificial intelligence (AI), graph representation learning, mainly through graph neural networks and graph transformer architectures, stands out for its capability to capture intricate relationships and structures within clinical datasets. With diverse data-from patient records to imaging-graph AI models process data holistically by viewing modalities and entities within them as nodes interconnected by their relationships. Graph AI facilitates model transfer across clinical tasks, enabling models to generalize across patient populations without additional parameters and with minimal to no retraining. However, the importance of human-centered design and model interpretability in clinical decision-making cannot be overstated. Since graph AI models capture information through localized neural transformations defined on relational datasets, they offer both an opportunity and a challenge in elucidating model rationale. Knowledge graphs can enhance interpretability by aligning model-driven insights with medical knowledge. Emerging graph AI models integrate diverse data modalities through pretraining, facilitate interactive feedback loops, and foster human-AI collaboration, paving the way toward clinically meaningful predictions.

A novel hyperparameter search approach for accuracy and simplicity in disease prediction risk scoring

OBJECTIVE

Develop a novel technique to identify an optimal number of regression units corresponding to a single risk point, while creating risk scoring systems from logistic regression-based disease predictive models. The optimal value of this hyperparameter balances simplicity and accuracy, yielding risk scores of small scale and high accuracy for patient risk stratification.

MATERIALS AND METHODS

The proposed technique applies an adapted line search across all potential hyperparameter values. Additionally, DeLong test is integrated to ensure the selected value produces an accuracy insignificantly different from the best achievable risk score accuracy. We assessed the approach through two case studies predicting diabetic retinopathy (DR) within six months and hip fracture readmissions (HFR) within 30 days, involving cohorts of 90 400 diabetic patients and 18 065 hip fracture patients.

RESULTS

Our scores achieve accuracies insignificantly different from those obtained by existing approaches, reaching AUROCs of 0.803 and 0.645 for DR and HFR predictions, respectively. Regarding the scale, our scores ranged 0-53 for DR and 0-15 for HFR, while scores produced by existing methods frequently spanned hundreds or thousands.

DISCUSSION

According to the assessment, our risk scores offer simple and accurate predictions for diseases. Furthermore, our new DR score provides a competitive alternative to state-of-the-art risk scores for DR, while our HFR case study presents the first risk score for this condition.

CONCLUSION

Our technique offers a generalizable framework for crafting precise risk scores of compact scales, addressing the demand for user-friendly and effective risk stratification tool in healthcare.

Disease Trajectories from Healthcare Data: Methodologies, Key Results, and Future Perspectives

Disease trajectories, defined as sequential, directional disease associations, have become an intense research field driven by the availability of electronic population-wide healthcare data and sufficient computational power. Here, we provide an overview of disease trajectory studies with a focus on European work, including ontologies used as well as computational methodologies for the construction of disease trajectories. We also discuss different applications of disease trajectories from descriptive risk identification to disease progression, patient stratification, and personalized predictions using machine learning. We describe challenges and opportunities in the area that eventually will benefit from initiatives such as the European Health Data Space, which, with time, will make it possible to analyze data from cohorts comprising hundreds of millions of patients.

Multi-head CRF classifier for biomedical multi-class named entity recognition on Spanish clinical notes

The identification of medical concepts from clinical narratives has a large interest in the biomedical scientific community due to its importance in treatment improvements or drug development research. Biomedical named entity recognition (NER) in clinical texts is crucial for automated information extraction, facilitating patient record analysis, drug development, and medical research. Traditional approaches often focus on single-class NER tasks, yet recent advancements emphasize the necessity of addressing multi-class scenarios, particularly in complex biomedical domains. This paper proposes a strategy to integrate a multi-head conditional random field (CRF) classifier for multi-class NER in Spanish clinical documents. Our methodology overcomes overlapping entity instances of different types, a common challenge in traditional NER methodologies, by using a multi-head CRF model. This architecture enhances computational efficiency and ensures scalability for multi-class NER tasks, maintaining high performance. By combining four diverse datasets, SympTEMIST, MedProcNER, DisTEMIST, and PharmaCoNER, we expand the scope of NER to encompass five classes: symptoms, procedures, diseases, chemicals, and proteins. To the best of our knowledge, these datasets combined create the largest Spanish multi-class dataset focusing on biomedical entity recognition and linking for clinical notes, which is important to train a biomedical model in Spanish. We also provide entity linking to the multi-lingual Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT) vocabulary, with the eventual goal of performing biomedical relation extraction. Through experimentation and evaluation of Spanish clinical documents, our strategy provides competitive results against single-class NER models. For NER, our system achieves a combined micro-averaged F1-score of 78.73, with clinical mentions normalized to SNOMED CT with an end-to-end F1-score of 54.51. The code to run our system is publicly available at https://github.com/ieeta-pt/Multi-Head-CRF. Database URL: https://github.com/ieeta-pt/Multi-Head-CRF.

Unsupervised Feature Selection to Identify Important ICD-10 and ATC Codes for Machine Learning on a Cohort of Patients With Coronary Heart Disease: Retrospective Study

Background

The application of machine learning in health care often necessitates the use of hierarchical codes such as the International Classification of Diseases (ICD) and Anatomical Therapeutic Chemical (ATC) systems. These codes classify diseases and medications, respectively, thereby forming extensive data dimensions. Unsupervised feature selection tackles the "curse of dimensionality" and helps to improve the accuracy and performance of supervised learning models by reducing the number of irrelevant or redundant features and avoiding overfitting. Techniques for unsupervised feature selection, such as filter, wrapper, and embedded methods, are implemented to select the most important features with the most intrinsic information. However, they face challenges due to the sheer volume of ICD and ATC codes and the hierarchical structures of these systems.

Objective

The objective of this study was to compare several unsupervised feature selection methods for ICD and ATC code databases of patients with coronary artery disease in different aspects of performance and complexity and select the best set of features representing these patients.

Methods

We compared several unsupervised feature selection methods for 2 ICD and 1 ATC code databases of 51,506 patients with coronary artery disease in Alberta, Canada. Specifically, we used the Laplacian score, unsupervised feature selection for multicluster data, autoencoder-inspired unsupervised feature selection, principal feature analysis, and concrete autoencoders with and without ICD or ATC tree weight adjustment to select the 100 best features from over 9000 ICD and 2000 ATC codes. We assessed the selected features based on their ability to reconstruct the initial feature space and predict 90-day mortality following discharge. We also compared the complexity of the selected features by mean code level in the ICD or ATC tree and the interpretability of the features in the mortality prediction task using Shapley analysis.

Results

In feature space reconstruction and mortality prediction, the concrete autoencoder-based methods outperformed other techniques. Particularly, a weight-adjusted concrete autoencoder variant demonstrated improved reconstruction accuracy and significant predictive performance enhancement, confirmed by DeLong and McNemar tests (P<.05). Concrete autoencoders preferred more general codes, and they consistently reconstructed all features accurately. Additionally, features selected by weight-adjusted concrete autoencoders yielded higher Shapley values in mortality prediction than most alternatives.

Conclusions

This study scrutinized 5 feature selection methods in ICD and ATC code data sets in an unsupervised context. Our findings underscore the superiority of the concrete autoencoder method in selecting salient features that represent the entire data set, offering a potential asset for subsequent machine learning research. We also present a novel weight adjustment approach for the concrete autoencoders specifically tailored for ICD and ATC code data sets to enhance the generalizability and interpretability of the selected features.

Accessible Ecosystem for Clinical Research (Federated Learning for Everyone): Development and Usability Study

BACKGROUND

The integrity and reliability of clinical research outcomes rely heavily on access to vast amounts of data. However, the fragmented distribution of these data across multiple institutions, along with ethical and regulatory barriers, presents significant challenges to accessing relevant data. While federated learning offers a promising solution to leverage insights from fragmented data sets, its adoption faces hurdles due to implementation complexities, scalability issues, and inclusivity challenges.

OBJECTIVE

This paper introduces Federated Learning for Everyone (FL4E), an accessible framework facilitating multistakeholder collaboration in clinical research. It focuses on simplifying federated learning through an innovative ecosystem-based approach.

METHODS

The "degree of federation" is a fundamental concept of FL4E, allowing for flexible integration of federated and centralized learning models. This feature provides a customizable solution by enabling users to choose the level of data decentralization based on specific health care settings or project needs, making federated learning more adaptable and efficient. By using an ecosystem-based collaborative learning strategy, FL4E encourages a comprehensive platform for managing real-world data, enhancing collaboration and knowledge sharing among its stakeholders.

RESULTS

Evaluating FL4E's effectiveness using real-world health care data sets has highlighted its ecosystem-oriented and inclusive design. By applying hybrid models to 2 distinct analytical tasks-classification and survival analysis-within real-world settings, we have effectively measured the "degree of federation" across various contexts. These evaluations show that FL4E's hybrid models not only match the performance of fully federated models but also avoid the substantial overhead usually linked with these models. Achieving this balance greatly enhances collaborative initiatives and broadens the scope of analytical possibilities within the ecosystem.

CONCLUSIONS

FL4E represents a significant step forward in collaborative clinical research by merging the benefits of centralized and federated learning. Its modular ecosystem-based design and the "degree of federation" feature make it an inclusive, customizable framework suitable for a wide array of clinical research scenarios, promising to revolutionize the field through improved collaboration and data use. Detailed implementation and analyses are available on the associated GitHub repository.

Implementing a Biomedical Data Warehouse From Blueprint to Bedside in a Regional French University Hospital Setting: Unveiling Processes, Overcoming Challenges, and Extracting Clinical Insight

Background

Biomedical data warehouses (BDWs) have become an essential tool to facilitate the reuse of health data for both research and decisional applications. Beyond technical issues, the implementation of BDWs requires strong institutional data governance and operational knowledge of the European and national legal framework for the management of research data access and use.

Objective

In this paper, we describe the compound process of implementation and the contents of a regional university hospital BDW.

Methods

We present the actions and challenges regarding organizational changes, technical architecture, and shared governance that took place to develop the Nantes BDW. We describe the process to access clinical contents, give details about patient data protection, and use examples to illustrate merging clinical insights.

Unlabelled

More than 68 million textual documents and 543 million pieces of coded information concerning approximately 1.5 million patients admitted to CHUN between 2002 and 2022 can be queried and transformed to be made available to investigators. Since its creation in 2018, 269 projects have benefited from the Nantes BDW. Access to data is organized according to data use and regulatory requirements.

Conclusions

Data use is entirely determined by the scientific question posed. It is the vector of legitimacy of data access for secondary use. Enabling access to a BDW is a game changer for research and all operational situations in need of data. Finally, data governance must prevail over technical issues in institution data strategy vis-à-vis care professionals and patients alike.

Forecasting acute kidney injury and resource utilization in ICU patients using longitudinal, multimodal models

BACKGROUND

Advances in artificial intelligence (AI) have realized the potential of revolutionizing healthcare, such as predicting disease progression via longitudinal inspection of Electronic Health Records (EHRs) and lab tests from patients admitted to Intensive Care Units (ICU). Although substantial literature exists addressing broad subjects, including the prediction of mortality, length-of-stay, and readmission, studies focusing on forecasting Acute Kidney Injury (AKI), specifically dialysis anticipation like Continuous Renal Replacement Therapy (CRRT) are scarce. The technicality of how to implement AI remains elusive.

OBJECTIVE

This study aims to elucidate the important factors and methods that are required to develop effective predictive models of AKI and CRRT for patients admitted to ICU, using EHRs in the Medical Information Mart for Intensive Care (MIMIC) database.

METHODS

We conducted a comprehensive comparative analysis of established predictive models, considering both time-series measurements and clinical notes from MIMIC-IV databases. Subsequently, we proposed a novel multi-modal model which integrates embeddings of top-performing unimodal models, including Long Short-Term Memory (LSTM) and BioMedBERT, and leverages both unstructured clinical notes and structured time series measurements derived from EHRs to enable the early prediction of AKI and CRRT.

RESULTS

Our multimodal model achieved a lead time of at least 12 h ahead of clinical manifestation, with an Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.888 for AKI and 0.997 for CRRT, as well as an Area Under the Precision Recall Curve (AUPRC) of 0.727 for AKI and 0.840 for CRRT, respectively, which significantly outperformed the baseline models. Additionally, we performed a SHapley Additive exPlanation (SHAP) analysis using the expected gradients algorithm, which highlighted important, previously underappreciated predictive features for AKI and CRRT.

CONCLUSION

Our study revealed the importance and the technicality of applying longitudinal, multimodal modeling to improve early prediction of AKI and CRRT, offering insights for timely interventions. The performance and interpretability of our model indicate its potential for further assessment towards clinical applications, to ultimately optimize AKI management and enhance patient outcomes.

Redefining Health Care Data Interoperability: Empirical Exploration of Large Language Models in Information Exchange

BACKGROUND

Efficient data exchange and health care interoperability are impeded by medical records often being in nonstandardized or unstructured natural language format. Advanced language models, such as large language models (LLMs), may help overcome current challenges in information exchange.

OBJECTIVE

This study aims to evaluate the capability of LLMs in transforming and transferring health care data to support interoperability.

METHODS

Using data from the Medical Information Mart for Intensive Care III and UK Biobank, the study conducted 3 experiments. Experiment 1 assessed the accuracy of transforming structured laboratory results into unstructured format. Experiment 2 explored the conversion of diagnostic codes between the coding frameworks of the ICD-9-CM (International Classification of Diseases, Ninth Revision, Clinical Modification), and Systematized Nomenclature of Medicine Clinical Terms (SNOMED-CT) using a traditional mapping table and a text-based approach facilitated by the LLM ChatGPT. Experiment 3 focused on extracting targeted information from unstructured records that included comprehensive clinical information (discharge notes).

RESULTS

The text-based approach showed a high conversion accuracy in transforming laboratory results (experiment 1) and an enhanced consistency in diagnostic code conversion, particularly for frequently used diagnostic names, compared with the traditional mapping approach (experiment 2). In experiment 3, the LLM showed a positive predictive value of 87.2% in extracting generic drug names.

CONCLUSIONS

This study highlighted the potential role of LLMs in significantly improving health care data interoperability, demonstrated by their high accuracy and efficiency in data transformation and exchange. The LLMs hold vast potential for enhancing medical data exchange without complex standardization for medical terms and data structure.

Prediction of aneurysmal subarachnoid hemorrhage in comparison with other stroke types using routine care data

Aneurysmal subarachnoid hemorrhage (aSAH) can be prevented by early detection and treatment of intracranial aneurysms in high-risk individuals. We investigated whether individuals at high risk of aSAH in the general population can be identified by developing an aSAH prediction model with electronic health records (EHR) data. To assess the aSAH model's relative performance, we additionally developed prediction models for acute ischemic stroke (AIS) and intracerebral hemorrhage (ICH) and compared the discriminative performance of the models. We included individuals aged ≥35 years without history of stroke from a Dutch routine care database (years 2007-2020) and defined outcomes aSAH, AIS and ICH using International Classification of Diseases (ICD) codes. Potential predictors included sociodemographic data, diagnoses, medications, and blood measurements. We cross-validated a Cox proportional hazards model with an elastic net penalty on derivation cohorts and reported the c-statistic and 10-year calibration on validation cohorts. We examined 1,040,855 individuals (mean age 54.6 years, 50.9% women) for a total of 10,173,170 person-years (median 11 years). 17,465 stroke events occurred during follow-up: 723 aSAH, 14,659 AIS, and 2,083 ICH. The aSAH model's c-statistic was 0.61 (95%CI 0.57-0.65), which was lower than the c-statistic of the AIS (0.77, 95%CI 0.77-0.78) and ICH models (0.77, 95%CI 0.75-0.78). All models were well-calibrated. The aSAH model identified 19 predictors, of which the 10 strongest included age, female sex, population density, socioeconomic status, oral contraceptive use, gastroenterological complaints, obstructive airway medication, epilepsy, childbirth complications, and smoking. Discriminative performance of the aSAH prediction model was moderate, while it was good for the AIS and ICH models. We conclude that it is currently not feasible to accurately identify individuals at increased risk for aSAH using EHR data.

Development and external validation of deep learning clinical prediction models using variable-length time series data

OBJECTIVES

To compare and externally validate popular deep learning model architectures and data transformation methods for variable-length time series data in 3 clinical tasks (clinical deterioration, severe acute kidney injury [AKI], and suspected infection).

MATERIALS AND METHODS

This multicenter retrospective study included admissions at 2 medical centers that spanned 2007-2022. Distinct datasets were created for each clinical task, with 1 site used for training and the other for testing. Three feature engineering methods (normalization, standardization, and piece-wise linear encoding with decision trees [PLE-DTs]) and 3 architectures (long short-term memory/gated recurrent unit [LSTM/GRU], temporal convolutional network, and time-distributed wrapper with convolutional neural network [TDW-CNN]) were compared in each clinical task. Model discrimination was evaluated using the area under the precision-recall curve (AUPRC) and the area under the receiver operating characteristic curve (AUROC).

RESULTS

The study comprised 373 825 admissions for training and 256 128 admissions for testing. LSTM/GRU models tied with TDW-CNN models with both obtaining the highest mean AUPRC in 2 tasks, and LSTM/GRU had the highest mean AUROC across all tasks (deterioration: 0.81, AKI: 0.92, infection: 0.87). PLE-DT with LSTM/GRU achieved the highest AUPRC in all tasks.

DISCUSSION

When externally validated in 3 clinical tasks, the LSTM/GRU model architecture with PLE-DT transformed data demonstrated the highest AUPRC in all tasks. Multiple models achieved similar performance when evaluated using AUROC.

CONCLUSION

The LSTM architecture performs as well or better than some newer architectures, and PLE-DT may enhance the AUPRC in variable-length time series data for predicting clinical outcomes during external validation.

MixEHR-SurG: A joint proportional hazard and guided topic model for inferring mortality-associated topics from electronic health records

Survival models can help medical practitioners to evaluate the prognostic importance of clinical variables to patient outcomes such as mortality or hospital readmission and subsequently design personalized treatment regimes. Electronic Health Records (EHRs) hold the promise for large-scale survival analysis based on systematically recorded clinical features for each patient. However, existing survival models either do not scale to high dimensional and multi-modal EHR data or are difficult to interpret. In this study, we present a supervised topic model called MixEHR-SurG to simultaneously integrate heterogeneous EHR data and model survival hazard. Our contributions are three-folds: (1) integrating EHR topic inference with Cox proportional hazards likelihood; (2) integrating patient-specific topic hyperparameters using the PheCode concepts such that each topic can be identified with exactly one PheCode-associated phenotype; (3) multi-modal survival topic inference. This leads to a highly interpretable survival topic model that can infer PheCode-specific phenotype topics associated with patient mortality. We evaluated MixEHR-SurG using a simulated dataset and two real-world EHR datasets: the Quebec Congenital Heart Disease (CHD) data consisting of 8211 subjects with 75,187 outpatient claim records of 1767 unique ICD codes; the MIMIC-III consisting of 1458 subjects with multi-modal EHR records. Compared to the baselines, MixEHR-SurG achieved a superior dynamic AUROC for mortality prediction, with a mean AUROC score of 0.89 in the simulation dataset and a mean AUROC of 0.645 on the CHD dataset. Qualitatively, MixEHR-SurG associates severe cardiac conditions with high mortality risk among the CHD patients after the first heart failure hospitalization and critical brain injuries with increased mortality among the MIMIC-III patients after their ICU discharge. Together, the integration of the Cox proportional hazards model and EHR topic inference in MixEHR-SurG not only leads to competitive mortality prediction but also meaningful phenotype topics for in-depth survival analysis. The software is available at GitHub: https://github.com/li-lab-mcgill/MixEHR-SurG.

Decision discovery using clinical decision support system decision log data for supporting the nurse decision-making process

BACKGROUND

Decision-making in healthcare is increasingly complex; notably in hospital environments where the information density is high, e.g., emergency departments, oncology departments, and psychiatry departments. This study aims to discover decisions from logged data to improve the decision-making process.

METHODS

The Design Science Research Methodology (DSRM) was chosen to design an artifact (algorithm) for the discovery and visualization of decisions. The DSRM's different activities are explained, from the definition of the problem to the evaluation of the artifact. During the design and development activities, the algorithm itself is created. During the demonstration and evaluation activities, the algorithm was tested with an authentic synthetic dataset.

RESULTS

The results show the design and simulation of an algorithm for the discovery and visualization of decisions. A fuzzy classifier algorithm was adapted for (1) discovering decisions from a decision log and (2) visualizing the decisions using the Decision Model and Notation standard.

CONCLUSIONS

In this paper, we show that decisions can be discovered from a decision log and visualized for the improvement of the decision-making process of healthcare professionals or to support the periodic evaluation of protocols and guidelines.

The OMOP common data model in Australian primary care data: Building a quality research ready harmonised dataset

BACKGROUND

The use of routinely collected health data for secondary research purposes is increasingly recognised as a methodology that advances medical research, improves patient outcomes, and guides policy. This secondary data, as found in electronic medical records (EMRs), can be optimised through conversion into a uniform data structure to enable analysis alongside other comparable health metric datasets. This can be achieved with the Observational Medical Outcomes Partnership Common Data Model (OMOP-CDM), which employs a standardised vocabulary to facilitate systematic analysis across various observational databases. The concept behind the OMOP-CDM is the conversion of data into a common format through the harmonisation of terminologies, vocabularies, and coding schemes within a unique repository. The OMOP model enhances research capacity through the development of shared analytic and prediction techniques; pharmacovigilance for the active surveillance of drug safety; and 'validation' analyses across multiple institutions across Australia, the United States, Europe, and the Asia Pacific. In this research, we aim to investigate the use of the open-source OMOP-CDM in the PATRON primary care data repository.

METHODS

We used standard structured query language (SQL) to construct, extract, transform, and load scripts to convert the data to the OMOP-CDM. The process of mapping distinct free-text terms extracted from various EMRs presented a substantial challenge, as many terms could not be automatically matched to standard vocabularies through direct text comparison. This resulted in a number of terms that required manual assignment. To address this issue, we implemented a strategy where our clinical mappers were instructed to focus only on terms that appeared with sufficient frequency. We established a specific threshold value for each domain, ensuring that more than 95% of all records were linked to an approved vocabulary like SNOMED once appropriate mapping was completed. To assess the data quality of the resultant OMOP dataset we utilised the OHDSI Data Quality Dashboard (DQD) to evaluate the plausibility, conformity, and comprehensiveness of the data in the PATRON repository according to the Kahn framework.

RESULTS

Across three primary care EMR systems we converted data on 2.03 million active patients to version 5.4 of the OMOP common data model. The DQD assessment involved a total of 3,570 individual evaluations. Each evaluation compared the outcome against a predefined threshold. A 'FAIL' occurred when the percentage of non-compliant rows exceeded the specified threshold value. In this assessment of the primary care OMOP database described here, we achieved an overall pass rate of 97%.

CONCLUSION

The OMOP CDM's widespread international use, support, and training provides a well-established pathway for data standardisation in collaborative research. Its compatibility allows the sharing of analysis packages across local and international research groups, which facilitates rapid and reproducible data comparisons. A suite of open-source tools, including the OHDSI Data Quality Dashboard (Version 1.4.1), supports the model. Its simplicity and standards-based approach facilitates adoption and integration into existing data processes.

FIT calculator: a multi-risk prediction framework for medical outcomes using cardiorespiratory fitness data

Accurately predicting patients' risk for specific medical outcomes is paramount for effective healthcare management and personalized medicine. While a substantial body of literature addresses the prediction of diverse medical conditions, existing models predominantly focus on singular outcomes, limiting their scope to one disease at a time. However, clinical reality often entails patients concurrently facing multiple health risks across various medical domains. In response to this gap, our study proposes a novel multi-risk framework adept at simultaneous risk prediction for multiple clinical outcomes, including diabetes, mortality, and hypertension. Leveraging a concise set of features extracted from patients' cardiorespiratory fitness data, our framework minimizes computational complexity while maximizing predictive accuracy. Moreover, we integrate a state-of-the-art instance-based interpretability technique into our framework, providing users with comprehensive explanations for each prediction. These explanations afford medical practitioners invaluable insights into the primary health factors influencing individual predictions, fostering greater trust and utility in the underlying prediction models. Our approach thus stands to significantly enhance healthcare decision-making processes, facilitating more targeted interventions and improving patient outcomes in clinical practice. Our prediction framework utilizes an automated machine learning framework, Auto-Weka, to optimize machine learning models and hyper-parameter configurations for the simultaneous prediction of three medical outcomes: diabetes, mortality, and hypertension. Additionally, we employ a local interpretability technique to elucidate predictions generated by our framework. These explanations manifest visually, highlighting key attributes contributing to each instance's prediction for enhanced interpretability. Using automated machine learning techniques, the models simultaneously predict hypertension, mortality, and diabetes risks, utilizing only nine patient features. They achieved an average AUC of 0.90 ± 0.001 on the hypertension dataset, 0.90 ± 0.002 on the mortality dataset, and 0.89 ± 0.001 on the diabetes dataset through tenfold cross-validation. Additionally, the models demonstrated strong performance with an average AUC of 0.89 ± 0.001 on the hypertension dataset, 0.90 ± 0.001 on the mortality dataset, and 0.89 ± 0.001 on the diabetes dataset using bootstrap evaluation with 1000 resamples.