Big Data Management for Healthcare Systems: Architecture, Requirements, and Implementation

Factors influencing the use of big data within healthcare services: a systematic review

Background: The emergence of big data holds the promise of aiding healthcare providers by identifying patterns and converting vast quantities of data into actionable insights facilitating the provision of precision medicine and decision-making. Objective: This study aimed to investigate the factors influencing use of big data within healthcare services to facilitate their use. Method: A systematic review was conducted in February 2024, adhering to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Database searches for articles published between 01 January 2020 and 18 February 2024 and included PubMed, Scopus, ProQuest and Cochrane Library. The Authority, Accuracy, Coverage, Objectivity, Date, Significance ( AACODS) checklist was used to evaluate the quality of the included articles. Subsequently, a thematic analysis was conducted on the findings of the review, using the Boyatzis approach. Results: A final selection of 46 studies were included in this systematic review. A significant proportion of these studies demonstrated acceptable quality, and the level of bias was deemed satisfactory. Thematic analysis identified seven major themes that influenced the use of big data in healthcare services. These themes were grouped into four primary categories: performance expectancy, effort expectancy, social influence, and facilitating conditions. Factors associated with "effort expectancy" were the most highly cited in the included studies (67%), while those related to "social influence" received the fewest citations (15%). Conclusion: This study underscored the critical role of "effort expectancy" factors, particularly those under the theme of "data complexity and management," in the process of using big data in healthcare services. Implications: Results of this study provide groundwork for future research to explore facilitators and barriers to using big data in health care, particularly in relation to data complexity and the efficient and effective management of big data, with significant implications for healthcare administrators and policymakers.

Breaking Digital Health Barriers: Development and Validation of an LLM-Based Tool for Automated OMOP Mapping

BACKGROUND

The integration of diverse clinical data sources requires standardization through models like OMOP (Observational Medical Outcomes Partnership). However, mapping data elements to OMOP concepts demands significant technical expertise and time. While large healthcare systems often have resources for OMOP conversion, smaller clinical trials and studies frequently lack such support, leaving valuable research data siloed.

OBJECTIVE

To develop and validate a user-friendly tool that leverages large language models to automate the OMOP conversion process for clinical trial, electronic health record, and registry data.

METHODS

We developed a three-tiered semantic matching system using GPT-3 embeddings to transform heterogeneous clinical data to the OMOP common data model. The system processes input terms by generating vector embeddings, computing cosine similarity against precomputed OHDSI vocabulary embeddings, and ranking potential matches. We validated the system using two independent datasets: a development set of 76 NIH HEAL Initiative clinical trial common data elements (CDEs) for chronic pain and opioid use disorders, and a separate validation set of electronic health record concepts from the NIH N3C COVID-19 enclave. The architecture combines UMLS semantic frameworks with asynchronous processing for efficient concept mapping, made available through an open-source implementation.

RESULTS

The system achieved an Area Under the Receiver Operating Characteristic Curve (AUC) of 0.9975 for mapping clinical trial CDE terms. Precision ranged from 0.92 to 0.99 and recall from 0.88 to 0.97 across similarity thresholds from 0.85 to 1.0. In practical application, the tool successfully automated mappings that previously required manual informatics expertise, reducing the technical barriers for research teams to participate in large-scale data sharing initiatives. Representative mappings demonstrated high accuracy, such as demographic terms achieving 100% similarity with corresponding LOINC concepts. The implementation successfully processes diverse data types through both individual term mapping and batch processing capabilities.

CONCLUSIONS

Our validated LLM-based tool effectively automates the transformation of clinical data into OMOP format while maintaining high accuracy. The combination of semantic matching capabilities and researcher-friendly interface makes data harmonization accessible to smaller research teams without requiring extensive informatics support. This has direct implications for accelerating clinical research data standardization and enabling broader participation in initiatives like the NIH HEAL Data Ecosystem.

CLINICALTRIAL

Artificial intelligence in precision medicine for lung cancer: A bibliometric analysis

Background

The increasing body of evidence has been stimulating the application of artificial intelligence (AI) in precision medicine research for lung cancer. This trend necessitates a comprehensive overview of the growing number of publications to facilitate researchers' understanding of this field.

Method

The bibliometric data for the current analysis was extracted from the Web of Science Core Collection database, CiteSpace, VOSviewer ,and an online website were applied to the analysis.

Results

After the data were filtered, this search yielded 4062 manuscripts. And 92.27% of the papers were published from 2014 onwards. The main contributing countries were China, the United States, India, Japan, and Korea. These publications were mainly published in the following scientific disciplines, including Radiology Nuclear Medicine, Medical Imaging, Oncology, and Computer Science Notably, Li Weimin and Aerts Hugo J. W. L. stand out as leading authorities in this domain. In the keyword co-occurrence and co-citation cluster analysis of the publication, the knowledge base was divided into four clusters that are more easily understood, including screening, diagnosis, treatment, and prognosis.

Conclusion

This bibliometric study reveals deep learning frameworks and AI-based radiomics are receiving attention. High-quality and standardized data have the potential to revolutionize lung cancer screening and diagnosis in the era of precision medicine. However, the importance of high-quality clinical datasets, the development of new and combined AI models, and their consistent assessment for advancing research on AI applications in lung cancer are highlighted before current research can be effectively applied in clinical practice.

A systems approach to developing user requirements for increased pulmonary rehabilitation uptake by COPD patients

Chronic Obstructive Pulmonary Disease is a progressive lung disease associated with anxiety, depression, and reduced health-related quality of life. Pulmonary rehabilitation (PR) is a cost-effective and transformative treatment, but 31% of referred patients do not take up their PR appointment. The study aimed to develop user requirements for an intervention to increase PR uptake. A systems approach, the Engineering Better Care framework, was used to develop a system map of the PR pathway, translate evidence-based user needs into user requirements, and validate the user requirements in a stakeholder workshop. Eight user requirements addressed patient and health care practitioner needs to understand what PR entails, understand the benefits of PR and have positive conversations about PR to address patient concerns. The solution-independent user requirements can be applied to the development of any intervention sharing similar goals. The study demonstrates potential in taking a systems approach to more challenges within respiratory medicine.

An integrated machine learning predictive scheme for longitudinal laboratory data to evaluate the factors determining renal function changes in patients with different chronic kidney disease stages

Background and objectives

Chronic kidney disease (CKD) is a global health concern. This study aims to identify key factors associated with renal function changes using the proposed machine learning and important variable selection (ML&IVS) scheme on longitudinal laboratory data. The goal is to predict changes in the estimated glomerular filtration rate (eGFR) in a cohort of patients with CKD stages 3-5.

Design

A retrospective cohort study.

Setting and participants

A total of 710 outpatients who presented with stable nondialysis-dependent CKD stages 3-5 at the Shin-Kong Wu Ho-Su Memorial Hospital Medical Center from 2016 to 2021.

Methods

This study analyzed trimonthly laboratory data including 47 indicators. The proposed scheme used stochastic gradient boosting, multivariate adaptive regression splines, random forest, eXtreme gradient boosting, and light gradient boosting machine algorithms to evaluate the important factors for predicting the results of the fourth eGFR examination, especially in patients with CKD stage 3 and those with CKD stages 4-5, with or without diabetes mellitus (DM).

Main outcome measurement

Subsequent eGFR level after three consecutive laboratory data assessments.

Results

Our ML&IVS scheme demonstrated superior predictive capabilities and identified significant factors contributing to renal function changes in various CKD groups. The latest levels of eGFR, blood urea nitrogen (BUN), proteinuria, sodium, and systolic blood pressure as well as mean levels of eGFR, BUN, proteinuria, and triglyceride were the top 10 significantly important factors for predicting the subsequent eGFR level in patients with CKD stages 3-5. In individuals with DM, the latest levels of BUN and proteinuria, mean levels of phosphate and proteinuria, and variations in diastolic blood pressure levels emerged as important factors for predicting the decline of renal function. In individuals without DM, all phosphate patterns and latest albumin levels were found to be key factors in the advanced CKD group. Moreover, proteinuria was identified as an important factor in the CKD stage 3 group without DM and CKD stages 4-5 group with DM.

Conclusion

The proposed scheme highlighted factors associated with renal function changes in different CKD conditions, offering valuable insights to physicians for raising awareness about renal function changes.

Shifting machine learning for healthcare from development to deployment and from models to data

In the past decade, the application of machine learning (ML) to healthcare has helped drive the automation of physician tasks as well as enhancements in clinical capabilities and access to care. This progress has emphasized that, from model development to model deployment, data play central roles. In this Review, we provide a data-centric view of the innovations and challenges that are defining ML for healthcare. We discuss deep generative models and federated learning as strategies to augment datasets for improved model performance, as well as the use of the more recent transformer models for handling larger datasets and enhancing the modelling of clinical text. We also discuss data-focused problems in the deployment of ML, emphasizing the need to efficiently deliver data to ML models for timely clinical predictions and to account for natural data shifts that can deteriorate model performance.

Perspective Toward Machine Learning Implementation in Pediatric Medicine: Mixed Methods Study

Background

Given the costs of machine learning implementation, a systematic approach to prioritizing which models to implement into clinical practice may be valuable.

Objective

The primary objective was to determine the health care attributes respondents at 2 pediatric institutions rate as important when prioritizing machine learning model implementation. The secondary objective was to describe their perspectives on implementation using a qualitative approach.

Methods

In this mixed methods study, we distributed a survey to health system leaders, physicians, and data scientists at 2 pediatric institutions. We asked respondents to rank the following 5 attributes in terms of implementation usefulness: the clinical problem was common, the clinical problem caused substantial morbidity and mortality, risk stratification led to different actions that could reasonably improve patient outcomes, reducing physician workload, and saving money. Important attributes were those ranked as first or second most important. Individual qualitative interviews were conducted with a subsample of respondents.

Results

Among 613 eligible respondents, 275 (44.9%) responded. Qualitative interviews were conducted with 17 respondents. The most common important attributes were risk stratification leading to different actions (205/275, 74.5%) and clinical problem causing substantial morbidity or mortality (177/275, 64.4%). The attributes considered least important were reducing physician workload and saving money. Qualitative interviews consistently prioritized implementations that improved patient outcomes.

Conclusions

Respondents prioritized machine learning model implementation where risk stratification would lead to different actions and clinical problems that caused substantial morbidity and mortality. Implementations that improved patient outcomes were prioritized. These results can help provide a framework for machine learning model implementation.

An Efficient and Privacy-Preserving Scheme for Disease Prediction in Modern Healthcare Systems

With the Internet of Things (IoT), mobile healthcare applications can now offer a variety of dimensionalities and online services. Disease Prediction Systems (DPS) increase the speed and accuracy of diagnosis, improving the quality of healthcare services. However, privacy is garnering an increasing amount of attention these days, especially concerning personal healthcare data, which are sensitive. There are a variety of prevailing privacy preservation techniques for disease prediction that are rendered. Nonetheless, there is a chance of medical users being affected by numerous disparate diseases. Therefore, it is vital to consider multi-label instances, which might decrease the accuracy. Thus, this paper proposes an efficient privacy-preserving (PP) scheme for patient healthcare data collected from IoT devices aimed at disease prediction in the modern Health Care System (HCS). The proposed system utilizes the Log of Round value-based Elliptic Curve Cryptography (LR-ECC) to enhance the security level during data transfer after the initial authentication phase. The authorized healthcare staff can securely download the patient data on the hospital side. Utilizing the Herding Genetic Algorithm-based Deep Learning Neural Network (EHGA-DLNN) can test these data with the trained system to predict the diseases. The experimental results demonstrate that the proposed approach improves prediction accuracy, privacy, and security compared to the existing methods.

Software Measurement by Using Artificial Intelligence

Artificial intelligence (AI) is a subfield of computer science concerned with developing intelligent machines capable of performing tasks similar to those performed by humans. This human-created intelligence began more than 60 years ago. The goal of previous generations of applications was to demonstrate generic human-like behaviour. The goal has expanded with the advancement and increased compliance of this technology. It includes areas such as healthcare, gaming, and smart devices. The COVID-19 epidemic has posed a significant barrier to maintaining a sustainable strategy for mental health support clients with major mental illnesses and clinicians who have had to shift delivery modes quickly. In this study, we have conducted a systematic literature review (SLR) to provide an overview of the current state of the literature related to software measurement of healthcare using artificial intelligence. The study followed a secondary research strategy. The systematic literature review aim was to analyze software measurement of mental health illness in terms of previous literature. This study screened out of 28 research papers out of 1076 initial searches. We used Science Direct, IEEE Xplore, Springer Link, ACM, and Hindawi as database search engines. The research objective was to explore the needs of software applications and automation in the healthcare sector to bring efficiency to the systems. The research concluded that the healthcare setting crucially requires the implementation of software automation.

Artificial intelligence: opportunities in lung cancer

PURPOSE OF REVIEW

In this article, we focus on the role of artificial intelligence in the management of lung cancer. We summarized commonly used algorithms, current applications and challenges of artificial intelligence in lung cancer.

RECENT FINDINGS

Feature engineering for tabular data and computer vision for image data are commonly used algorithms in lung cancer research. Furthermore, the use of artificial intelligence in lung cancer has extended to the entire clinical pathway including screening, diagnosis and treatment. Lung cancer screening mainly focuses on two aspects: identifying high-risk populations and the automatic detection of lung nodules. Artificial intelligence diagnosis of lung cancer covers imaging diagnosis, pathological diagnosis and genetic diagnosis. The artificial intelligence clinical decision-support system is the main application of artificial intelligence in lung cancer treatment. Currently, the challenges of artificial intelligence applications in lung cancer mainly focus on the interpretability of artificial intelligence models and limited annotated datasets; and recent advances in explainable machine learning, transfer learning and federated learning might solve these problems.

SUMMARY

Artificial intelligence shows great potential in many aspects of the management of lung cancer, especially in screening and diagnosis. Future studies on interpretability and privacy are needed for further application of artificial intelligence in lung cancer.

Big data in the healthcare system: a synergy with artificial intelligence and blockchain technology

In the last decades big data has facilitating and improving our daily duties in the medical research and clinical fields; the strategy to get to this point is understanding how to organize and analyze the data in order to accomplish the final goal that is improving healthcare system, in terms of cost and benefits, quality of life and outcome patient. The main objective of this review is to illustrate the state-of-art of big data in healthcare, its features and architecture. We also would like to demonstrate the different application and principal mechanisms of big data in the latest technologies known as blockchain and artificial intelligence, recognizing their benefits and limitations. Perhaps, medical education and digital anatomy are unexplored fields that might be profitable to investigate as we are proposing. The healthcare system can be revolutionized using these different technologies. Thus, we are explaining the basis of these systems focused to the medical arena in order to encourage medical doctors, nurses, biotechnologies and other healthcare professions to be involved and create a more efficient and efficacy system.

Changing Health-Related Behaviors 6: Analysis, Interpretation, and Application of Big Data

The behavior of individuals can affect both their own health and the health of those around them. Furthermore, the behavior of healthcare providers obviously affects the health of those receiving the care. In both of these cases, and in spite of its known benefits, behavior change is difficult for most people. To make change easier, big data can provide insight through an objective and nonjudgmental perspective. It may also help make specific, individualized, evidence-based recommendations for effective change. We provide a historical perspective on data and health and then describe the value of adding big data systems and how they are implemented. We discuss some of the sources of big data and how it is collected. We also review the additional challenges for analysis, interpretation, and application of big data that require specific technologies. We end with a summary of current uses of big data for behavior change and suggestions for additional approaches, which may be of benefit.

Biomedical Image Classification in a Big Data Architecture Using Machine Learning Algorithms

In modern-day medicine, medical imaging has undergone immense advancements and can capture several biomedical images from patients. In the wake of this, to assist medical specialists, these images can be used and trained in an intelligent system in order to aid the determination of the different diseases that can be identified from analyzing these images. Classification plays an important role in this regard; it enhances the grouping of these images into categories of diseases and optimizes the next step of a computer-aided diagnosis system. The concept of classification in machine learning deals with the problem of identifying to which set of categories a new population belongs. When category membership is known, the classification is done on the basis of a training set of data containing observations. The goal of this paper is to perform a survey of classification algorithms for biomedical images. The paper then describes how these algorithms can be applied to a big data architecture by using the Spark framework. This paper further proposes the classification workflow based on the observed optimal algorithms, Support Vector Machine and Deep Learning as drawn from the literature. The algorithm for the feature extraction step during the classification process is presented and can be customized in all other steps of the proposed classification workflow.

Artificial intelligence in urological oncology: An update and future applications

There continues to be rapid developments and research in the field of Artificial Intelligence (AI) in Urological Oncology worldwide. In this review we discuss the basics of AI, application of AI per tumour group (Renal, Prostate and Bladder Cancer) and application of AI in Robotic Urological Surgery. We also discuss future applications of AI being developed with the benefits to patients with Urological Oncology.

Development and utility assessment of a machine learning bloodstream infection classifier in pediatric patients receiving cancer treatments

Background

Objectives were to build a machine learning algorithm to identify bloodstream infection (BSI) among pediatric patients with cancer and hematopoietic stem cell transplantation (HSCT) recipients, and to compare this approach with presence of neutropenia to identify BSI.

Methods

We included patients 0–18 years of age at cancer diagnosis or HSCT between January 2009 and November 2018. Eligible blood cultures were those with no previous blood culture (regardless of result) within 7 days. The primary outcome was BSI. Four machine learning algorithms were used: elastic net, support vector machine and two implementations of gradient boosting machine (GBM and XGBoost). Model training and evaluation were performed using temporally disjoint training (60%), validation (20%) and test (20%) sets. The best model was compared to neutropenia alone in the test set.

Results

Of 11,183 eligible blood cultures, 624 (5.6%) were positive. The best model in the validation set was GBM, which achieved an area-under-the-receiver-operator-curve (AUROC) of 0.74 in the test set. Among the 2236 in the test set, the number of false positives and specificity of GBM vs. neutropenia were 508 vs. 592 and 0.76 vs. 0.72 respectively. Among 139 test set BSIs, six (4.3%) non-neutropenic patients were identified by GBM. All received antibiotics prior to culture result availability.

Conclusions

We developed a machine learning algorithm to classify BSI. GBM achieved an AUROC of 0.74 and identified 4.3% additional true cases in the test set. The machine learning algorithm did not perform substantially better than using presence of neutropenia alone to predict BSI.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12885-020-07618-2.

Fuzzy-Based Symmetrical Multi-Criteria Decision-Making Procedure for Evaluating the Impact of Harmful Factors of Healthcare Information Security

Growing concern about healthcare information security in the wake of alarmingly rising cyber-attacks is being given symmetrical priority by current researchers and cyber security experts. Intruders are penetrating symmetrical mechanisms of healthcare information security continuously. In the same league, the paper presents an overview on the current situation of healthcare information and presents a layered model of healthcare information management in organizations. The paper also evaluates the various factors that have a key contribution in healthcare information security breaches through a hybrid fuzzy-based symmetrical methodology of AHP-TOPSIS. Furthermore, for assessing the effect of the calculated results, the authors have tested the results on local hospital software of Varanasi. Tested results of the factors are validated through the comparison and sensitivity analysis in this study. Tabulated results of the proposed study propose a symmetrical mechanism as the most conversant technique which can be employed by the experts and researchers for preparing security guidelines and strategies.

A Hadoop-Based Platform for Patient Classification and Disease Diagnosis in Healthcare Applications

Nowadays, the increasing number of patients accompanied with the emergence of new symptoms and diseases makes heath monitoring and assessment a complicated task for medical staff and hospitals. Indeed, the processing of big and heterogeneous data collected by biomedical sensors along with the need of patients’ classification and disease diagnosis become major challenges for several health-based sensing applications. Thus, the combination between remote sensing devices and the big data technologies have been proven as an efficient and low cost solution for healthcare applications. In this paper, we propose a robust big data analytics platform for real time patient monitoring and decision making to help both hospital and medical staff. The proposed platform relies on big data technologies and data analysis techniques and consists of four layers: real time patient monitoring, real time decision and data storage, patient classification and disease diagnosis, and data retrieval and visualization. To evaluate the performance of our platform, we implemented our platform based on the Hadoop ecosystem and we applied the proposed algorithms over real health data. The obtained results show the effectiveness of our platform in terms of efficiently performing patient classification and disease diagnosis in healthcare applications.

Development and Usability Testing of an Emergency Alert Device for Elderly People and People with Disabilities

The objectives of this study were to develop and evaluate the effectiveness of an emergency alert device for elderly people and people with disabilities by usability testing. There were two phases in this study: (1) development of a prototype for an emergency alert device and (2) usability testing of the device. Results presented development of the prototype, which comprised parts for sending and receiving signals. There were two kinds of alarms for emergency calls known as conscious and unconscious alerts. Participants in the usability testing phase included 12 specialists and 161 users that comprised 146 elderly people or people with disabilities and 15 caregivers or community health volunteers. The instruments used were a rating scale, usability checklist, and individual interviews regarding the usability, general appearance, and use of the device. The users agreed with the overall aspects regarding usability of the device, its general appearance, and use (X¯ ± SD = 4.24 ± 0.88, 4.11 ± 0.90, and 4.37 ± 0.83, respectively). Most of the participants, both specialists and users, gave their perspectives on improving the size, color of the letters displayed, type of wristband, and method for sending signals.

Big Data Analytics and Processing Platform in Czech Republic Healthcare

Big data analytics (BDA) in healthcare has made a positive difference in the integration of Artificial Intelligence (AI) in advancements of analytical capabilities, while lowering the costs of medical care. The aim of this study is to improve the existing healthcare eSystem by implementing a Big Data Analytics (BDA) platform and to meet the requirements of the Czech Republic National Health Service (Tender-Id. VZ0036628, No. Z2017-035520). In addition to providing analytical capabilities on Linux platforms supporting current and near-future AI with machine-learning and data-mining algorithms, there is the need for ethical considerations mandating new ways to preserve privacy, all of which are preconditioned by the growing body of regulations and expectations. The presented BDA platform, has met all requirements (N > 100), including the healthcare industry-standard Transaction Processing Performance Council (TPC-H) decision support benchmark in compliance with the European Union (EU) and the Czech Republic legislations. Currently, the presented Proof of Concept (PoC) that has been upgraded to a production environment has unified isolated parts of Czech Republic healthcare over the past seven months. The reported PoC BDA platform, artefacts, and concepts are transferrable to healthcare systems in other countries interested in developing or upgrading their own national healthcare infrastructure in a cost-effective, secure, scalable and high-performance manner.

Wearables Meet IoT: Synergistic Personal Area Networks (SPANs)

Wearable monitoring and mobile health (mHealth) revolutionized healthcare diagnostics and delivery, while the exponential increase of deployed "things" in the Internet of things (IoT) transforms our homes and industries. "Things" with embedded activity and vital sign sensors that we refer to as "smart stuff" can interact with wearable and ambient sensors. A dynamic, ad-hoc personal area network can span multiple domains and facilitate processing in synergistic personal area networks-SPANs. The synergy of information from multiple sensors can provide: (a) New information that cannot be generated from existing data alone, (b) user identification, (c) more robust assessment of physiological signals, and (d) automatic annotation of events/records. In this paper, we present possible new applications of SPANs and results of feasibility studies. Preliminary tests indicate that users interact with smart stuff-in our case, a smart water bottle-dozens of times a day and sufficiently long to collect vital signs of the users. Synergistic processing of sensors from the smartwatch and objects of everyday use may provide user identification and assessment of new parameters that individual sensors could not generate, such as pulse wave velocity (PWV) and blood pressure. As a result, SPANs facilitate seamless monitoring and annotation of vital signs dozens of times per day, every day, every time the smart object is used, without additional setup of sensors and initiation of measurements. SPANs creates a dynamic "opportunistic bubble" for ad-hoc integration with other sensors of interest around the user, wherever they go. Continuous long-term monitoring of user's activity and vital signs can provide better diagnostic procedures and personalized feedback to motivate a proactive approach to health and wellbeing.

Decision-Making based on Big Data Analytics for People Management in Healthcare Organizations

Big data analytics enables large-scale data sets integration, supporting people management decisions, and cost-effectiveness evaluation of healthcare organizations. The purpose of this article is to address the decision-making process based on big data analytics in Healthcare organizations, to identify main big data analytics able to support healthcare leaders' decisions and to present some strategies to enhance efficiency along the healthcare value chain. Our research was based on a systematic review. During the literature review, we will be presenting as well the different applications of big data in the healthcare context and a proposal for a predictive model for people management processes. Our research underlines the importance big data analytics can add to the efficiency of the decision-making process, through a predictive model and real-time analytics, assisting in the collection, management, and integration of data in healthcare organizations.