Medical data quality assessment: On the development of an automated framework for medical data curation

Issues and Limitations on the Road to Fair and Inclusive AI Solutions for Biomedical Challenges

OBJECTIVE

In this paper, we explore the correlation between performance reporting and the development of inclusive AI solutions for biomedical problems. Our study examines the critical aspects of bias and noise in the context of medical decision support, aiming to provide actionable solutions. Contributions: A key contribution of our work is the recognition that measurement processes introduce noise and bias arising from human data interpretation and selection. We introduce the concept of "noise-bias cascade" to explain their interconnected nature. While current AI models handle noise well, bias remains a significant obstacle in achieving practical performance in these models. Our analysis spans the entire AI development lifecycle, from data collection to model deployment.

RECOMMENDATIONS

To effectively mitigate bias, we assert the need to implement additional measures such as rigorous study design; appropriate statistical analysis; transparent reporting; and diverse research representation. Furthermore, we strongly recommend the integration of uncertainty measures during model deployment to ensure the utmost fairness and inclusivity. These comprehensive recommendations aim to minimize both bias and noise, thereby improving the performance of future medical decision support systems.

Leveraging healthcare professionals' insights to enhance data quality in medical big data platforms: A qualitative study

Objective

This study aims to explore the awareness, attitudes, and actual usage of medical big data platforms among healthcare professionals to provide practical guidance and theoretical support for improving data quality for the development of medical informatization.

Method

Semistructured interviews were conducted with 19 doctors and nurses from a tertiary hospital in Wuhan City between April and June 2024.

Results

The analysis yielded seven major themes and nine subthemes: cognitive status, Value of Medical Big Data Platforms, data trust (subjective data, objective data), purposes of data recording (patient condition observation, self-protection, task completion), practical challenges (conflict between work purposes and recording requirements, inconsistent departmental training standards, influence of leadership style and team culture), standardization of data recording, and concerns about data privacy.

Conclusion

The insufficient understanding of medical big data platforms among healthcare professionals affects the quality of data recording and research value, emphasizing the necessity of strengthening training, standardizing data recording, providing technical support, and ensuring data security.

Pregnancy-Associated Plasma Protein A (PAPP-A) as a Predictor of Third Trimester Obesity: Insights from the CRIOBES Project

Introduction: Our objective in this article was to develop a predictive model for obesity in the third trimester of pregnancy using the plasma and clinical biomarkers that are managed within the Chromosomopathies Programme in the Andalusian Public Healthcare System. Methods: The epidemiological design was observational, of the unmatched case-control type. The geographical environment was the Seville Primary Healthcare District (DSAP Sevilla). The information was collected between 2011 and 2021. The reference cohort consisted of women who had carried a pregnancy to term. The variables and biomarkers studied correspond to those managed within the primary-care Pregnancy Integrated Care Pathway (ICP). Unconditional binary logistic regression (BLR) models were created, with the outcome variable being whether or not the women were obese in their third trimester of pregnancy. Results: A total of 423 controls and 104 cases of obesity were obtained for women in their third trimester who had not been obese in their first trimester. The average age for the sample group (P50) was 34 years old. The final, most parsimonious model included the variables PAPP-A (p = 0.074), beta-hCG (p = 0.1631), and systolic blood pressure (SBP) (p = 0.085). ROC curve = 0.75 (C.I. at 95%: 0.63-0.86). Discussion: The results of this research can only be extrapolated to primary care and to pregnancies with no complications. PAPP-A has been shown in our research to be a significant predictor of obesity risk in the third trimester of pregnancies with no complications (OR = 0.53; C.I. at 95%: 0.39-0.66; p = 0.04 in the single-variant study; OR = 0.58; C.I. at 95%: 0.29-0.93; p = 0.074 in the multi-variant analysis). This predictive capacity is further enhanced from an operational perspective by beta-hCG and 12-week SBP.

Identification and evolution of predictors of Sjögren's disease-associated mucosa-associated lymphoid tissue lymphoma development over time: a case-control study

BACKGROUND

Non-Hodgkin lymphomas have a substantial impact on individuals with Sjögren's disease. This study focuses on mucosal-associated lymphoid tissue (MALT) lymphomas, which constitute the majority of Sjögren's disease-associated non-Hodgkin lymphomas. We aimed to identify reliable lymphoma predictors in patients with Sjögren's disease and study their progression over time.

METHODS

In this case-control study, patients diagnosed with Sjögren's disease-associated MALT lymphoma, with a minimum of 3 years between Sjögren's disease diagnosis and MALT lymphoma diagnosis, were included from three centres specialising in Sjögren's disease (University of Athens, Athens, Greece; University of Pisa, Pisa, Italy; and University of Udine, Udine, Italy) and matched 1:1 with control participants with Sjögren's disease who did not have lymphoma according to age, sex, disease duration at last follow up, and treatment modality. Three harmonised datasets were constructed, curated, and analysed to identify MALT lymphoma predictors, representing three distinct timepoints in lymphomagenesis progression: V1 at Sjögren's disease diagnosis, V2 3-4 years before lymphoma diagnosis, and V3 0·5-1·5 years before lymphoma diagnosis. All recruited patients fulfilled the 2016 American College of Rheumatology-European League Against Rheumatism criteria for Sjögren's disease. The primary outcome was to identify MALT lymphoma predictors in Sjögren's disease, present at the timepoint of Sjögren's disease diagnosis and 3-4 years before the diagnosis of MALT lymphoma. A fast correlation-based feature selection and logistic regression model was used at V1 and V2 to identify MALT lymphoma predictors. The progression of potential predictors was studied across V1, V2, and V3. Histological parameters were not included in the analysis. An individual with lived experience of Sjögren's disease was involved in the study design.

FINDINGS

80 patients with Sjögren's disease-associated MALT lymphoma were included in the V1 dataset, 68 in the V2 dataset, and 80 in the V3 dataset, and matched to control participants with Sjögren's disease who did not have lymphoma. In both groups, 72 (90%) of 80 participants were women and eight (10%) were men. The mean age at Sjögren's disease diagnosis was 48·6 years (SD 11·6) in the lymphoma group and 48·7 years (11·5) in the control group. All patients were White, with 88 (55%) of 160 individuals of Greek nationality and 72 (45%) of Italian nationality. At the V1 timepoint, rheumatoid factor was the only independent lymphoma predictor (odds ratio 3·33 [95% CI 1·96-5·64]). At the V2 timepoint, rheumatoid factor (3·66 [95% CI 2·08-6·42]) and European League Against Rheumatism Sjögren's Syndrome Disease Activity Index ≥5 (3·88 [1·69-8·90]) were identified as independent lymphoma risk factors. The high disease activity during the transition from the V1 to V2 timepoint was attributed to specific B-cell-derived manifestations, including cryoglobulinaemia and glandular, cutaneous, and hematological manifestations.

INTERPRETATION

Following up patients with high-risk of Sjögren's disease-associated MALT lymphoma based on the temporal progression of predictors presents an opportunity for early diagnosis and potential therapeutic interventions. Rheumatoid factor was the earliest and most persistent independent predictor of lymphoma. Specific B-cell manifestations in combination with rheumatoid factor indicate a more advanced stage of the lymphomagenesis process.

FUNDING

European Commission-Horizon 2020.

Toward Ensuring Data Quality in Multi-Site Cancer Imaging Repositories

Cancer remains a major global health challenge, affecting diverse populations across various demographics. Integrating Artificial Intelligence (AI) into clinical settings to enhance disease outcome prediction presents notable challenges. This study addresses the limitations of AI-driven cancer care due to low-quality datasets by proposing a comprehensive three-step methodology to ensure high data quality in large-scale cancer-imaging repositories. Our methodology encompasses (i) developing a Data Quality Conceptual Model with specific metrics for assessment, (ii) creating a detailed data-collection protocol and a rule set to ensure data homogeneity and proper integration of multi-source data, and (iii) implementing a Data Integration Quality Check Tool (DIQCT) to verify adherence to quality requirements and suggest corrective actions. These steps are designed to mitigate biases, enhance data integrity, and ensure that integrated data meets high-quality standards. We applied this methodology within the INCISIVE project, an EU-funded initiative aimed at a pan-European cancer-imaging repository. The use-case demonstrated the effectiveness of our approach in defining quality rules and assessing compliance, resulting in improved data integration and higher data quality. The proposed methodology can assist the deployment of big data centralized or distributed repositories with data from diverse data sources, thus facilitating the development of AI tools.

The Diagnosis of Cardiovascular Disease Using Simple Blood Biomarkers Through AI and Big Data

Cardiovascular disease (CVD) is the leading cause of global mortality, diagnosed primarily through costly imaging modalities which are often overused in asymptomatic patients. Our project aims to develop an AI-based solution for CVD risk stratification using routine blood biomarkers, serving as a pre-imaging test. We used anonymized data from over 500,000 UK Biobank (UKB) patients with CVD assessments. Initially, 701 features including demographics, blood tests, medical conditions, and clinical assessments were selected. The UKB dataset was refined using an automated data curation pipeline to deal with outliers, duplicated fields, and missing values. Then, a hybrid XGBoost classifier was employed, with a scalable loss function, to address overfitting effects during the training process, yielding 0.83 accuracy, 0.82 sensitivity, and 0.84 specificity in diagnosing CVD comorbidities. Key biomarkers identified included blood pressure, BMI, and age. To our knowledge, this is the first case study which utilizes the UKB data towards the identification of cost-effective CVD (non-imaging) risk factors, thus reducing the reliance on imaging modalities.

From Scientific Research to Practical Implementations: Applications to Improve Data Quality in Child Welfare

Child welfare decisions have life-impacting consequences which, often times, are underpinned by limited or inadequate data and poor quality. Though research of data quality has gained popularity and made advancements in various practical areas, it has not made significant inroads for child welfare fields or data systems. Poor data quality can hinder service decision-making, impacting child behavioral health and well-being as well as increasing unnecessary expenditure of time and resources. Poor data quality can also undermine the validity of research and slow policymaking processes. The purpose of this commentary is to summarize the data quality research base in other fields, describe obstacles and uniqueness to improve data quality in child welfare, and propose necessary steps to scientific research and practical implementation that enables researchers and practitioners to improve the quality of child welfare services based on the enhanced quality of data.

AI-Enhanced Predictive Modeling for Identifying Depression and Delirium in Cardiovascular Patients Scheduled for Cardiac Surgery

Several studies have demonstrated a critical association between cardiovascular disease (CVD) and mental health, revealing that approximately one-third of individuals with CVD also experience depression. This comorbidity significantly increases the risk of cardiac complications and mortality, a risk that persists regardless of traditional factors. Addressing this issue, our study pioneers a straightforward, explainable, and data-driven pipeline for predicting depression in CVD patients.

METHODS

Our study was conducted at a cardiac surgical intensive care unit. A total of 224 participants who were scheduled for elective coronary artery bypass graft surgery (CABG) were enrolled in the study. Prior to surgery, each patient underwent psychiatric evaluation to identify major depressive disorder (MDD) based on the DSM-5 criteria. An advanced data curation workflow was applied to eliminate outliers and inconsistencies and improve data quality. An explainable AI-empowered pipeline was developed, where sophisticated machine learning techniques, including the AdaBoost, random forest, and XGBoost algorithms, were trained and tested on the curated data based on a stratified cross-validation approach.

RESULTS

Our findings identified a significant correlation between the biomarker "sRAGE" and depression (r = 0.32, p = 0.038). Among the applied models, the random forest classifier demonstrated superior accuracy in predicting depression, with notable scores in accuracy (0.62), sensitivity (0.71), specificity (0.53), and area under the curve (0.67).

CONCLUSIONS

This study provides compelling evidence that depression in CVD patients, particularly those with elevated "sRAGE" levels, can be predicted with a 62% accuracy rate. Our AI-driven approach offers a promising way for early identification and intervention, potentially revolutionizing care strategies in this vulnerable population.

Effective data quality management for electronic medical record data using SMART DATA

OBJECTIVES

In the medical field, we face many challenges, including the high cost of data collection and processing, difficult standards issues, and complex preprocessing techniques. It is necessary to establish an objective and systematic data quality management system that ensures data reliability, mitigates risks caused by incorrect data, reduces data management costs, and increases data utilization. We introduce the concept of SMART data in a data quality management system and conducted a case study using real-world data on colorectal cancer.

METHODS

We defined the data quality management system from three aspects (Construction - Operation - Utilization) based on the life cycle of medical data. Based on this, we proposed the "SMART DATA" concept and tested it on colorectal cancer data, which is actual real-world data.

RESULTS

We define "SMART DATA" as systematized, high-quality data collected based on the life cycle of data construction, operation, and utilization through quality control activities for medical data. In this study, we selected a scenario using data on colorectal cancer patients from a single medical institution provided by the Clinical Oncology Network (CONNECT). As SMART DATA, we curated 1,724 learning data and 27 Clinically Critical Set (CCS) data for colorectal cancer prediction. These datasets contributed to the development and fine-tuning of the colorectal cancer prediction model, and it was determined that CCS cases had unique characteristics and patterns that warranted additional clinical review and consideration in the context of colorectal cancer prediction.

CONCLUSIONS

In this study, we conducted primary research to develop a medical data quality management system. This will standardize medical data extraction and quality control methods and increase the utilization of medical data. Ultimately, we aim to provide an opportunity to develop a medical data quality management methodology and contribute to the establishment of a medical data quality management system.

Prognostic Signals From Moderate Valve Disease in Big Data: An Artefact of Digital Imaging and Communications in Medicine Structured Reporting?

BACKGROUND

Recent studies have identified an association between moderate aortic stenosis (AS) and outcome. We assessed whether Digital Imaging and Communications in Medicine (DICOM) structured reporting (SR), which captures and inserts echocardiographic measurements and text data directly into radiological reports, may lead to misclassifying patients with severe AS as moderate.

METHODS

Moderate or severe AS cases were filtered from an echocardiography data set based on aortic valve area (AVA) < 1.5 cm2, indexed AVA (AVAi) ≤ 0.85 cm2/m2, mean pressure gradient ≥ 25 mm Hg, dimensionless severity index (DSI) ≤ 0.5, or peak velocity > 3 m/sec. Data validation was conducted by verification of each parameter. All echocardiographic parameters and definitions of AS were compared pre- and postvalidation by taking differences in measurements. Misclassification rates were assessed by determining the percentage of cases that changed AS severity classification and impact on outcomes. Patients were followed over 4.3 ± 1.5 years.

RESULTS

Of 2,595 validated echocardiograms with AS, up to 36% of the echocardiographic parameters for AS criteria had a >10% difference between DICOM-SR and manual validation, the highest with mean pressure gradient (36%) and the lowest with DSI (6.5%). The validation process changed the reported degree of AS in up to 20.6% of echocardiograms with resultant changes in AS severity and its association with mortality or heart failure-related hospitalizations. In contrast to multiple quantitative metrics in DICOM-SR after manual validation, clinicians' evaluation of AS severity was unable to distinguish composite outcomes over 3 years between moderate and severe AS. The risk of composite outcomes was significantly increased when severe AS was evidenced by at least 1 echocardiographic parameter of severe AS (hazard ratio = 1.24; 95% CI, 1.12-1.37; P < .001). The greatest hazard was based on DSI only (hazard ratio = 1.26; 95% CI, 1.10-1.44; P < .001), which was higher after manual validation compared to DICOM-SR. Averaging of repeated echo measures including invalid values contributed the most to erroneous data.

CONCLUSIONS

Nonpeak data in DICOM-SR led to incorrect categorization of a high proportion of patients based on AS severity definitions. Standardization of data fields and curation to ensure that only peak values are imported from DICOM-SR data are essential.

Development and validation of the SickKids Enterprise-wide Data in Azure Repository (SEDAR)

Objectives

To describe the processes developed by The Hospital for Sick Children (SickKids) to enable utilization of electronic health record (EHR) data by creating sequentially transformed schemas for use across multiple user types.

Methods

We used Microsoft Azure as the cloud service provider and named this effort the SickKids Enterprise-wide Data in Azure Repository (SEDAR). Epic Clarity data from on-premises was copied to a virtual network in Microsoft Azure. Three sequential schemas were developed. The Filtered Schema added a filter to retain only SickKids and valid patients. The Curated Schema created a data structure that was easier to navigate and query. Each table contained a logical unit such as patients, hospital encounters or laboratory tests. Data validation of randomly sampled observations in the Curated Schema was performed. The SK-OMOP Schema was designed to facilitate research and machine learning. Two individuals mapped medical elements to standard Observational Medical Outcomes Partnership (OMOP) concepts.

Results

A copy of Clarity data was transferred to Microsoft Azure and updated each night using log shipping. The Filtered Schema and Curated Schema were implemented as stored procedures and executed each night with incremental updates or full loads. Data validation required up to 16 iterations for each Curated Schema table. OMOP concept mapping achieved at least 80 % coverage for each SK-OMOP table.

Conclusions

We described our experience in creating three sequential schemas to address different EHR data access requirements. Future work should consider replicating this approach at other institutions to determine whether approaches are generalizable.

Data Quality in Health Research: Integrative Literature Review

BACKGROUND

Decision-making and strategies to improve service delivery must be supported by reliable health data to generate consistent evidence on health status. The data quality management process must ensure the reliability of collected data. Consequently, various methodologies to improve the quality of services are applied in the health field. At the same time, scientific research is constantly evolving to improve data quality through better reproducibility and empowerment of researchers and offers patient groups tools for secured data sharing and privacy compliance.

OBJECTIVE

Through an integrative literature review, the aim of this work was to identify and evaluate digital health technology interventions designed to support the conducting of health research based on data quality.

METHODS

A search was conducted in 6 electronic scientific databases in January 2022: PubMed, SCOPUS, Web of Science, Institute of Electrical and Electronics Engineers Digital Library, Cumulative Index of Nursing and Allied Health Literature, and Latin American and Caribbean Health Sciences Literature. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist and flowchart were used to visualize the search strategy results in the databases.

RESULTS

After analyzing and extracting the outcomes of interest, 33 papers were included in the review. The studies covered the period of 2017-2021 and were conducted in 22 countries. Key findings revealed variability and a lack of consensus in assessing data quality domains and metrics. Data quality factors included the research environment, application time, and development steps. Strategies for improving data quality involved using business intelligence models, statistical analyses, data mining techniques, and qualitative approaches.

CONCLUSIONS

The main barriers to health data quality are technical, motivational, economical, political, legal, ethical, organizational, human resources, and methodological. The data quality process and techniques, from precollection to gathering, postcollection, and analysis, are critical for the final result of a study or the quality of processes and decision-making in a health care organization. The findings highlight the need for standardized practices and collaborative efforts to enhance data quality in health research. Finally, context guides decisions regarding data quality strategies and techniques.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1101/2022.05.31.22275804.

Electronic health record data quality assessment and tools: a systematic review

OBJECTIVE

We extended a 2013 literature review on electronic health record (EHR) data quality assessment approaches and tools to determine recent improvements or changes in EHR data quality assessment methodologies.

MATERIALS AND METHODS

We completed a systematic review of PubMed articles from 2013 to April 2023 that discussed the quality assessment of EHR data. We screened and reviewed papers for the dimensions and methods defined in the original 2013 manuscript. We categorized papers as data quality outcomes of interest, tools, or opinion pieces. We abstracted and defined additional themes and methods though an iterative review process.

RESULTS

We included 103 papers in the review, of which 73 were data quality outcomes of interest papers, 22 were tools, and 8 were opinion pieces. The most common dimension of data quality assessed was completeness, followed by correctness, concordance, plausibility, and currency. We abstracted conformance and bias as 2 additional dimensions of data quality and structural agreement as an additional methodology.

DISCUSSION

There has been an increase in EHR data quality assessment publications since the original 2013 review. Consistent dimensions of EHR data quality continue to be assessed across applications. Despite consistent patterns of assessment, there still does not exist a standard approach for assessing EHR data quality.

CONCLUSION

Guidelines are needed for EHR data quality assessment to improve the efficiency, transparency, comparability, and interoperability of data quality assessment. These guidelines must be both scalable and flexible. Automation could be helpful in generalizing this process.

Boosting the performance of MALT lymphoma classification in patients with primary Sjögren's Syndrome through data augmentation: a case study

Sjögren's Syndrome (SS) patients with mucosa associated lymphoid tissue lymphomas (MALTLs) and diffuse large B-cell lymphomas (DLBCLs) have 10-year survival rates of 80% and 40%, respectively. This highlights the unique biologic burden of the two histologic forms, as well as, the need for early detection and thorough monitoring of these patients. The lack of MALTL patients and the fact that most studies are single cohort and combine patients with different lymphoma subtypes narrow the understanding of MALTL progression. Here, we propose a data augmentation pipeline that utilizes an advanced synthetic data generator which is trained on a Pan European data hub with primary SS (pSS) patients to yield a high-quality synthetic data pool. The latter is used for the development of an enhanced MALTL classification model. Four scenarios were defined to assess the reliability of augmentation. Our results revealed an overall improvement in the accuracy, sensitivity, specificity, and AUC by 7%, 6.3%, 9%, and 6.3%, respectively. This is the first case study that utilizes data augmentation to reflect the progression of MALTL in pSS.

Data Quality in Health Care: Main Concepts and Assessment Methodologies

INTRODUCTION

 In the health care environment, a huge volume of data is produced on a daily basis. However, the processes of collecting, storing, sharing, analyzing, and reporting health data usually face with numerous challenges that lead to producing incomplete, inaccurate, and untimely data. As a result, data quality issues have received more attention than before.

OBJECTIVE

 The purpose of this article is to provide an insight into the data quality definitions, dimensions, and assessment methodologies.

METHODS

 In this article, a scoping literature review approach was used to describe and summarize the main concepts related to data quality and data quality assessment methodologies. Search terms were selected to find the relevant articles published between January 1, 2012 and September 31, 2022. The retrieved articles were then reviewed and the results were reported narratively.

RESULTS

 In total, 23 papers were included in the study. According to the results, data quality dimensions were various and different methodologies were used to assess them. Most studies used quantitative methods to measure data quality dimensions either in paper-based or computer-based medical records. Only two studies investigated respondents' opinions about data quality.

CONCLUSION

 In health care, high-quality data not only are important for patient care, but also are vital for improving quality of health care services and better decision making. Therefore, using technical and nontechnical solutions as well as constant assessment and supervision is suggested to improve data quality.

Predicting the need for mechanical ventilation and mortality in hospitalized COVID-19 patients who received heparin

Although several studies have utilized AI (artificial intelligence)-based solutions to enhance the decision making for mechanical ventilation, as well as, for mortality in COVID-19, the extraction of explainable predictors regarding heparin's effect in intensive care and mortality has been left unresolved. In the present study, we developed an explainable AI (XAI) workflow to shed light into predictors for admission in the intensive care unit (ICU), as well as, for mortality across those hospitalized COVID-19 patients who received heparin. AI empowered classifiers, such as, the hybrid Extreme gradient boosting (HXGBoost) with customized loss functions were trained on time-series curated clinical data to develop robust AI models. Shapley additive explanation analysis (SHAP) was conducted to determine the positive or negative impact of the predictors in the model's output. The HXGBoost predicted the risk for intensive care and mortality with 0.84 and 0.85 accuracy, respectively. SHAP analysis indicated that the low percentage of lymphocytes at day 7 along with increased FiO₂ at days 1 and 5, low SatO₂ at days 3 and 7 increase the probability for mortality and highlight the positive effect of heparin administration at the early days of hospitalization for reducing mortality.

Detection of Asymptomatic Carotid Artery Stenosis through Machine Learning

Carotid artery disease, the pathological condition of carotid arteries, is considered as the most significant cause of cerebral events and stroke. Carotid artery disease is considered as an inflammatory process, which involves the deposition and accumulation of atherosclerotic plaque inside the carotid intima, resulting in the narrowing of the arteries. Carotid artery stenosis (CAS) is either symptomatic or asymptomatic and its presence and location is determined by different imaging modalities, such as the carotid duplex ultrasound, the computed tomography angiography, the magnetic resonance angiography (MRA) and the cerebral angiography. The aim of this study is to present a machine learning model for the diagnosis and identification of individuals of asymptomatic carotid artery stenosis, using as input typical health data. More specifically, the overall model is trained with typical demographics, clinical data, risk factors and medical treatment data and is able to classify the individuals into high risk (Class 1-CAS group) and low risk (Class 0-non CAS group) individuals. In the presented study, we implemented a statistical analysis to check the data quality and the distribution into the two classes. Different feature selection techniques, in combination with classification schemes were applied for the development of our machine learning model. The overall methodology has been trained and tested using 881 cases (443 subjects in low risk class and 438 in high risk class). The highest accuracy 0.82 and an area under curve 0.9 were achieved using the relief feature selection technique and the random forest classification scheme.

A "smart" Imputation Approach for Effective Quality Control Across Complex Clinical Data Structures

The overwhelming need to improve the quality of complex data structures in healthcare is more important than ever. Although data quality has been the point of interest in many studies, none of them has focused on the development of quantitative and explainable methods for data imputation. In this work, we propose a "smart" imputation workflow to address missing data across complex data structures in the context of in silico clinical trials. AI algorithms were utilized to produce high-quality virtual patient profiles. A search algorithm was then developed to extract the best virtual patient profiles through the definition of a profile matching score (PMS). A case study was conducted, where the real dataset was randomly contaminated with multiple missing values (e.g., 10 to 50%). In total, 10000 virtual patient profiles with less than 0.02 Kullback-Leibler (KL) divergence were produced to estimate the PMS distribution. The best generator achieved the lowest average squared absolute difference (0.4) and average correlation difference (0.02) with the real dataset highlighting its increased effectiveness for data imputation across complex clinical data structures.

Bayesian Inference-Based Gaussian Mixture Models With Optimal Components Estimation Towards Large-Scale Synthetic Data Generation for In Silico Clinical Trials

Goal: To develop a computationally efficient and unbiased synthetic data generator for large-scale in silico clinical trials (CTs). Methods: We propose the BGMM-OCE, an extension of the conventional BGMM (Bayesian Gaussian Mixture Models) algorithm to provide unbiased estimations regarding the optimal number of Gaussian components and yield high-quality, large-scale synthetic data at reduced computational complexity. Spectral clustering with efficient eigenvalue decomposition is applied to estimate the hyperparameters of the generator. A case study is conducted to compare the performance of BGMM-OCE against four straightforward synthetic data generators for in silico CTs in hypertrophic cardiomyopathy (HCM). Results: The BGMM-OCE generated 30000 virtual patient profiles having the lowest coefficient-of-variation (0.046), inter- and intra-correlation differences (0.017, and 0.016, respectively) with the real ones in reduced execution time. Conclusions: BGMM-OCE overcomes the lack of population size in HCM which obscures the development of targeted therapies and robust risk stratification models.

ICU admission and mortality classifiers for COVID-19 patients based on subgroups of dynamically associated profiles across multiple timepoints

The coronavirus disease 2019 (COVID-19) which is caused by severe acute respiratory syndrome coronavirus type 2 (SARS-CoV-2) is consistently causing profound wounds in the global healthcare system due to its increased transmissibility. Currently, there is an urgent unmet need to identify the underlying dynamic associations among COVID-19 patients and distinguish patient subgroups with common clinical profiles towards the development of robust classifiers for ICU admission and mortality. To address this need, we propose a four step pipeline which: (i) enhances the quality of multiple timeseries clinical data through an automated data curation workflow, (ii) deploys Dynamic Bayesian Networks (DBNs) for the detection of features with increased connectivity based on dynamic association analysis across multiple points, (iii) utilizes Self Organizing Maps (SOMs) and trajectory analysis for the early identification of COVID-19 patients with common clinical profiles, and (iv) trains robust multiple additive regression trees (MART) for ICU admission and mortality classification based on the extracted homogeneous clusters, to identify risk factors and biomarkers for disease progression. The contribution of the extracted clusters and the dynamically associated clinical data improved the classification performance for ICU admission to sensitivity 0.83 and specificity 0.83, and for mortality to sensitivity 0.74 and specificity 0.76. Additional information was included to enhance the performance of the classifiers yielding an increase by 4% in sensitivity and specificity for mortality. According to the risk factor analysis, the number of lymphocytes, SatO2, PO2/FiO2, and O2 supply type were highlighted as risk factors for ICU admission and the percentage of neutrophils and lymphocytes, PO2/FiO2, LDH, and ALP for mortality, among others. To our knowledge, this is the first study that combines dynamic modeling with clustering analysis to identify homogeneous groups of COVID-19 patients towards the development of robust classifiers for ICU admission and mortality.

G. van Roon J, Xavier M, De Vita S, Tzioufas AG, Fotiadis DI. Addressing the clinical unmet needs in primary Sjögren's Syndrome through the sharing, harmonization and federated analysis of 21 European cohorts

For many decades, the clinical unmet needs of primary Sjögren's Syndrome (pSS) have been left unresolved due to the rareness of the disease and the complexity of the underlying pathogenic mechanisms, including the pSS-associated lymphomagenesis process. Here, we present the HarmonicSS cloud-computing exemplar which offers beyond the state-of-the-art data analytics services to address the pSS clinical unmet needs, including the development of lymphoma classification models and the identification of biomarkers for lymphomagenesis. The users of the platform have been able to successfully interlink, curate, and harmonize 21 regional, national, and international European cohorts of 7,551 pSS patients with respect to the ethical and legal issues for data sharing. Federated AI algorithms were trained across the harmonized databases, with reduced execution time complexity, yielding robust lymphoma classification models with 85% accuracy, 81.25% sensitivity, 85.4% specificity along with 5 biomarkers for lymphoma development. To our knowledge, this is the first GDPR compliant platform that provides federated AI services to address the pSS clinical unmet needs.

Developing a systematic approach to assessing data quality in secondary use of clinical data based on intended use

INTRODUCTION

Secondary use of electronic health record (EHR) data for research requires that the data are fit for use. Data quality (DQ) frameworks have traditionally focused on structural conformance and completeness of clinical data extracted from source systems. In this paper, we propose a framework for evaluating semantic DQ that will allow researchers to evaluate fitness for use prior to analyses.

METHODS

We reviewed current DQ literature, as well as experience from recent multisite network studies, and identified gaps in the literature and current practice. Derived principles were used to construct the conceptual framework with attention to both analytic fitness and informatics practice.

RESULTS

We developed a systematic framework that guides researchers in assessing whether a data source is fit for use for their intended study or project. It combines tools for evaluating clinical context with DQ principles, as well as factoring in the characteristics of the data source, in order to develop semantic DQ checks.

CONCLUSIONS

Our framework provides a systematic process for DQ development. Further work is needed to codify practices and metadata around both structural and semantic data quality.

A Multimodal Approach for the Risk Prediction of Intensive Care and Mortality in Patients with COVID-19

BACKGROUND

Although several studies have been launched towards the prediction of risk factors for mortality and admission in the intensive care unit (ICU) in COVID-19, none of them focuses on the development of explainable AI models to define an ICU scoring index using dynamically associated biological markers.

METHODS

We propose a multimodal approach which combines explainable AI models with dynamic modeling methods to shed light into the clinical features of COVID-19. Dynamic Bayesian networks were used to seek associations among cytokines across four time intervals after hospitalization. Explainable gradient boosting trees were trained to predict the risk for ICU admission and mortality towards the development of an ICU scoring index.

RESULTS

Our results highlight LDH, IL-6, IL-8, Cr, number of monocytes, lymphocyte count, TNF as risk predictors for ICU admission and survival along with LDH, age, CRP, Cr, WBC, lymphocyte count for mortality in the ICU, with prediction accuracy 0.79 and 0.81, respectively. These risk factors were combined with dynamically associated biological markers to develop an ICU scoring index with accuracy 0.9.

CONCLUSIONS

to our knowledge, this is the first multimodal and explainable AI model which quantifies the risk of intensive care with accuracy up to 0.9 across multiple timepoints.

Variational Gaussian Mixture Models with robust Dirichlet concentration priors for virtual population generation in hypertrophic cardiomyopathy: a comparison study

Nowadays, there is a growing need for the development of computationally efficient virtual population generators for large-scale in-silico clinical trials. In this work, we utilize the Gaussian Mixture Models (GMM) with variational Bayesian inference (BGMM) using robust estimations of Dirichlet concentration priors for the generation of virtual populations. The estimations were based on an exponential transformation of the number of Gaussian components. The proposed method was compared against state-of-the-art virtual data generators, such as, the Bayesian networks, the supervised tree ensembles (STE), the unsupervised tree ensembles (UTE), and the artificial neural networks (ANN) towards the generation of 20000 virtual patients with hypertrophic cardiomyopathy (HCM). Our results suggest that the proposed BGMM can yield virtual distributions with small inter- and intra-correlation difference (0.013 and 0.012), in lower execution time (4.321 sec) than STE which achieved the second-best performance.

A Machine Learning Model for the Identification of High risk Carotid Atherosclerotic Plaques

Carotid artery disease is an inflammatory condition involving the deposition and accumulation of lipid species and leucocytes from blood into the arterial wall, which causes the narrowing of the carotid arteries on either side of the neck. Different imaging modalities can by implemented to determine the presence and the location of carotid artery stenosis, such as carotid ultrasound, computed tomography angiography (CTA), magnetic resonance angiography (MRA), or cerebral angiography. However, except of the presence and the degree of stenosis of the carotid arteries, the vulnerability of the carotid atherosclerotic plaques constitutes a significant factor for the progression of the disease and the presence of disease symptoms. In this study, our aim is to develop and present a machine learning model for the identification of high risk plaques using non imaging based features and non-invasive imaging based features. Firstly, we implemented statistical analysis to identify the most statistical significant features according to the defined output, and subsequently, we implemented different feature selection techniques and classification schemes for the development of our machine learning model. The overall methodology has been trained and tested using 208 cases of 107 cases of low risk plaques and 101 cases of high risk plaques. The highest accuracy of 0.76 was achieved using the relief feature selection technique and the support vector machine classification scheme. The innovative aspect of the proposed machine learning model is both the different categories of the utilized input features and the definition of the problem to be solved.

Multiple additive regression trees with hybrid loss for classification tasks across heterogeneous clinical data in distributed environments: a case study

Multiple additive regression trees (MART) have been widely used in the literature for various classification tasks. However, the overfitting effects of MART across heterogeneous and highly imbalanced big data structures within distributed environments has not yet been investigated. In this work, we utilize distributed MART with hybrid loss to resolve overfitting effects during the training of disease classification models in a case study with 10 heterogeneous and distributed clinical datasets. Lexical and semantic analysis methods were utilized to match heterogeneous terminologies with 80% overlap. Data augmentation was used to resolve class imbalance yielding virtual data with goodness of fit 0.01 and correlation difference 0.02. Our results highlight the favorable performance of the proposed distributed MART on the augmented data with an average increase by 7.3% in the accuracy, 6.8% in sensitivity, 10.4% in specificity, for a specific loss function topology.

A federated AI strategy for the classification of patients with Mucosa Associated Lymphoma Tissue (MALT) lymphoma across multiple harmonized cohorts

Mucosa Associated Lymphoma Tissue (MALT) type is an extremely rare type of lymphoma which occurs in less than 3% of patients with primary Sjögren's Syndrome (pSS). No reported studies so far have been able to investigate risk factors for MALT development across multiple cohort databases with sufficient statistical power. Here, we present a generalized, federated AI (artificial intelligence) strategy which enables the training of AI algorithms across multiple harmonized databases. A case study is conducted towards the development of MALT classification models across 17 databases on pSS. Advanced AI algorithms were developed, including federated Multinomial Naïve Bayes (FMNB), federated gradient boosting trees (FGBT), FGBT with dropouts (FDART), and the federated Multilayer Perceptron (FMLP). The FDART with dropout rate 0.3 achieved the best performance with sensitivity 0.812, and specificity 0.829, yielding 8 biomarkers as prominent for MALT development.

Big Data in Nephrology

A huge array of data in nephrology is collected through patient registries, large epidemiological studies, electronic health records, administrative claims, clinical trial repositories, mobile health devices and molecular databases. Application of these big data, particularly using machine-learning algorithms, provides a unique opportunity to obtain novel insights into kidney diseases, facilitate personalized medicine and improve patient care. Efforts to make large volumes of data freely accessible to the scientific community, increased awareness of the importance of data sharing and the availability of advanced computing algorithms will facilitate the use of big data in nephrology. However, challenges exist in accessing, harmonizing and integrating datasets in different formats from disparate sources, improving data quality and ensuring that data are secure and the rights and privacy of patients and research participants are protected. In addition, the optimism for data-driven breakthroughs in medicine is tempered by scepticism about the accuracy of calibration and prediction from in silico techniques. Machine-learning algorithms designed to study kidney health and diseases must be able to handle the nuances of this specialty, must adapt as medical practice continually evolves, and must have global and prospective applicability for external and future datasets.

Assessment of the Quality Management System for Clinical Nutrition in Jiangsu: Survey Study

BACKGROUND

An electronic system that automatically collects medical information can realize timely monitoring of patient health and improve the effectiveness and accuracy of medical treatment. To our knowledge, the application of artificial intelligence (AI) in medical service quality assessment has been minimally evaluated, especially for clinical nutrition departments in China. From the perspective of medical ethics, patient safety comes before any other factors within health science, and this responsibility belongs to the quality management system (QMS) within medical institutions.

OBJECTIVE

This study aims to evaluate the QMS for clinical nutrition in Jiangsu, monitor its performance in quality assessment and human resource management from a nutrition aspect, and investigate the application and development of AI in medical quality control.

METHODS

The participants for this study were the staff of 70 clinical nutrition departments of the tertiary hospitals in Jiangsu Province, China. These departments are all members of the Quality Management System of Clinical Nutrition in Jiangsu (QMSNJ). An online survey was conducted on all 341 employees within all clinical nutrition departments based on the staff information from the surveyed medical institutions. The questionnaire contains five sections, and the data analysis and AI evaluation were focused on human resource information.

RESULTS

A total of 330 questionnaires were collected, with a response rate of 96.77% (330/341). A QMS for clinical nutrition was built for clinical nutrition departments in Jiangsu and achieved its target of human resource improvements, especially among dietitians. The growing number of participating departments (an increase of 42.8% from 2018 to 2020) and the significant growth of dietitians (t93.4=-0.42; P=.02) both show the advancements of the QMSNJ.

CONCLUSIONS

As the first innovation of an online platform for quality management in Jiangsu, the Jiangsu Province Clinical Nutrition Management Platform was successfully implemented as a QMS for this study. This multidimensional electronic system can help the QMSNJ and clinical nutrition departments achieve quality assessment from various aspects so as to realize the continuous improvement of clinical nutrition. The use of an online platform and AI technology for quality assessment is worth recommending and promoting in the future.

A computational pipeline for data augmentation towards the improvement of disease classification and risk stratification models: A case study in two clinical domains

Virtual population generation is an emerging field in data science with numerous applications in healthcare towards the augmentation of clinical research databases with significant lack of population size. However, the impact of data augmentation on the development of AI (artificial intelligence) models to address clinical unmet needs has not yet been investigated. In this work, we assess whether the aggregation of real with virtual patient data can improve the performance of the existing risk stratification and disease classification models in two rare clinical domains, namely the primary Sjögren's Syndrome (pSS) and the hypertrophic cardiomyopathy (HCM), for the first time in the literature. To do so, multivariate approaches, such as, the multivariate normal distribution (MVND), and straightforward ones, such as, the Bayesian networks, the artificial neural networks (ANNs), and the tree ensembles are compared against their performance towards the generation of high-quality virtual data. Both boosting and bagging algorithms, such as, the Gradient boosting trees (XGBoost), the AdaBoost and the Random Forests (RFs) were trained on the augmented data to evaluate the performance improvement for lymphoma classification and HCM risk stratification. Our results revealed the favorable performance of the tree ensemble generators, in both domains, yielding virtual data with goodness-of-fit 0.021 and KL-divergence 0.029 in pSS and 0.029, 0.027 in HCM, respectively. The application of the XGBoost on the augmented data revealed an increase by 10.9% in accuracy, 10.7% in sensitivity, 11.5% in specificity for lymphoma classification and 16.1% in accuracy, 16.9% in sensitivity, 13.7% in specificity in HCM risk stratification.

A computational workflow for the detection of candidate diagnostic biomarkers of Kawasaki disease using time-series gene expression data

Unlike autoimmune diseases, there is no known constitutive and disease-defining biomarker for systemic autoinflammatory diseases (SAIDs). Kawasaki disease (KD) is one of the "undiagnosed" types of SAIDs whose pathogenic mechanism and gene mutation still remain unknown. To address this issue, we have developed a sequential computational workflow which clusters KD patients with similar gene expression profiles across the three different KD phases (Acute, Subacute and Convalescent) and utilizes the resulting clustermap to detect prominent genes that can be used as diagnostic biomarkers for KD. Self-Organizing Maps (SOMs) were employed to cluster patients with similar gene expressions across the three phases through inter-phase and intra-phase clustering. Then, false discovery rate (FDR)-based feature selection was applied to detect genes that significantly deviate across the per-phase clusters. Our results revealed five genes as candidate biomarkers for KD diagnosis, namely, the HLA-DQB1, HLA-DRA, ZBTB48, TNFRSF13C, and CASD1. To our knowledge, these five genes are reported for the first time in the literature. The impact of the discovered genes for KD diagnosis against the known ones was demonstrated by training boosting ensembles (AdaBoost and XGBoost) for KD classification on common platform and cross-platform datasets. The classifiers which were trained on the proposed genes from the common platform data yielded an average increase by 4.40% in accuracy, 5.52% in sensitivity, and 3.57% in specificity than the known genes in the Acute and Subacute phases, followed by a notable increase by 2.30% in accuracy, 2.20% in sensitivity, and 4.70% in specificity in the cross-platform analysis.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

A machine learning-based pipeline for modeling medical, socio-demographic, lifestyle and self-reported psychological traits as predictors of mental health outcomes after breast cancer diagnosis: An initial effort to define resilience effects

Displaying resilience following a diagnosis of breast cancer is crucial for successful adaptation to illness, well-being, and health outcomes. Several theoretical and computational models have been proposed toward understanding the complex process of illness adaptation, involving a large variety of patient sociodemographic, lifestyle, medical, and psychological characteristics. To date, conventional multivariate statistical methods have been used extensively to model resilience. In the present work we describe a computational pipeline designed to identify the most prominent predictors of mental health outcomes following breast cancer diagnosis. A machine learning framework was developed and tested on the baseline data (recorded immediately post diagnosis) from an ongoing prospective, multinational study. This fully annotated dataset includes socio-demographic, lifestyle, medical and self-reported psychological characteristics of women recently diagnosed with breast cancer (N = 609). Nine different feature selection and cross-validated classification schemes were compared on their performance in classifying patients into low vs high depression symptom severity. Best-performing approaches involved a meta-estimator combined with a Support Vector Machines (SVMs) classification algorithm, exhibiting balanced accuracy of 0.825, and a fair balance between sensitivity (90%) and specificity (74%). These models consistently identified a set of psychological traits (optimism, perceived ability to cope with trauma, resilience as trait, ability to comprehend the illness), and subjective perceptions of personal functionality (physical, social, cognitive) as key factors accounting for concurrent depression symptoms. A comprehensive supervised learning pipeline is proposed for the identification of predictors of depression symptoms which could severely impede adaptation to illness.

Primary Sjögren's Syndrome of Early and Late Onset: Distinct Clinical Phenotypes and Lymphoma Development

Objectives

To study the clinical, serological and histologic features of primary Sjögren’s syndrome (pSS) patients with early (young ≤35 years) or late (old ≥65 years) onset and to explore the differential effect on lymphoma development.

Methods

From a multicentre study population of 1997 consecutive pSS patients, those with early or late disease onset, were matched and compared with pSS control patients of middle age onset. Data driven analysis was applied to identify the independent variables associated with lymphoma in both age groups.

Results

Young pSS patients (19%, n = 379) had higher frequency of salivary gland enlargement (SGE, lymphadenopathy, Raynaud’s phenomenon, autoantibodies, C4 hypocomplementemia, hypergammaglobulinemia, leukopenia, and lymphoma (10.3% vs. 5.7%, p = 0.030, OR = 1.91, 95% CI: 1.11–3.27), while old pSS patients (15%, n = 293) had more frequently dry mouth, interstitial lung disease, and lymphoma (6.8% vs. 2.1%, p = 0.011, OR = 3.40, 95% CI: 1.34–8.17) compared to their middle-aged pSS controls, respectively. In young pSS patients, cryoglobulinemia, C4 hypocomplementemia, lymphadenopathy, and SGE were identified as independent lymphoma associated factors, as opposed to old pSS patients in whom SGE, C4 hypocomplementemia and male gender were the independent lymphoma associated factors. Early onset pSS patients displayed two incidence peaks of lymphoma within 3 years of onset and after 10 years, while in late onset pSS patients, lymphoma occurred within the first 6 years.

Conclusion

Patients with early and late disease onset constitute a significant proportion of pSS population with distinct clinical phenotypes. They possess a higher prevalence of lymphoma, with different predisposing factors and lymphoma distribution across time.

A Rule-Based Data Quality Assessment System for Electronic Health Record Data

OBJECTIVE

Rule-based data quality assessment in health care facilities was explored through compilation, implementation, and evaluation of 63,397 data quality rules in a single-center case study to assess the ability of rules-based data quality assessment to identify data errors of importance to physicians and system owners.

METHODS

We applied a design science framework to design, demonstrate, test, and evaluate a scalable framework with which data quality rules can be managed and used in health care facilities for data quality assessment and monitoring.

RESULTS

We identified 63,397 rules partitioned into 28 logic templates. A total of 819,683 discrepancies were identified by 4.5% of the rules. Nine out of 11 participating clinical and operational leaders indicated that the rules identified data quality problems and articulated next steps that they wanted to take based on the reported information.

DISCUSSION

The combined rule template and knowledge table approach makes governance and maintenance of otherwise large rule sets manageable. Identified challenges to rule-based data quality monitoring included the lack of curated and maintained knowledge sources relevant to data error detection and lack of organizational resources to support clinical and operational leaders with investigation and characterization of data errors and pursuit of corrective and preventative actions. Limitations of our study included implementation within a single center and dependence of the results on the implemented rule set.

CONCLUSION

This study demonstrates a scalable framework (up to 63,397 rules) with which data quality rules can be implemented and managed in health care facilities to identify data errors. The data quality problems identified at the implementation site were important enough to prompt action requests from clinical and operational leaders.

Data-Influence Analytics in Predictive Models Applied to Asthma Disease

Asthma is one of the most common chronic diseases around the world and represents a serious problem in human health. Predictive models have become important in medical sciences because they provide valuable information for data-driven decision-making. In this work, a methodology of data-influence analytics based on mixed-effects logistic regression models is proposed for detecting potentially influential observations which can affect the quality of these models. Global and local influence diagnostic techniques are used simultaneously in this detection, which are often used separately. In addition, predictive performance measures are considered for this analytics. A study with children and adolescent asthma real data, collected from a public hospital of São Paulo, Brazil, is conducted to illustrate the proposed methodology. The results show that the influence diagnostic methodology is helpful for obtaining an accurate predictive model that provides scientific evidence when data-driven medical decision-making.

Advancing computer-aided drug discovery (CADD) by big data and data-driven machine learning modeling

Advancing a new drug to market requires substantial investments in time as well as financial resources. Crucial bioactivities for drug candidates, including their efficacy, pharmacokinetics (PK), and adverse effects, need to be investigated during drug development. With advancements in chemical synthesis and biological screening technologies over the past decade, a large amount of biological data points for millions of small molecules have been generated and are stored in various databases. These accumulated data, combined with new machine learning (ML) approaches, such as deep learning, have shown great potential to provide insights into relevant chemical structures to predict in vitro, in vivo, and clinical outcomes, thereby advancing drug discovery and development in the big data era.

Generation of virtual patient data for in-silico cardiomyopathies drug development using tree ensembles: a comparative study

In-silico clinical platforms have been recently used as a new revolutionary path for virtual patients (VP) generation and further analysis, such as, drug development. Advanced individualized models have been developed to enhance flexibility and reliability of the virtual patient cohorts. This study focuses on the implementation and comparison of three different methodologies for generating virtual data for in-silico clinical trials. Towards this direction, three computational methods, namely: (i) the multivariate log-normal distribution (log- MVND), (ii) the supervised tree ensembles, and (iii) the unsupervised tree ensembles are deployed and evaluated against their performance towards the generation of high-quality virtual data using the goodness of fit (gof) and the dataset correlation matrix as performance evaluation measures. Our results reveal the dominance of the tree ensembles towards the generation of virtual data with similar distributions (gof values less than 0.2) and correlation patterns (average difference less than 0.03).

Emerging drugs for primary Sjögren's syndrome

Introduction: Primary Sjögren's syndrome (pSS) is an autoimmune systemic disease characterized by a complex and not yet completely elucidated etiopathogenesis, where autoimmune manifestations coexist with different degree of lymphoproliferation, resulting in multiple possible scenarios extremely heterogeneous from patient to patient. Although considerable progress has been made in the identifications of potential novel therapeutic targets in recent years, the biological complexity of pSS, combined to such heterogeneous clinical manifestations, makes the treatment of pSS, even today, a great challenge. Areas covered: A therapy specifically approved for pSS is still lacking. In recent years, several novel promising agents are being tested in pSS. Based on a deep revision of drugs evaluated for pSS therapy, it is striking that several clinical trials, some of them testing very promising agents, failed. Expert opinion: a renewal of clinical trial design, including the definition of novel inclusion criteria and outcome measures, together with the development of a stratification model of pSS patients and the advance in the definition of pathogenetic mechanisms underlying peculiar pSS subsets, represent preliminary and crucial steps to overcome the current therapeutic impasse in pSS.

Inaccurate Labels in Weakly-Supervised Deep Learning: Automatic Identification and Correction and Their Impact on Classification Performance

In data-driven deep learning-based modeling, data quality may substantially influence classification performance. Correct data labeling for deep learning modeling is critical. In weakly-supervised learning, a challenge lies in dealing with potentially inaccurate or mislabeled training data. In this paper, we proposed an automated methodological framework to identify mislabeled data using two metric functions, namely, Cross-entropy Loss that indicates divergence between a prediction and ground truth, and Influence function that reflects the dependence of a model on data. After correcting the identified mislabels, we measured their impact on the classification performance. We also compared the mislabeling effects in three experiments on two different real-world clinical questions. A total of 10,500 images were studied in the contexts of clinical breast density category classification and breast cancer malignancy diagnosis. We used intentionally flipped labels as mislabels to evaluate the proposed method at a varying proportion of mislabeled data included in model training. We also compared the effects of our method to two published schemes for breast density category classification. Experiment results show that when the dataset contains 10% of mislabeled data, our method can automatically identify up to 98% of these mislabeled data by examining/checking the top 30% of the full dataset. Furthermore, we show that correcting the identified mislabels leads to an improvement in the classification performance. Our method provides a feasible solution for weakly-supervised deep learning modeling in dealing with inaccurate labels.

Overcoming the Barriers That Obscure the Interlinking and Analysis of Clinical Data Through Harmonization and Incremental Learning

Goal: To present a framework for data sharing, curation, harmonization and federated data analytics to solve open issues in healthcare, such as, the development of robust disease prediction models. Methods: Data curation is applied to remove data inconsistencies. Lexical and semantic matching methods are used to align the structure of the heterogeneous, curated cohort data along with incremental learning algorithms including class imbalance handling and hyperparameter optimization to enable the development of disease prediction models. Results: The applicability of the framework is demonstrated in a case study of primary Sjögren's Syndrome, yielding harmonized data with increased quality and more than 85% agreement, along with lymphoma prediction models with more than 80% sensitivity and specificity. Conclusions: The framework provides data quality, harmonization and analytics workflows that can enhance the statistical power of heterogeneous clinical data and enables the development of robust models for disease prediction.

Predicting Lymphoma Development by Exploiting Genetic Variants and Clinical Findings in a Machine Learning-Based Methodology With Ensemble Classifiers in a Cohort of Sjögren's Syndrome Patients

Lymphoma development constitutes one of the most serious clinico-pathological manifestations of patients with Sjögren's Syndrome (SS). Over the last decades the risk for lymphomagenesis in SS patients has been studied aiming to identify novel biomarkers and risk factors predicting lymphoma development in this patient population. Objective: The current study aims to explore whether genetic susceptibility profiles of SS patients along with known clinical, serological and histological risk factors enhance the accuracy of predicting lymphoma development in this patient population. Methods: The potential predicting role of both genetic variants, clinical and laboratory risk factors were investigated through a Machine Learning-based (ML) framework which encapsulates ensemble classifiers. Results: Ensemble methods empower the classification accuracy with approaches which are sensitive to minor perturbations in the training phase. The evaluation of the proposed methodology based on a 10-fold stratified cross validation procedure yielded considerable results in terms of balanced accuracy (GB: 0.7780 ± 0.1514, RF Gini: 0.7626 ± 0.1787, RF Entropy: 0.7590 ± 0.1837). Conclusions: The initial clinical, serological, histological and genetic findings at an early diagnosis have been exploited in an attempt to establish predictive tools in clinical practice and further enhance our understanding towards lymphoma development in SS.

Predicting lymphoma development in patients with Sjögren's syndrome

Introduction: The issue of predicting lymphoma in primary Sjögren's syndrome (pSS) starts from its clinical and biologic essence, i.e., an autoimmune exocrinopathy with sicca syndrome, inflammation and lymphoproliferation of MALT (mucosa-associated lymphoid tissue) in exocrine glands. Areas covered: The two major predictors to be firstly focused are persistent salivary gland (SG) swelling and cryoglobulinemic vasculitis with related features as purpura and low C4, or the sole serum cryoglobulinemia repeatedly detected. They are pathogenetically linked and reflect a heavier MALT involvement by histopathology, with the expansion of peculiar rheumatoid factor (RF)-positive clones/idiotypes. Other predictors include lymphadenopathy, splenomegaly, neutropenia, lymphopenia, serum beta2-microglobulin, monoclonal immunoglobulins, light chains, and RF. Composite indexes/scores may also predict lymphoma. Expert opinion: Prediction at baseline needs amelioration, and must be repeated in the follow-up. Careful clinical characterization, with harmonization and stratification of large cohorts, is a relevant preliminary step. Validated and new biomarkers are needed in biologic fluids and tissues. SG echography with automatic scoring could represent a future imaging biomarker, still lacking. Scoring MALT involvement in pSS, as an additional tool to evaluate disease activity and possibly to predict lymphoma, is welcomed. All these efforts are now ongoing within the HarmonicSS project and in other research initiatives in pSS.