A computerised decision support system for cardiovascular risk management 'live' in the electronic health record environment: development, validation and implementation-the Utrecht Cardiovascular Cohort Initiative

Analytical validation of Exandra: a clinical decision support system for promoting guideline-directed therapy of type-2 diabetes in primary care - a collaborative study with experts from Diabetes Canada

BACKGROUND

Individuals with type 2 diabetes (T2D) have a high prevalence of cardiovascular and renal comorbidities. Despite clinical practice guidelines recommending the use of cardiorenal protective medications, many people with T2D are not prescribed these medications. A clinical decision support system called Exandra was developed to provide treatment recommendations for individuals with T2D based on current clinical practice guidelines from Diabetes Canada. The current study aimed to medically validate Exandra via review by external medical experts in T2D.

METHODS

Validation of Exandra took place in two phases. Test cases using simulated clinical scenarios and recommendations were generated by Exandra. In Phase 1 of the validation, reviewers evaluated whether they agreed with Exandra's recommendations with a "yes," "no," or "not sure" response. In Phase 2, reviewers were interviewed about their "no" and "not sure" responses to determine possible reasons and potential fixes to the Exandra system. The primary outcome was the precision rate of Exandra following the interviews and final adjudication of the cases. The target precision rate was 90%.

RESULTS

Exandra displayed an overall precision rate of 95.5%. A large proportion of cases that were initially labeled "no" or "not sure" by reviewers were changed to "yes" following the interview phase. This was largely due to the validation using a simplified user interface compared with the complexity of the actual Exandra system, and reviewers needing clarification of how the outputs would be displayed on the Exandra platform.

CONCLUSION

Exandra displayed a high level of accuracy and precision in providing guideline-directed recommendations for managing T2D and its common comorbidities. The results of this study indicate that Exandra is a promising tool for improving the management of T2D and its comorbidities.

A data management system for precision medicine

Precision, or personalised medicine has advanced requirements for medical data management systems (MedDMSs). MedDMS for precision medicine should be able to process hundreds of parameters from multiple sites, be adaptable while remaining in sync at multiple locations, real-time syncing to analytics and be compliant with international privacy legislation. This paper describes the LogiqSuite software solution, aimed to support a precision medicine solution at the patient care (LogiqCare), research (LogiqScience) and data science (LogiqAnalytics) level. LogiqSuite is certified and compliant with international medical data and privacy legislations. This paper evaluates a MedDMS in five types of use cases for precision medicine, ranging from data collection to algorithm development and from implementation to integration with real-world data. The MedDMS is evaluated in seven precision medicine data science projects in prehospital triage, cardiovascular disease, pulmonology, and oncology. The P4O2 consortium uses the MedDMS as an electronic case report form (eCRF) that allows real-time data management and analytics in long covid and pulmonary diseases. In an acute myeloid leukaemia, study data from different sources were integrated to facilitate easy descriptive analytics for various research questions. In the AIDPATH project, LogiqCare is used to process patient data, while LogiqScience is used for pseudonymous CAR-T cell production for cancer treatment. In both these oncological projects the data in LogiqAnalytics is also used to facilitate machine learning to develop new prediction models for clinical-decision support (CDS). The MedDMS is also evaluated for real-time recording of CDS data from U-Prevent for cardiovascular risk management and from the Stroke Triage App for prehospital triage. The MedDMS is discussed in relation to other solutions for privacy-by-design, integrated data stewardship and real-time data analytics in precision medicine. LogiqSuite is used for multi-centre research study data registrations and monitoring, data analytics in interdisciplinary consortia, design of new machine learning / artificial intelligence (AI) algorithms, development of new or updated prediction models, integration of care with advanced therapy production, and real-world data monitoring in using CDS tools. The integrated MedDMS application supports data management for care and research in precision medicine.

Healthcare dashboard technologies and data visualization for lipid management: A scoping review

BACKGROUND

Lipid disorders significantly increase cardiovascular disease (CVD) risk, the leading cause of mortality worldwide. Effective lipid management is critical for improving health outcomes. Traditional screening methods face challenges due to data complexity and the need for timely decision-making. Data visualization and dashboard technologies offer clear, actionable insights and supporting informed decision-making. This study investigates the use of these technologies in lipid management and their impacts on the quadruple aim of healthcare.

METHODS

This scoping review followed the Joanna Briggs Institute (JBI) approach, focusing on studies involving dashboard technologies or data visualization in lipid management. A comprehensive search across multiple databases (Embase, Web of Science, PubMed, Scopus, CINAHL) and gray literature was conducted, including English-language publications from 2014 to 2024. Data were analyzed using quantitative descriptive and qualitative content analysis to evaluate the key features, clinical applications, and outcomes.

RESULTS

Twenty-seven studies met the inclusion criteria, primarily focusing on dashboard utilization by physicians for managing diabetes and CVD, utilizing electronic medical records and clinical guidelines. Key analysis methods included comparing key performance indicators (KPIs) (85.2%) and trend analysis (74.1%). Lipid management workflows emphasized prevention (88.9%) and treatment planning (77.8%). Interventions included care packages (comprehensive sets of interventions for patient care), decision support systems, web-based tools, and mobile health solutions. Regarding Quadruple Aim outcomes: 12 studies focused on improving population health (8 positive, 4 no change), 9 on clinical outcomes (5 positive, 4 no change), 6 on provider work life (5 positive), 5 on patient experience (positive changes in education and time management), and 2 on cost reduction (1 positive, 1 negative).

CONCLUSIONS

Dashboards are important tools in managing lipid disorders in managing lipid disorders, integrating with educational tools, collaborative care models, and decision support systems. Although they are effective in enhancing population health and clinical experiences, their impact on patient outcomes and cost reduction requires further exploration. Future research should focus on detailed evaluations of dashboard impacts on patient outcomes and cost-effectiveness, emphasizing precision prevention of chronic diseases.

The use of imputation in clinical decision support systems: a cardiovascular risk management pilot vignette study among clinicians

Aims

A major challenge of the use of prediction models in clinical care is missing data. Real-time imputation may alleviate this. However, to what extent clinicians accept this solution remains unknown. We aimed to assess acceptance of real-time imputation for missing patient data in a clinical decision support system (CDSS) including 10-year cardiovascular absolute risk for the individual patient.

Methods and results

We performed a vignette study extending an existing CDSS with the real-time imputation method joint modelling imputation (JMI). We included 17 clinicians to use the CDSS with three different vignettes, describing potential use cases (missing data, no risk estimate; imputed values, risk estimate based on imputed data; complete information). In each vignette, missing data were introduced to mimic a situation as could occur in clinical practice. Acceptance of end-users was assessed on three different axes: clinical realism, comfortableness, and added clinical value. Overall, the imputed predictor values were found to be clinically reasonable and according to the expectations. However, for binary variables, use of a probability scale to express uncertainty was deemed inconvenient. The perceived comfortableness with imputed risk prediction was low, and confidence intervals were deemed too wide for reliable decision-making. The clinicians acknowledged added value for using JMI in clinical practice when used for educational, research, or informative purposes.

Conclusion

Handling missing data in CDSS via JMI is useful, but more accurate imputations are needed to generate comfort in clinicians for use in routine care. Only then can CDSS create clinical value by improving decision-making.

Artificial Intelligence-Based Clinical Decision Support Systems in Cardiovascular Diseases

Despite all the advancements in science, medical knowledge, healthcare, and the healthcare industry, cardiovascular disease (CVD) remains the leading cause of morbidity and mortality worldwide. The main reasons are the inadequacy of preventive health services and delays in diagnosis due to the increasing population, the failure of physicians to apply guide-based treatments, the lack of continuous patient follow-up, and the low compliance of patients with doctors' recommendations. Artificial intelligence (AI)-based clinical decision support systems (CDSSs) are systems that support complex decision-making processes by using AI techniques such as data analysis, foresight, and optimization. Artificial intelligence-based CDSSs play an important role in patient care by providing more accurate and personalized information to healthcare professionals in risk assessment, diagnosis, treatment optimization, and monitoring and early warning of CVD. These are just some examples, and the use of AI for CVD decision support systems is rapidly evolving. However, for these systems to be fully reliable and effective, they need to be trained with accurate data and carefully evaluated by medical professionals.

Ten Topics to Get Started in Medical Informatics Research

The vast and heterogeneous data being constantly generated in clinics can provide great wealth for patients and research alike. The quickly evolving field of medical informatics research has contributed numerous concepts, algorithms, and standards to facilitate this development. However, these difficult relationships, complex terminologies, and multiple implementations can present obstacles for people who want to get active in the field. With a particular focus on medical informatics research conducted in Germany, we present in our Viewpoint a set of 10 important topics to improve the overall interdisciplinary communication between different stakeholders (eg, physicians, computational experts, experimentalists, students, patient representatives). This may lower the barriers to entry and offer a starting point for collaborations at different levels. The suggested topics are briefly introduced, then general best practice guidance is given, and further resources for in-depth reading or hands-on tutorials are recommended. In addition, the topics are set to cover current aspects and open research gaps of the medical informatics domain, including data regulations and concepts; data harmonization and processing; and data evaluation, visualization, and dissemination. In addition, we give an example on how these topics can be integrated in a medical informatics curriculum for higher education. By recognizing these topics, readers will be able to (1) set clinical and research data into the context of medical informatics, understanding what is possible to achieve with data or how data should be handled in terms of data privacy and storage; (2) distinguish current interoperability standards and obtain first insights into the processes leading to effective data transfer and analysis; and (3) value the use of newly developed technical approaches to utilize the full potential of clinical data.

The Application of Knowledge-Based Clinical Decision Support Systems to Enhance Adherence to Evidence-Based Medicine in Chronic Disease

Among the technology-based solutions, clinical decision support systems (CDSSs) have the ability to keep up with clinicians with the latest evidence in a smart way. Hence, the main objective of our study was to investigate the applicability and characteristics of CDSSs regarding chronic disease. The Web of Science, Scopus, OVID, and PubMed databases were searched using keywords from January 2000 to February 2023. The review was completed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist. Then, an analysis was done to determine the characteristics and applicability of CDSSs. The quality of the appraisal was assessed using the Mixed Methods Appraisal Tool checklist (MMAT). A systematic database search yielded 206 citations. Eventually, 38 articles from sixteen countries met the inclusion criteria and were accepted for final analysis. The main approaches of all studies can be classified into adherence to evidence-based medicine (84.2%), early and accurate diagnosis (81.6%), identifying high-risk patients (50%), preventing medical errors (47.4%), providing up-to-date information to healthcare providers (36.8%), providing patient care remotely (21.1%), and standardizing care (71.1%). The most common features among the knowledge-based CDSSs included providing guidance and advice for physicians (92.11%), generating patient-specific recommendations (84.21%), integrating into electronic medical records (60.53%), and using alerts or reminders (60.53%). Among thirteen different methods to translate the knowledge of evidence into machine-interpretable knowledge, 34.21% of studies utilized the rule-based logic technique while 26.32% of studies used rule-based decision tree modeling. For CDSS development and translating knowledge, diverse methods and techniques were applied. Therefore, the development of a standard framework for the development of knowledge-based decision support systems should be considered by informaticians.

One Heartbeat Away from a Prediction Model for Cardiovascular Diseases in Patients with Chronic Kidney Disease: A Systematic Review

INTRODUCTION

Patients with chronic kidney disease (CKD) have a high risk of cardiovascular disease (CVD). Prediction models, combining clinical and laboratory characteristics, are commonly used to estimate an individual's CVD risk. However, these models are not specifically developed for patients with CKD and may therefore be less accurate. In this review, we aim to give an overview of CVD prognostic studies available, and their methodological quality, specifically for patients with CKD.

METHODS

MEDLINE was searched for papers reporting CVD prognostic studies in patients with CKD published between 2012 and 2021. Characteristics regarding patients, study design, outcome measurement, and prediction models were compared between included studies. The risk of bias of studies reporting on prognostic factors or the development/validation of a prediction model was assessed with, respectively, the QUIPS and PROBAST tool.

RESULTS

In total, 134 studies were included, of which 123 studies tested the incremental value of one or more predictors to existing models or common risk factors, while only 11 studies reported on the development or validation of a prediction model. Substantial heterogeneity in cohort and study characteristics, such as sample size, event rate, and definition of outcome measurements, was observed across studies. The most common predictors were age (87%), sex (75%), diabetes (70%), and estimated glomerular filtration rate (69%). Most of the studies on prognostic factors have methodological shortcomings, mostly due to a lack of reporting on clinical and methodological information. Of the 11 studies on prediction models, six developed and internally validated a model and four externally validated existing or developed models. Only one study on prognostic models showed a low risk of bias and high applicability.

CONCLUSION

A large quantity of prognostic studies has been published, yet their usefulness remains unclear due to incomplete presentation, and lack of external validation of prognostic models. Our review can be used to select the most appropriate prognostic model depending on the patient population, outcome, and risk of bias. Future collaborative efforts should aim at improving existing models by externally validating them, evaluating the addition of new predictors, and assessment of the clinical impact.

REGISTRATION

We have registered the protocol of our systematic review on PROSPERO (CRD42021228043).

Optimizing cardiovascular risk assessment and registration in a developing cardiovascular learning health care system: Women benefit most

Since 2015 we organized a uniform, structured collection of a fixed set of cardiovascular risk factors according the (inter)national guidelines on cardiovascular risk management. We evaluated the current state of a developing cardiovascular towards learning healthcare system-the Utrecht Cardiovascular Cohort Cardiovascular Risk Management (UCC-CVRM)-and its potential effect on guideline adherence in cardiovascular risk management. We conducted a before-after study comparing data from patients included in UCC-CVRM (2015-2018) and patients treated in our center before UCC-CVRM (2013-2015) who would have been eligible for UCC-CVRM using the Utrecht Patient Oriented Database (UPOD). Proportions of cardiovascular risk factor measurement before and after UCC-CVRM initiation were compared, as were proportions of patients that required (change of) blood pressure, lipid, or blood glucose lowering treatment. We estimated the likelihood to miss patients with hypertension, dyslipidemia, and elevated HbA1c before UCC-CVRM for the whole cohort and stratified for sex. In the present study, patients included up to October 2018 (n = 1904) were matched with 7195 UPOD patients with similar age, sex, department of referral and diagnose description. Completeness of risk factor measurement increased, ranging from 0% -77% before to 82%-94% after UCC-CVRM initiation. Before UCC-CVRM, we found more unmeasured risk factors in women compared to men. This sex-gap resolved in UCC-CVRM. The likelihood to miss hypertension, dyslipidemia, and elevated HbA1c was reduced by 67%, 75% and 90%, respectively, after UCC-CVRM initiation. A finding more pronounced in women compared to men. In conclusion, a systematic registration of the cardiovascular risk profile substantially improves guideline adherent assessment and decreases the risk of missing patients with elevated levels with an indication for treatment. The sex-gap disappeared after UCC-CVRM initiation. Thus, an LHS approach contributes to a more inclusive insight into quality of care and prevention of cardiovascular disease (progression).

Interpretability of Clinical Decision Support Systems Based on Artificial Intelligence from Technological and Medical Perspective: A Systematic Review

Background

Artificial intelligence (AI) has developed rapidly, and its application extends to clinical decision support system (CDSS) for improving healthcare quality. However, the interpretability of AI-driven CDSS poses significant challenges to widespread application.

Objective

This study is a review of the knowledge-based and data-based CDSS literature regarding interpretability in health care. It highlights the relevance of interpretability for CDSS and the area for improvement from technological and medical perspectives.

Methods

A systematic search was conducted on the interpretability-related literature published from 2011 to 2020 and indexed in the five databases: Web of Science, PubMed, ScienceDirect, Cochrane, and Scopus. Journal articles that focus on the interpretability of CDSS were included for analysis. Experienced researchers also participated in manually reviewing the selected articles for inclusion/exclusion and categorization.

Results

Based on the inclusion and exclusion criteria, 20 articles from 16 journals were finally selected for this review. Interpretability, which means a transparent structure of the model, a clear relationship between input and output, and explainability of artificial intelligence algorithms, is essential for CDSS application in the healthcare setting. Methods for improving the interpretability of CDSS include ante-hoc methods such as fuzzy logic, decision rules, logistic regression, decision trees for knowledge-based AI, and white box models, post hoc methods such as feature importance, sensitivity analysis, visualization, and activation maximization for black box models. A number of factors, such as data type, biomarkers, human-AI interaction, needs of clinicians, and patients, can affect the interpretability of CDSS.

Conclusions

The review explores the meaning of the interpretability of CDSS and summarizes the current methods for improving interpretability from technological and medical perspectives. The results contribute to the understanding of the interpretability of CDSS based on AI in health care. Future studies should focus on establishing formalism for defining interpretability, identifying the properties of interpretability, and developing an appropriate and objective metric for interpretability; in addition, the user's demand for interpretability and how to express and provide explanations are also the directions for future research.

Using an Integrated Framework to Investigate the Facilitators and Barriers of Health Information Technology Implementation in Noncommunicable Disease Management: Systematic Review

BACKGROUND

Noncommunicable disease (NCD) management is critical for reducing attributable health burdens. Although health information technology (HIT) is a crucial strategy to improve chronic disease management, many health care systems have failed in implementing HIT. There has been a lack of research on the implementation process of HIT for chronic disease management.

OBJECTIVE

We aimed to identify the barriers and facilitators of HIT implementation, analyze how these factors influence the implementation process, and identify key areas for future action. We will develop a framework for understanding implementation determinants to synthesize available evidence.

METHODS

We conducted a systematic review to understand the barriers and facilitators of the implementation process. We searched MEDLINE, Cochrane, Embase, Scopus, and CINAHL for studies published between database inception and May 5, 2022. Original studies involving HIT-related interventions for NCD management published in peer-reviewed journals were included. Studies that did not discuss relevant outcome measures or did not have direct contact with or observation of stakeholders were excluded. The analysis was conducted in 2 parts. In part 1, we analyzed how the intrinsic attributes of HIT interventions affect the successfulness of implementation by using the intervention domain of the Consolidated Framework for Implementation Research (CFIR). In part 2, we focused on the extrinsic factors of HIT using an integrated framework, which was developed based on the CFIR and the levels of change framework by Ferlie and Shortell.

RESULTS

We identified 51 papers with qualitative, mixed-method, and cross-sectional methodologies. Included studies were heterogeneous regarding disease populations and HIT interventions. In part 1, having a relative advantage over existing health care systems was the most prominent intrinsic facilitator (eg, convenience, improvement in quality of care, and increase in access). Poor usability was the most noted intrinsic barrier of HIT. In part 2, we mapped the various factors of implementation to the integrated framework (the coordinates are shown as level of change-CFIR). The key barriers to the extrinsic factors of HIT included health literacy and lack of digital skills (individual-characteristics of individuals). The key facilitators included physicians' suggestions, cooperation (interpersonal-process), integration into a workflow, and adequate management of data (organizational-inner setting). The importance of health data security was identified. Self-efficacy issues of patients and organizational readiness for implementation were highlighted.

CONCLUSIONS

Internal factors of HIT and external human factors of implementation interplay in HIT implementation for chronic disease management. Strategies for improvement include ensuring HIT has a relative advantage over existing health care; tackling usability issues; and addressing underlying socioeconomic, interpersonal, and organizational conditions. Further research should focus on studying various stakeholders, such as service providers and administrative workforces; various disease populations, such as those with obesity and mental diseases; and various countries, including low- and middle-income countries.

Integration of Artificial Intelligence and Blockchain Technology in Healthcare and Agriculture

Over the last decade, the healthcare sector has accelerated its digitization and electronic health records (EHRs). As information technology progresses, the notion of intelligent health also gathers popularity. By combining technologies such as the internet of things (IoT) and artificial intelligence (AI), innovative healthcare modifies and enhances traditional medical systems in terms of efficiency, service, and personalization. On the other side, intelligent healthcare systems are incredibly vulnerable to data breaches and other malicious assaults. Recently, blockchain technology has emerged as a potentially transformative option for enhancing data management, access control, and integrity inside healthcare systems. Integrating these advanced approaches in agriculture is critical for managing food supply chains, drug supply chains, quality maintenance, and intelligent prediction. This study reviews the literature, formulates a research topic, and analyzes the applicability of blockchain to the agriculture/food industry and healthcare, with a particular emphasis on AI and IoT. This article summarizes research on the newest blockchain solutions paired with AI technologies for strengthening and inventing new technological standards for the healthcare ecosystems and food industry.

Data mining to retrieve smoking status from electronic health records in general practice. EUROPEAN HEART JOURNAL. DIGITAL HEALTH 2022;3:437-444. [PMID: 36712169 PMCID: PMC9707867 DOI: 10.1093/ehjdh/ztac031]

Aims

Optimize and assess the performance of an existing data mining algorithm for smoking status from hospital electronic health records (EHRs) in general practice EHRs.

Methods and results

We optimized an existing algorithm in a training set containing all clinical notes from 498 individuals (75 712 contact moments) from the Julius General Practitioners' Network (JGPN). Each moment was classified as either 'current smoker', 'former smoker', 'never smoker', or 'no information'. As a reference, we manually reviewed EHRs. Algorithm performance was assessed in an independent test set (n = 494, 78 129 moments) using precision, recall, and F1-score. Test set algorithm performance for 'current smoker' was precision 79.7%, recall 78.3%, and F1-score 0.79. For former smoker, it was precision 73.8%, recall 64.0%, and F1-score 0.69. For never smoker, it was precision 92.0%, recall 74.9%, and F1-score 0.83. On a patient level, performance for ever smoker (current and former smoker combined) was precision 87.9%, recall 94.7%, and F1-score 0.91. For never smoker, it was 98.0, 82.0, and 0.89%, respectively. We found a more narrative writing style in general practice than in hospital EHRs.

Conclusion

Data mining can successfully retrieve smoking status information from general practice clinical notes with a good performance for classifying ever and never smokers. Differences between general practice and hospital EHRs call for optimization of data mining algorithms when applied beyond a primary development setting.

The Applied Data Analytics in Medicine Program: Lessons Learned From Four Years' Experience With Personalizing Health Care in an Academic Teaching Hospital

The University Medical Center (UMC) Utrecht piloted a hospital-wide innovation data analytics program over the past 4 years. The goal was, based on available data and innovative data analytics methodologies, to answer clinical questions to improve patient care. In this viewpoint, we aimed to support and inspire others pursuing similar efforts by sharing the three principles of the program: the data analytics value chain (data, insight, action, value), the innovation funnel (structured innovation approach with phases and gates), and the multidisciplinary team (patients, clinicians, and data scientists). We also discussed our most important lessons learned: the importance of a clinical question, collaboration challenges between health care professionals and different types of data scientists, the win-win result of our collaboration with external partners, the prerequisite of available meaningful data, the (legal) complexity of implementation, organizational power, and the embedding of collaborative efforts in the health care system as a whole.

Big Data and Digital Solutions: Laying the Foundation for Cardiovascular Population Management CME

There are huge gaps in evidence-based cardiovascular care at the national, organizational, practice, and provider level that can be attributed to variation in provider attitudes, lack of incentives for positive change and care standardization, and observed uncertainty in clinical decision making. Big data analytics and digital application platforms-such as patient care dashboards, clinical decision support systems, mobile patient engagement applications, and key performance indicators-offer unique opportunities for value-based healthcare delivery and efficient cardiovascular population management. Successful implementation of big data solutions must include a multidisciplinary approach, including investment in big data platforms, harnessing technology to create novel digital applications, developing digital solutions that can inform the actions of clinical and policy decision makers and relevant stakeholders, and optimizing engagement strategies with the public and information-empowered patients.

Real-time imputation of missing predictor values improved the application of prediction models in daily practice

OBJECTIVES

In clinical practice, many prediction models cannot be used when predictor values are missing. We, therefore, propose and evaluate methods for real-time imputation.

STUDY DESIGN AND SETTING

We describe (i) mean imputation (where missing values are replaced by the sample mean), (ii) joint modeling imputation (JMI, where we use a multivariate normal approximation to generate patient-specific imputations), and (iii) conditional modeling imputation (CMI, where a multivariable imputation model is derived for each predictor from a population). We compared these methods in a case study evaluating the root mean squared error (RMSE) and coverage of the 95% confidence intervals (i.e., the proportion of confidence intervals that contain the true predictor value) of imputed predictor values.

RESULTS

-RMSE was lowest when adopting JMI or CMI, although imputation of individual predictors did not always lead to substantial improvements as compared to mean imputation. JMI and CMI appeared particularly useful when the values of multiple predictors of the model were missing. Coverage reached the nominal level (i.e., 95%) for both CMI and JMI.

CONCLUSION

Multiple imputations using either CMI or JMI is recommended when dealing with missing predictor values in real-time settings.

Real-time imputation of missing predictor values in clinical practice

Aims

Use of prediction models is widely recommended by clinical guidelines, but usually requires complete information on all predictors, which is not always available in daily practice. We aim to describe two methods for real-time handling of missing predictor values when using prediction models in practice.

Methods and results

We compare the widely used method of mean imputation (M-imp) to a method that personalizes the imputations by taking advantage of the observed patient characteristics. These characteristics may include both prediction model variables and other characteristics (auxiliary variables). The method was implemented using imputation from a joint multivariate normal model of the patient characteristics (joint modelling imputation; JMI). Data from two different cardiovascular cohorts with cardiovascular predictors and outcome were used to evaluate the real-time imputation methods. We quantified the prediction model's overall performance [mean squared error (MSE) of linear predictor], discrimination (c-index), calibration (intercept and slope), and net benefit (decision curve analysis). When compared with mean imputation, JMI substantially improved the MSE (0.10 vs. 0.13), c-index (0.70 vs. 0.68), and calibration (calibration-in-the-large: 0.04 vs. 0.06; calibration slope: 1.01 vs. 0.92), especially when incorporating auxiliary variables. When the imputation method was based on an external cohort, calibration deteriorated, but discrimination remained similar.

Conclusions

We recommend JMI with auxiliary variables for real-time imputation of missing values, and to update imputation models when implementing them in new settings or (sub)populations.

Low-Density Lipoprotein Cholesterol Target Attainment in Patients With Established Cardiovascular Disease: Analysis of Routine Care Data

BACKGROUND

Direct feedback on quality of care is one of the key features of a learning health care system (LHS), enabling health care professionals to improve upon the routine clinical care of their patients during practice.

OBJECTIVE

This study aimed to evaluate the potential of routine care data extracted from electronic health records (EHRs) in order to obtain reliable information on low-density lipoprotein cholesterol (LDL-c) management in cardiovascular disease (CVD) patients referred to a tertiary care center.

METHODS

We extracted all LDL-c measurements from the EHRs of patients with a history of CVD referred to the University Medical Center Utrecht. We assessed LDL-c target attainment at the time of referral and per year. In patients with multiple measurements, we analyzed LDL-c trajectories, truncated at 6 follow-up measurements. Lastly, we performed a logistic regression analysis to investigate factors associated with improvement of LDL-c at the next measurement.

RESULTS

Between February 2003 and December 2017, 250,749 LDL-c measurements were taken from 95,795 patients, of whom 23,932 had a history of CVD. At the time of referral, 51% of patients had not reached their LDL-c target. A large proportion of patients (55%) had no follow-up LDL-c measurements. Most of the patients with repeated measurements showed no change in LDL-c levels over time: the transition probability to remain in the same category was up to 0.84. Sequence clustering analysis showed more women (odds ratio 1.18, 95% CI 1.07-1.10) in the cluster with both most measurements off target and the most LDL-c measurements furthest from the target. Timing of drug prescription was difficult to determine from our data, limiting the interpretation of results regarding medication management.

CONCLUSIONS

Routine care data can be used to provide feedback on quality of care, such as LDL-c target attainment. These routine care data show high off-target prevalence and little change in LDL-c over time. Registrations of diagnosis; follow-up trajectory, including primary and secondary care; and medication use need to be improved in order to enhance usability of the EHR system for adequate feedback.