Assessing the use of HL7 FHIR for implementing the FAIR guiding principles: a case study of the MIMIC-IV Emergency Department module

Objectives

To provide a real-world example on how and to what extent Health Level Seven Fast Healthcare Interoperability Resources (FHIR) implements the Findable, Accessible, Interoperable, and Reusable (FAIR) guiding principles for scientific data. Additionally, presents a list of FAIR implementation choices for supporting future FAIR implementations that use FHIR.

Materials and methods

A case study was conducted on the Medical Information Mart for Intensive Care-IV Emergency Department (MIMIC-ED) dataset, a deidentified clinical dataset converted into FHIR. The FAIRness of this dataset was assessed using a set of common FAIR assessment indicators.

Results

The FHIR distribution of MIMIC-ED, comprising an implementation guide and demo data, was more FAIR compared to the non-FHIR distribution. The FAIRness score increased from 60 to 82 out of 95 points, a relative improvement of 37%. The most notable improvements were observed in interoperability, with a score increase from 5 to 19 out of 19 points, and reusability, with a score increase from 8 to 14 out of 24 points. A total of 14 FAIR implementation choices were identified.

Discussion

Our work examined how and to what extent the FHIR standard contributes to FAIR data. Challenges arose from interpreting the FAIR assessment indicators. This study stands out for providing a real-world example of a dataset that was made more FAIR using FHIR.

Conclusion

To the best of our knowledge, this is the first study that formally assessed the conformance of a FHIR dataset to the FAIR principles. FHIR improved the accessibility, interoperability, and reusability of MIMIC-ED. Future research should focus on implementing FHIR in research data infrastructures.

Strain on Scarce Intensive Care Beds Drives Reduced Patient Volumes, Patient Selection, and Worse Outcome: A National Cohort Study

OBJECTIVES

Strain on ICUs during the COVID-19 pandemic required stringent triage at the ICU to distribute resources appropriately. This could have resulted in reduced patient volumes, patient selection, and worse outcome of non-COVID-19 patients, especially during the pandemic peaks when the strain on ICUs was extreme. We analyzed this potential impact on the non-COVID-19 patients.

DESIGN

A national cohort study.

SETTING

Data of 71 Dutch ICUs.

PARTICIPANTS

A total of 120,393 patients in the pandemic non-COVID-19 cohort (from March 1, 2020 to February 28, 2022) and 164,737 patients in the prepandemic cohort (from January 1, 2018 to December 31, 2019).

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Volume, patient characteristics, and mortality were compared between the pandemic non-COVID-19 cohort and the prepandemic cohort, focusing on the pandemic period and its peaks, with attention to strata of specific admission types, diagnoses, and severity. The number of admitted non-COVID-19 patients during the pandemic period and its peaks were, respectively, 26.9% and 34.2% lower compared with the prepandemic cohort. The pandemic non-COVID-19 cohort consisted of fewer medical patients (48.1% vs. 50.7%), fewer patients with comorbidities (36.5% vs. 40.6%), and more patients on mechanical ventilation (45.3% vs. 42.4%) and vasoactive medication (44.7% vs. 38.4%) compared with the prepandemic cohort. Case-mix adjusted mortality during the pandemic period and its peaks was higher compared with the prepandemic period, odds ratios were, respectively, 1.08 (95% CI, 1.05-1.11) and 1.10 (95% CI, 1.07-1.13).

CONCLUSIONS

In non-COVID-19 patients the strain on healthcare has driven lower patient volume, selection of fewer comorbid patients who required more intensive support, and a modest increase in the case-mix adjusted mortality.

The effect of computerised decision support alerts tailored to intensive care on the administration of high-risk drug combinations, and their monitoring: a cluster randomised stepped-wedge trial

BACKGROUND

Drug-drug interactions (DDIs) can harm patients admitted to the intensive care unit (ICU). Yet, clinical decision support systems (CDSSs) aimed at helping physicians prevent DDIs are plagued by low-yield alerts, causing alert fatigue and compromising patient safety. The aim of this multicentre study was to evaluate the effect of tailoring potential DDI alerts to the ICU setting on the frequency of administered high-risk drug combinations.

METHODS

We implemented a cluster randomised stepped-wedge trial in nine ICUs in the Netherlands. Five ICUs already used potential DDI alerts. Patients aged 18 years or older admitted to the ICU with at least two drugs administered were included. Our intervention was an adapted CDSS, only providing alerts for potential DDIs considered as high risk. The intervention was delivered at the ICU level and targeted physicians. We hypothesised that showing only relevant alerts would improve CDSS effectiveness and lead to a decreased number of administered high-risk drug combinations. The order in which the intervention was implemented in the ICUs was randomised by an independent researcher. The primary outcome was the number of administered high-risk drug combinations per 1000 drug administrations per patient and was assessed in all included patients. This trial was registered in the Netherlands Trial Register (identifier NL6762) on Nov 26, 2018, and is now closed.

FINDINGS

In total, 10 423 patients admitted to the ICU between Sept 1, 2018, and Sept 1, 2019, were assessed and 9887 patients were included. The mean number of administered high-risk drug combinations per 1000 drug administrations per patient was 26·2 (SD 53·4) in the intervention group (n=5534), compared with 35·6 (65·0) in the control group (n=4353). Tailoring potential DDI alerts to the ICU led to a 12% decrease (95% CI 5-18%; p=0·0008) in the number of administered high-risk drug combinations per 1000 drug administrations per patient, after adjusting for clustering and prognostic factors.

INTERPRETATION

This cluster randomised stepped-wedge trial showed that tailoring potential DDI alerts to the ICU setting significantly reduced the number of administered high-risk drug combinations. Our list of high-risk drug combinations can be used in other ICUs, and our strategy of tailoring alerts based on clinical relevance could be applied to other clinical settings.

FUNDING

ZonMw.

Quality improvement of Dutch ICUs from 2009 to 2021: A registry based observational study

PURPOSE

To investigate the development in quality of ICU care over time using the Dutch National Intensive Care Evaluation (NICE) registry.

MATERIALS AND METHODS

We included data from all ICU admissions in the Netherlands from those ICUs that submitted complete data between 2009 and 2021 to the NICE registry. We determined median and interquartile range for eight quality indicators. To evaluate changes over time on the indicators, we performed multilevel regression analyses, once without and once with the COVID-19 years 2020 and 2021 included. Additionally we explored between-ICU heterogeneity by calculating intraclass correlation coefficients (ICC).

RESULTS

705,822 ICU admissions from 55 (65%) ICUs were included in the analyses. ICU length of stay (LOS), duration of mechanical ventilation (MV), readmissions, in-hospital mortality, hypoglycemia, and pressure ulcers decreased significantly between 2009 and 2019 (OR <1). After including the COVID-19 pandemic years, the significant change in MV duration, ICU LOS, and pressure ulcers disappeared. We found an ICC ≤0.07 on the quality indicators for all years, except for pressure ulcers with an ICC of 0.27 for 2009 to 2021.

CONCLUSIONS

Quality of Dutch ICU care based on seven indicators significantly improved from 2009 to 2019 and between-ICU heterogeneity is medium to small, except for pressure ulcers. The COVID-19 pandemic disturbed the trend in quality improvement, but unaltered the between-ICU heterogeneity.

Characteristics and outcome of COVID-19 patients admitted to the ICU: a nationwide cohort study on the comparison between the consecutive stages of the COVID-19 pandemic in the Netherlands, an update

BACKGROUND

Previously, we reported a decreased mortality rate among patients with COVID-19 who were admitted at the ICU during the final upsurge of the second wave (February-June 2021) in the Netherlands. We examined whether this decrease persisted during the third wave and the phases with decreasing incidence of COVID-19 thereafter and brought up to date the information on patient characteristics.

METHODS

Data from the National Intensive Care Evaluation (NICE)-registry of all COVID-19 patients admitted to an ICU in the Netherlands were used. Patient characteristics and rates of in-hospital mortality (the primary outcome) during the consecutive periods after the first wave (periods 2-9, May 25, 2020-January 31, 2023) were compared with those during the first wave (period 1, February-May 24, 2020).

RESULTS

After adjustment for patient characteristics and ICU occupancy rate, the mortality risk during the initial upsurge of the third wave (period 6, October 5, 2021-January, 31, 2022) was similar to that of the first wave (ORadj = 1.01, 95%-CI [0.88-1.16]). The mortality rates thereafter decreased again (e.g., period 9, October 5, 2022-January, 31, 2023: ORadj = 0.52, 95%-CI [0.41-0.66]). Among the SARS-CoV-2 positive patients, there was a huge drop in the proportion of patients with COVID-19 as main reason for ICU admission: from 88.2% during the initial upsurge of the third wave to 51.7%, 37.3%, and 41.9% for the periods thereafter. Restricting the analysis to these patients did not modify the results on mortality.

CONCLUSIONS

The results show variation in mortality rates among critically ill COVID-19 patients across the calendar time periods that is not explained by differences in case-mix and ICU occupancy rates or by varying proportions of patients with COVID-19 as main reason for ICU admission. The consistent increase in mortality during the initial, rising phase of each separate wave might be caused by the increased virulence of the contemporary virus strain and lacking immunity to the new strain, besides unmeasured patient-, treatment- and healthcare system characteristics.

Adverse drug events caused by three high-risk drug-drug interactions in patients admitted to intensive care units: A multicentre retrospective observational study

AIMS

Knowledge about adverse drug events caused by drug-drug interactions (DDI-ADEs) is limited. We aimed to provide detailed insights about DDI-ADEs related to three frequent, high-risk potential DDIs (pDDIs) in the critical care setting: pDDIs with international normalized ratio increase (INR+ ) potential, pDDIs with acute kidney injury (AKI) potential, and pDDIs with QTc prolongation potential.

METHODS

We extracted routinely collected retrospective data from electronic health records of intensive care units (ICUs) patients (≥18 years), admitted to ten hospitals in the Netherlands between January 2010 and September 2019. We used computerized triggers (e-triggers) to preselect patients with potential DDI-ADEs. Between September 2020 and October 2021, clinical experts conducted a retrospective manual patient chart review on a subset of preselected patients, and assessed causality, severity, preventability, and contribution to ICU length of stay of DDI-ADEs using internationally prevailing standards.

RESULTS

In total 85 422 patients with ≥1 pDDI were included. Of these patients, 32 820 (38.4%) have been exposed to one of the three pDDIs. In the exposed group, 1141 (3.5%) patients were preselected using e-triggers. Of 237 patients (21%) assessed, 155 (65.4%) experienced an actual DDI-ADE; 52.9% had severity level of serious or higher, 75.5% were preventable, and 19.3% contributed to a longer ICU length of stay. The positive predictive value was the highest for DDI-INR+ e-trigger (0.76), followed by DDI-AKI e-trigger (0.57).

CONCLUSION

The highly preventable nature and severity of DDI-ADEs, calls for action to optimize ICU patient safety. Use of e-triggers proved to be a promising preselection strategy.

Correctly structured problem lists lead to better and faster clinical decision-making in electronic health records compared to non-curated problem lists: A single-blinded crossover randomized controlled trial

BACKGROUND

Correctly structured problem lists in electronic health records (EHRs) offer major benefits to patient care. Without structured lists, diagnosis information is often scatteredly documented in free text, which may contribute to errors and inefficient information retrieval. This study aims to assess whether EHRs with correctly structured problem lists result in better and faster clinical decision-making compared to non-curated problem lists.

METHODS

Two versions of two patient records (A and B) were created in an EHR training environment: one version included diagnosis information structured and coded on the problem list ("correctly structured problem list"), the other version had missing problem list diagnoses and diagnosis information partly documented in free text ("non-curated problem list"). In this single-blinded crossover randomized controlled trial, healthcare providers, who can prescribe medications, from two Dutch university medical center locations first evaluated a randomized version of patient A, then B. Participants were asked to motivate their answer to two medication prescription questions. One (test) question required information similarly presented in both record versions. The second (comparison) question required information documented on problem lists and/or in notes. The primary outcome measure was the correctness of the motivated answer to the comparison question. Secondary outcome measure was the time to answer and motivate both questions correctly.

RESULTS

As planned, 160 participants enrolled. Two were excluded for not meeting inclusion criteria. Correctly structured problem lists increased providers' ability to answer the comparison question correctly (56.3 % versus 33.5 %, McNemar odds ratio 2.80 (1.65-4.93) 95 %-CI). Median time to answer both questions correctly was significantly lower for EHRs with correctly structured problem lists (Wilcoxon-signed-rank test p = 0.00002, with incorrect answers coded equally at slowest time).

CONCLUSIONS

Correctly structured problem lists lead to better and faster clinical decision-making. Increased structured problem lists usage may be warranted for which implementation policies should be developed.

Influenza Season and Outcome After Elective Cardiac Surgery: An Observational Cohort Study

BACKGROUND

An asymptomatic respiratory viral infection during cardiac surgery could lead to pulmonary complications and increased mortality. For elective surgery, testing for respiratory viral infection before surgery or vaccination could reduce the number of these pulmonary complications. The aim of this study was to investigate the association between influenzalike illness (ILI) seasons and prolonged mechanical ventilation and inhospital mortality in a Dutch cohort of adult elective cardiac surgery patients.

METHODS

Cardiac surgery patients who were admitted to the intensive care unit between January 1, 2014, and February 1, 2020, were included. The primary endpoint was the duration of invasive mechanical ventilation in the ILI season compared with baseline season. Secondary endpoints were the median Pao2 to fraction of inspired oxygen ratio on days 1, 3, and 7 and postoperative inhospital mortality.

RESULTS

A total of 42,277 patients underwent cardiac surgery, 12,994 (30.7%) in the ILI season, 15,843 (37.5%) in the intermediate season, and 13,440 (31.8%) in the baseline season. No hazard rates indicative of a longer duration of invasive mechanical ventilation during the ILI season were found. No differences were found for the median Pao2 to fraction of inspired oxygen ratio between seasons. However, inhospital mortality was higher in the ILI season compared with baseline season (odds ratio 1.67; 95% CI, 1.14-2.46).

CONCLUSIONS

Patients undergoing cardiac surgery during the ILI season were at increased risk of inhospital mortality compared with patients in the baseline season. No evidence was found that this difference is caused by direct postoperative pulmonary complications.

Acute kidney injury associated with nephrotoxic drugs in critically ill patients: a multicenter cohort study using electronic health record data

Background

Nephrotoxic drugs frequently cause acute kidney injury (AKI) in adult intensive care unit (ICU) patients. However, there is a lack of large pharmaco-epidemiological studies investigating the associations between drugs and AKI. Importantly, AKI risk factors may also be indications or contraindications for drugs and thereby confound the associations. Here, we aimed to estimate the associations between commonly administered (potentially) nephrotoxic drug groups and AKI in adult ICU patients whilst adjusting for confounding.

Methods

In this multicenter retrospective observational study, we included adult ICU admissions to 13 Dutch ICUs. We measured exposure to 44 predefined (potentially) nephrotoxic drug groups. The outcome was AKI during ICU admission. The association between each drug group and AKI was estimated using etiological cause-specific Cox proportional hazard models and adjusted for confounding. To facilitate an (independent) informed assessment of residual confounding, we manually identified drug group-specific confounders using a large drug knowledge database and existing literature.

Results

We included 92 616 ICU admissions, of which 13 492 developed AKI (15%). We found 14 drug groups to be associated with a higher hazard of AKI after adjustment for confounding. These groups included established (e.g. aminoglycosides), less well established (e.g. opioids) and controversial (e.g. sympathomimetics with α- and β-effect) drugs.

Conclusions

The results confirm existing insights and provide new ones regarding drug associated AKI in adult ICU patients. These insights warrant caution and extra monitoring when prescribing nephrotoxic drugs in the ICU and indicate which drug groups require further investigation.

Conservative versus Liberal Oxygenation Targets in Intensive Care Unit Patients (ICONIC): A Randomized Clinical Trial

Rationale: Supplemental oxygen is widely administered to ICU patients, but appropriate oxygenation targets remain unclear. Objectives: This study aimed to determine whether a low-oxygenation strategy would lower 28-day mortality compared with a high-oxygenation strategy. Methods: This randomized multicenter trial included mechanically ventilated ICU patients with an expected ventilation duration of at least 24 hours. Patients were randomized 1:1 to a low-oxygenation (PaO2, 55-80 mm Hg; or oxygen saturation as measured by pulse oximetry, 91-94%) or high-oxygenation (PaO2, 110-150 mm Hg; or oxygen saturation as measured by pulse oximetry, 96-100%) target until ICU discharge or 28 days after randomization, whichever came first. The primary outcome was 28-day mortality. The study was stopped prematurely because of the COVID-19 pandemic when 664 of the planned 1,512 patients were included. Measurements and Main Results: Between November 2018 and November 2021, a total of 664 patients were included in the trial: 335 in the low-oxygenation group and 329 in the high-oxygenation group. The median achieved PaO2 was 75 mm Hg (interquartile range, 70-84) and 115 mm Hg (interquartile range, 100-129) in the low- and high-oxygenation groups, respectively. At Day 28, 129 (38.5%) and 114 (34.7%) patients had died in the low- and high-oxygenation groups, respectively (risk ratio, 1.11; 95% confidence interval, 0.9-1.4; P = 0.30). At least one serious adverse event was reported in 12 (3.6%) and 17 (5.2%) patients in the low- and high-oxygenation groups, respectively. Conclusions: Among mechanically ventilated ICU patients with an expected mechanical ventilation duration of at least 24 hours, using a low-oxygenation strategy did not result in a reduction of 28-day mortality compared with a high-oxygenation strategy. Clinical trial registered with the National Trial Register and the International Clinical Trials Registry Platform (NTR7376).

Determining and assessing characteristics of data element names impacting the performance of annotation using Usagi

INTRODUCTION

Hospitals generate large amounts of data and this data is generally modeled and labeled in a proprietary way, hampering its exchange and integration. Manually annotating data element names to internationally standardized data element identifiers is a time-consuming effort. Tools can support performing this task automatically. This study aimed to determine what factors influence the quality of automatic annotations.

METHODS

Data element names were used from the Dutch COVID-19 ICU Data Warehouse containing data on intensive care patients with COVID-19 from 25 hospitals in the Netherlands. In this data warehouse, the data had been merged using a proprietary terminology system while also storing the original hospital labels (synonymous names). Usagi, an OHDSI annotation tool, was used to perform the annotation for the data. A gold standard was used to determine if Usagi made correct annotations. Logistic regression was used to determine if the number of characters, number of words, match score (Usagi's certainty) and hospital label origin influenced Usagi's performance to annotate correctly.

RESULTS

Usagi automatically annotated 30.5% of the data element names correctly and 5.5% of the synonymous names. The match score is the best predictor for Usagi finding the correct annotation. It was determined that the AUC of data element names was 0.651 and 0.752 for the synonymous names respectively. The AUC for the individual hospital label origins varied between 0.460 to 0.905.

DISCUSSION

The results show that Usagi performed better to annotate the data element names than the synonymous names. The hospital origin in the synonymous names dataset was associated with the amount of correctly annotated concepts. Hospitals that performed better had shorter synonymous names and fewer words. Using shorter data element names or synonymous names should be considered to optimize the automatic annotating process. Overall, the performance of Usagi is too poor to completely rely on for automatic annotation.

Age Moderates the Effect of Obesity on Mortality Risk in Critically Ill Patients With COVID-19: A Nationwide Observational Cohort Study

OBJECTIVES

A high body mass index (BMI) is associated with an unfavorable disease course in COVID-19, but not among those who require admission to the ICU. This has not been examined across different age groups. We examined whether age modifies the association between BMI and mortality among critically ill COVID-19 patients.

DESIGN

An observational cohort study.

SETTING

A nationwide registry analysis of critically ill patients with COVID-19 registered in the National Intensive Care Evaluation registry.

PATIENTS

We included 15,701 critically ill patients with COVID-19 (10,768 males [68.6%] with median [interquartile range] age 64 yr [55-71 yr]), of whom 1,402 (8.9%) patients were less than 45 years.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

In the total sample and after adjustment for age, gender, Acute Physiology and Chronic Health Evaluation IV, mechanical ventilation, and use of vasoactive drugs, we found that a BMI greater than or equal to 30 kg/m 2 does not affect hospital mortality (adjusted odds ratio [OR adj ] = 0.98; 95% CI, 0.90-1.06; p = 0.62). For patients less than 45 years old, but not for those greater than or equal to 45 years old, a BMI greater than or equal to 30 kg/m 2 was associated with a lower hospital mortality (OR adj = 0.59; 95% CI, 0.36-0.96; p = 0.03).

CONCLUSIONS

A higher BMI may be favorably associated with a lower mortality among those less than 45 years old. This is in line with the so-called "obesity paradox" that was established for other groups of critically ill patients in broad age ranges. Further research is needed to understand this favorable association in young critically ill patients with COVID-19.

Evaluation of Patient-Friendly Diagnosis Clarifications in a Hospital Patient Portal

Background Medical data can be difficult to comprehend for patients, but only a limited number of patient-friendly terms and definitions are available to clarify medical concepts. Therefore, we developed an algorithm that generalizes diagnoses to more general concepts that do have patient-friendly terms and definitions in SNOMED CT. We implemented the generalizations, and diagnosis clarifications with synonyms and definitions that were already available, in the problem list of a hospital patient portal. Objective We aimed to assess the extent to which the clarifications cover the diagnoses in the problem list, the extent to which clarifications are used and appreciated by patient portal users, and to explore differences in viewing problems and clarifications between subgroups of users and diagnoses. Methods We measured the coverage of diagnoses by clarifications, usage of the problem list and the clarifications, and user, patient and diagnosis characteristics with aggregated, routinely available electronic health record and log file data. Additionally, patient portal users provided quantitative and qualitative feedback about the clarification quality. Results Of all patient portal users that viewed diagnoses on their problem list (n = 2,660), 89% had one or more diagnoses with clarifications. 55% of patient portal users viewed the clarifications. Users that rated the clarifications (n = 108) considered the clarifications to be of good quality on average, with a median rating per patient of 6 (interquartile range: 4 - 7; from 1 very bad to 7 very good). Users commented that they found clarifications to be clear and recognized the clarifications from their own experience, but sometimes also found the clarifications incomplete or disagreed with the diagnosis itself. Conclusions This study shows that the clarifications are used and appreciated by patient portal users. Further research and development will be dedicated to the maintenance and further quality improvement of the clarifications.

Drug-related causes attributed to acute kidney injury and their documentation in intensive care patients

PURPOSE

To investigate drug-related causes attributed to acute kidney injury (DAKI) and their documentation in patients admitted to the Intensive Care Unit (ICU).

METHODS

This study was conducted in an academic hospital in the Netherlands by reusing electronic health record (EHR) data of adult ICU admissions between November 2015 to January 2020. First, ICU admissions with acute kidney injury (AKI) stage 2 or 3 were identified. Subsequently, three modes of DAKI documentation in EHR were examined: diagnosis codes (structured data), allergy module (semi-structured data), and clinical notes (unstructured data).

RESULTS

n total 8124 ICU admissions were included, with 542 (6.7%) ICU admissions experiencing AKI stage 2 or 3. The ICU physicians deemed 102 of these AKI cases (18.8%) to be drug-related. These DAKI cases were all documented in the clinical notes (100%), one in allergy module (1%) and none via diagnosis codes. The clinical notes required the highest time investment to analyze.

CONCLUSIONS

Drug-related causes comprise a substantial part of AKI in the ICU patients. However, current unstructured DAKI documentation practice via clinical notes hampers our ability to gain better insights about DAKI occurrence. Therefore, both automating DAKI identification from the clinical notes and increasing structured DAKI documentation should be encouraged.

Electronic health record-based prediction models for in-hospital adverse drug event diagnosis or prognosis: a systematic review

OBJECTIVE

We conducted a systematic review to characterize and critically appraise developed prediction models based on structured electronic health record (EHR) data for adverse drug event (ADE) diagnosis and prognosis in adult hospitalized patients.

MATERIALS AND METHODS

We searched the Embase and Medline databases (from January 1, 1999, to July 4, 2022) for articles utilizing structured EHR data to develop ADE prediction models for adult inpatients. For our systematic evidence synthesis and critical appraisal, we applied the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies (CHARMS).

RESULTS

Twenty-five articles were included. Studies often did not report crucial information such as patient characteristics or the method for handling missing data. In addition, studies frequently applied inappropriate methods, such as univariable screening for predictor selection. Furthermore, the majority of the studies utilized ADE labels that only described an adverse symptom while not assessing causality or utilizing a causal model. None of the models were externally validated.

CONCLUSIONS

Several challenges should be addressed before the models can be widely implemented, including the adherence to reporting standards and the adoption of best practice methods for model development and validation. In addition, we propose a reorientation of the ADE prediction modeling domain to include causality as a fundamental challenge that needs to be addressed in future studies, either through acquiring ADE labels via formal causality assessments or the usage of adverse event labels in combination with causal prediction modeling.

Increased mortality in ICU patients ≥70 years old with COVID-19 compared to patients with other pneumonias

BACKGROUND

Patients over 70 years old represent a substantial proportion of the COVID-19 ICU population and their mortality rates are high. The aim of this study is to describe the outcomes of patients ≥70 years old admitted to Dutch ICUs with COVID-19, compared to patients ≥70 years old admitted to the ICU for bacterial and other viral pneumonias, with adjustments for age, comorbidities, severity of illness, and ICU occupancy rate.

METHODS

Retrospective cohort study including patients ≥70 years old admitted to Dutch ICUs, comparing patients admitted with COVID-19 from March 1st 2020 to January 1st 2022 with patients ≥70 years old admitted because of a bacterial and other viral pneumonia, both divided in a historical (i.e., January 1st 2017 to January 1st 2020) and current cohort (i.e., March 1st 2020 to January 1st 2022). Primary outcome is hospital mortality.

RESULTS

11,525 unique patients ≥70 years old admitted to Dutch ICUs were included; 5094 with COVID-19, 5334 with a bacterial pneumonia, and 1312 with another viral pneumonia. ICU-mortality and in-hospital mortality rates of the patients ≥70 years old admitted with COVID-19 were 39.7% and 47.6% respectively. ICU- and hospital mortality rates of the patients who were admitted in the same or in an historical time period with a bacterial pneumonia or other viral pneumonias were considerably lower (19.5% and 28.6% for patients with a bacterial pneumonia in the historical cohort and 19.1% and 28.8% in the same period, for the patients with other viral pneumonias 20.7% and 28.9%, and 22.7% and 31.8% respectively, all p < 0.001). Differences persisted after correction for several clinical characteristics and ICU occupancy rate.

CONCLUSIONS

In ICU-patients ≥70 years old, COVID-19 is more severe compared to bacterial or viral pneumonia.

Real-world Evidence of the Effects of Novel Treatments for COVID-19 on Mortality: A Nationwide Comparative Cohort Study of Hospitalized Patients in the First, Second, Third, and Fourth Waves in the Netherlands

BACKGROUND

Large clinical trials on drugs for hospitalized coronavirus disease 2019 (COVID-19) patients have shown significant effects on mortality. There may be a discrepancy with the observed real-world effect. We describe the clinical characteristics and outcomes of hospitalized COVID-19 patients in the Netherlands during 4 pandemic waves and analyze the association of the newly introduced treatments with mortality, intensive care unit (ICU) admission, and discharge alive.

METHODS

We conducted a nationwide retrospective analysis of hospitalized COVID-19 patients between February 27, 2020, and December 31, 2021. Patients were categorized into waves and into treatment groups (hydroxychloroquine, remdesivir, neutralizing severe acute respiratory syndrome coronavirus 2 monoclonal antibodies, corticosteroids, and interleukin [IL]-6 antagonists). Four types of Cox regression analyses were used: unadjusted, adjusted, propensity matched, and propensity weighted.

RESULTS

Among 5643 patients from 11 hospitals, we observed a changing epidemiology during 4 pandemic waves, with a decrease in median age (67-64 years; P < .001), in in-hospital mortality on the ward (21%-15%; P < .001), and a trend in the ICU (24%-16%; P = .148). In ward patients, hydroxychloroquine was associated with increased mortality (1.54; 95% CI, 1.22-1.96), and remdesivir was associated with a higher rate of discharge alive within 29 days (1.16; 95% CI, 1.03-1.31). Corticosteroids were associated with a decrease in mortality (0.82; 95% CI, 0.69-0.96); the results of IL-6 antagonists were inconclusive. In patients directly admitted to the ICU, hydroxychloroquine, corticosteroids, and IL-6 antagonists were not associated with decreased mortality.

CONCLUSIONS

Both remdesivir and corticosteroids were associated with better outcomes in ward patients with COVID-19. Continuous evaluation of real-world treatment effects is needed.

Assess and validate predictive performance of models for in-hospital mortality in COVID-19 patients: A retrospective cohort study in the Netherlands comparing the value of registry data with high-granular electronic health records

PURPOSE

To assess, validate and compare the predictive performance of models for in-hospital mortality of COVID-19 patients admitted to the intensive care unit (ICU) over two different waves of infections. Our models were built with high-granular Electronic Health Records (EHR) data versus less-granular registry data.

METHODS

Observational study of all COVID-19 patients admitted to 19 Dutch ICUs participating in both the national quality registry National Intensive Care Evaluation (NICE) and the EHR-based Dutch Data Warehouse (hereafter EHR). Multiple models were developed on data from the first 24 h of ICU admissions from February to June 2020 (first COVID-19 wave) and validated on prospective patients admitted to the same ICUs between July and December 2020 (second COVID-19 wave). We assessed model discrimination, calibration, and the degree of relatedness between development and validation population. Coefficients were used to identify relevant risk factors.

RESULTS

A total of 1533 patients from the EHR and 1563 from the registry were included. With high granular EHR data, the average AUROC was 0.69 (standard deviation of 0.05) for the internal validation, and the AUROC was 0.75 for the temporal validation. The registry model achieved an average AUROC of 0.76 (standard deviation of 0.05) in the internal validation and 0.77 in the temporal validation. In the EHR data, age, and respiratory-system related variables were the most important risk factors identified. In the NICE registry data, age and chronic respiratory insufficiency were the most important risk factors.

CONCLUSION

In our study, prognostic models built on less-granular but readily-available registry data had similar performance to models built on high-granular EHR data and showed similar transportability to a prospective COVID-19 population. Future research is needed to verify whether this finding can be confirmed for upcoming waves.

Comparison of patient characteristics and long‐term mortality between transferred and non‐transferred COVID‐19 patients in Dutch Intensive Care Units; A national cohort study

Background

COVID‐19 patients were often transferred to other intensive care units (ICUs) to prevent that ICUs would reach their maximum capacity. However, transferring ICU patients is not free of risk. We aim to compare the characteristics and outcomes of transferred versus non‐transferred COVID‐19 ICU patients in the Netherlands.

Methods

We included adult COVID‐19 patients admitted to Dutch ICUs between March 1, 2020 and July 1, 2021. We compared the patient characteristics and outcomes of non‐transferred and transferred patients and used a Directed Acyclic Graph to identify potential confounders in the relationship between transfer and mortality. We used these confounders in a Cox regression model with left truncation at the day of transfer to analyze the effect of transfers on mortality during the 180 days after ICU admission.

Results

We included 10,209 patients: 7395 non‐transferred and 2814 (27.6%) transferred patients. In both groups, the median age was 64 years. Transferred patients were mostly ventilated at ICU admission (83.7% vs. 56.2%) and included a larger proportion of low‐risk patients (70.3% vs. 66.5% with mortality risk <30%). After adjusting for age, APACHE IV mortality probability, BMI, mechanical ventilation, and vasoactive medication use, the hazard of mortality during the first 180 days was similar for transferred patients compared to non‐transferred patients (HR [95% CI] = 0.99 [0.91–1.08]).

Conclusions

Transferred COVID‐19 patients are more often mechanically ventilated and are less severely ill compared to non‐transferred patients. Furthermore, transferring critically ill COVID‐19 patients in the Netherlands is not associated with mortality during the first 180 days after ICU admission.

Clinical sepsis phenotypes in critically ill COVID-19 patients

Background

A greater understanding of disease heterogeneity may facilitate precision medicine for coronavirus disease 2019 (COVID-19). Previous work identified four distinct clinical phenotypes associated with outcome and treatment responses in non-COVID-19 sepsis patients, but it is unknown if and how these phenotypes are recapitulated in COVID-19 sepsis patients.

Methods

We applied the four non-COVID-19 sepsis phenotypes to a total of 52,274 critically ill patients, comprising two cohorts of COVID-19 sepsis patients (admitted before and after the introduction of dexamethasone as standard treatment) and three non-COVID-19 sepsis cohorts (non-COVID-19 viral pneumonia sepsis, bacterial pneumonia sepsis, and bacterial sepsis of non-pulmonary origin). Differences in proportions of phenotypes and their associated mortality were determined across these cohorts.

Results

Phenotype distribution was highly similar between COVID-19 and non-COVID-19 viral pneumonia sepsis cohorts, whereas the proportion of patients with the δ-phenotype was greater in both bacterial sepsis cohorts compared to the viral sepsis cohorts. The introduction of dexamethasone treatment was associated with an increased proportion of patients with the δ-phenotype (6% vs. 11% in the pre- and post-dexamethasone COVID-19 cohorts, respectively, p < 0.001). Across the cohorts, the α-phenotype was associated with the most favorable outcome, while the δ-phenotype was associated with the highest mortality. Survival of the δ-phenotype was markedly higher following the introduction of dexamethasone (60% vs 41%, p < 0.001), whereas no relevant differences in survival were observed for the other phenotypes among COVID-19 patients.

Conclusions

Classification of critically ill COVID-19 patients into clinical phenotypes may aid prognostication, prediction of treatment efficacy, and facilitation of personalized medicine.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13054-022-04118-6.

Performance assessment of ontology matching systems for FAIR data

Background

Ontology matching should contribute to the interoperability aspect of FAIR data (Findable, Accessible, Interoperable, and Reusable). Multiple data sources can use different ontologies for annotating their data and, thus, creating the need for dynamic ontology matching services. In this experimental study, we assessed the performance of ontology matching systems in the context of a real-life application from the rare disease domain. Additionally, we present a method for analyzing top-level classes to improve precision.

Results

We included three ontologies (NCIt, SNOMED CT, ORDO) and three matching systems (AgreementMakerLight 2.0, FCA-Map, LogMap 2.0). We evaluated the performance of the matching systems against reference alignments from BioPortal and the Unified Medical Language System Metathesaurus (UMLS). Then, we analyzed the top-level ancestors of matched classes, to detect incorrect mappings without consulting a reference alignment. To detect such incorrect mappings, we manually matched semantically equivalent top-level classes of ontology pairs. AgreementMakerLight 2.0, FCA-Map, and LogMap 2.0 had F1-scores of 0.55, 0.46, 0.55 for BioPortal and 0.66, 0.53, 0.58 for the UMLS respectively. Using vote-based consensus alignments increased performance across the board. Evaluation with manually created top-level hierarchy mappings revealed that on average 90% of the mappings’ classes belonged to top-level classes that matched.

Conclusions

Our findings show that the included ontology matching systems automatically produced mappings that were modestly accurate according to our evaluation. The hierarchical analysis of mappings seems promising when no reference alignments are available. All in all, the systems show potential to be implemented as part of an ontology matching service for querying FAIR data. Future research should focus on developing methods for the evaluation of mappings used in such mapping services, leading to their implementation in a FAIR data ecosystem.

Supplementary Information

The online version contains supplementary material available at (10.1186/s13326-022-00273-5).

Inaccurate recording of routinely collected data items influences identification of COVID-19 patients

Background

During the Coronavirus disease 2019 (COVID-19) pandemic it became apparent that it is difficult to extract standardized Electronic Health Record (EHR) data for secondary purposes like public health decision-making. Accurate recording of, for example, standardized diagnosis codes and test results is required to identify all COVID-19 patients. This study aimed to investigate if specific combinations of routinely collected data items for COVID-19 can be used to identify an accurate set of intensive care unit (ICU)-admitted COVID-19 patients.

Methods

The following routinely collected EHR data items to identify COVID-19 patients were evaluated: positive reverse transcription polymerase chain reaction (RT-PCR) test results; problem list codes for COVID-19 registered by healthcare professionals and COVID-19 infection labels. COVID-19 codes registered by clinical coders retrospectively after discharge were also evaluated. A gold standard dataset was created by evaluating two datasets of suspected and confirmed COVID-19-patients admitted to the ICU at a Dutch university hospital between February 2020 and December 2020, of which one set was manually maintained by intensivists and one set was extracted from the EHR by a research data management department. Patients were labeled ‘COVID-19′ if their EHR record showed diagnosing COVID-19 during or right before an ICU-admission. Patients were labeled ‘non-COVID-19′ if the record indicated no COVID-19, exclusion or only suspicion during or right before an ICU-admission or if COVID-19 was diagnosed and cured during non-ICU episodes of the hospitalization in which an ICU-admission took place. Performance was determined for 37 queries including real-time and retrospective data items. We used the F₁ score, which is the harmonic mean between precision and recall. The gold standard dataset was split into one subset including admissions between February and April and one subset including admissions between May and December to determine accuracy differences.

Results

The total dataset consisted of 402 patients: 196 ‘COVID-19′ and 206 ‘non-COVID-19′ patients. F₁ scores of search queries including EHR data items that can be extracted real-time ranged between 0.68 and 0.97 and for search queries including the data item that was retrospectively registered by clinical coders F₁ scores ranged between 0.73 and 0.99. F₁ scores showed no clear pattern in variability between the two time periods.

Conclusions

Our study showed that one cannot rely on individual routinely collected data items such as coded COVID-19 on problem lists to identify all COVID-19 patients. If information is not required real-time, medical coding from clinical coders is most reliable. Researchers should be transparent about their methods used to extract data. To maximize the ability to completely identify all COVID-19 cases alerts for inconsistent data and policies for standardized data capture could enable reliable data reuse.

The effect of treatment and clinical course during Emergency Department stay on severity scoring and predicted mortality risk in Intensive Care patients

Background

Treatment and the clinical course during Emergency Department (ED) stay before Intensive Care Unit (ICU) admission may affect predicted mortality risk calculated by the Acute Physiology and Chronic Health Evaluation (APACHE)-IV, causing lead-time bias. As a result, comparing standardized mortality ratios (SMRs) among hospitals may be difficult if they differ in the location where initial stabilization takes place. The aim of this study was to assess to what extent predicted mortality risk would be affected if the APACHE-IV score was recalculated with the initial physiological variables from the ED. Secondly, to evaluate whether ED Length of Stay (LOS) was associated with a change (delta) in these APACHE-IV scores.

Methods

An observational multicenter cohort study including ICU patients admitted from the ED. Data from two Dutch quality registries were linked: the Netherlands Emergency department Evaluation Database (NEED) and the National Intensive Care Evaluation (NICE) registry. The ICU APACHE-IV, predicted mortality, and SMR based on data of the first 24 h of ICU admission were compared with an ED APACHE-IV model, using the most deviating physiological variables from the ED or ICU.

Results

A total of 1398 patients were included. The predicted mortality from the ICU APACHE-IV (median 0.10; IQR 0.03–0.30) was significantly lower compared to the ED APACHE-IV model (median 0.13; 0.04–0.36; p < 0.01). The SMR changed from 0.63 (95%CI 0.54–0.72) to 0.55 (95%CI 0.47–0.63) based on ED APACHE-IV. Predicted mortality risk changed more than 5% in 321 (23.2%) patients by using the ED APACHE-IV. ED LOS > 3.9 h was associated with a slight increase in delta APACHE-IV of 1.6 (95% CI 0.4–2.8) compared to ED LOS < 1.7 h.

Conclusion

Predicted mortality risks and SMRs calculated by the APACHE IV scores are not directly comparable in patients admitted from the ED if hospitals differ in their policy to stabilize patients in the ED before ICU admission. Future research should focus on developing models to adjust for these differences.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13054-022-03986-2.

Heterogeneity in the identification of potential drug-drug interactions in the intensive care unit: A systematic review, critical appraisal, and reporting recommendations

Patients admitted to the intensive care unit (ICU) are frequently exposed to potential drug‐drug interactions (pDDIs). However, reported frequencies of pDDIs in the ICU vary widely between studies. This can be partly explained by significant variation in their methodological approach. Insight into methodological choices affecting pDDI frequency would allow for improved comparison and synthesis of reported pDDI frequencies. This study aimed to evaluate the association between methodological choices and pDDI frequency and formulate reporting recommendations for pDDI frequency studies in the ICU. The MEDLINE database was searched to identify papers reporting pDDI frequency in ICU patients. For each paper, the pDDI frequency and methodological choices such as pDDI definition and pDDI knowledge base were extracted, and the risk of bias was assessed. Each paper was categorized as reporting a low, medium, or high pDDI frequency. We sought associations between methodological choices and pDDI frequency group. Based on this comparison, reporting recommendations were formulated. Analysis of methodological choices showed significant heterogeneity between studies, and 65% of the studies had a medium to high risk of bias. High risk of bias, small sample size, and use of drug prescriptions instead of administrations were related to a higher pDDI frequency. The findings of this review may support researchers in designing a reliable methodology assessing pDDI frequency in ICU patients. The reporting recommendations may contribute to standardization, comparison, and synthesis of pDDI frequency studies, ultimately improving knowledge about pDDIs in and outside the ICU setting.

Incorrect application of the KDIGO acute kidney injury staging criteria

Background

Recent research demonstrated substantial heterogeneity in the Kidney Disease: Improving Global Outcomes (KDIGO) acute kidney injury (AKI) diagnosis and staging criteria implementations in clinical research. Here we report an additional issue in the implementation of the criteria: the incorrect description and application of a stage 3 serum creatinine (SCr) criterion. Instead of an increase in SCr to or beyond 4.0 mg/dL, studies apparently interpreted this criterion as an increase in SCr by 4.0 mg/dL.

Methods

Using a sample of 8124 consecutive intensive care unit (ICU) admissions, we illustrate the implications of such incorrect application. The AKI stage distributions associated with the correct and incorrect stage 3 SCr criterion implementations were compared, both with and without the stage 3 renal replacement therapy (RRT) criterion. In addition, we compared chronic kidney disease presence, ICU mortality rates and hospital mortality rates associated with each of the AKI stages and the misclassified cases.

Results

Where incorrect implementation of the SCr stage 3 criterion showed a stage 3 AKI rate of 29%, correct implementation revealed a rate of 34%, mainly due to shifts from stage 1 to stage 3. Without the stage 3 RRT criterion, the stage 3 AKI rates were 9% and 19% after incorrect and correct implementation, respectively. The ICU and hospital mortality rates in cases misclassified as stage 1 or 2 were similar to those in cases correctly classified as stage 1 instead of stage 3.

Conclusions

While incorrect implementation of the SCr stage 3 criterion has significant consequences for AKI severity epidemiology, consequences for clinical decision making may be less severe. We urge researchers and clinicians to verify their implementation of the AKI staging criteria.

One-year Mortality of Cancer Patients with an Unplanned ICU Admission: A Cohort Analysis Between 2008 and 2017 in the Netherlands

Introduction: A decrease in short-term mortality of critically ill cancer patients with an unplanned intensive care unit (ICU) admission has been described. Few studies describe a change over time of 1-year mortality. Therefore, we examined the 1-year mortality of cancer patients (hematological or solid) with an unplanned ICU admission and we described whether the mortality changed over time. Methods: We used the National Intensive Care Evaluation (NICE) registry and extracted all patients with an unplanned ICU admission in the Netherlands between 2008 and 2017. The primary outcome was 1-year mortality, analyzed with a mixed-effects Cox proportional hazard regression. We compared the 1-year mortality of cancer patients to that of patients without cancer. Furthermore, we examined changes in mortality over the study period. Results: We included 470,305 patients: 10,401 with hematological cancer, 35,920 with solid cancer, and 423,984 without cancer. The 1-year mortality rates were 60.1%, 46.2%, and 28.3% respectively (P< .01). Approximately 30% of the cancer patients surviving their hospital admission died within 1 year, this was 12% in patients without cancer. In hematological patients, 1-year mortality decreased between 2008 and 2011, after which it stabilized. In solid cancer patients, inspection showed neither an increasing nor decreasing trend over the inclusion period. For patients without cancer, 1-year mortality decreased between 2008 and 2013, after which it stabilized. A clear decrease in hospital mortality was seen within all three groups. Conclusion: The 1-year mortality of cancer patients with an unplanned ICU admission (hematological and solid) was higher than that of patients without cancer. About one-third of the cancer patients surviving their hospital admission died within 1 year after ICU admission. We found a decrease in 1-year mortality until 2011 in hematology patients and no decrease in solid cancer patients. Our results suggest that for many cancer patients, an unplanned ICU admission is still a way to recover from critical illness, and it does not necessarily lead to success in long-term survival. The underlying type of malignancy is an important factor for long-term outcomes in patients recovering from critical illness.

Higher 1-year mortality in women admitted to intensive care units after cardiac arrest: A nationwide overview from the Netherlands between 2010 and 2018

PURPOSE

We study sex differences in 1-year mortality of out-of-hospital cardiac arrest (OHCA) and in-hospital cardiac arrest (IHCA) patients admitted to the intensive care unit (ICU).

DATA

A retrospective cohort analysis of OHCA and IHCA patients registered in the NICE registry in the Netherlands. The primary and secondary outcomes were 1-year and hospital mortality, respectively.

RESULTS

We included 19,440 OHCA patients (5977 women, 30.7%) and 13,461 IHCA patients (4889 women, 36.3%). For OHCA, 1-year mortality was 63.9% in women and 52.6% in men (Hazard Ratio [HR] 1.28, 95% Confidence Interval [95% CI] 1.23-1.34). For IHCA, 1-year mortality was 60.0% in women and 57.0% in men (HR 1.09, 95% CI 1.04-1.14). In OHCA, hospital mortality was 57.4% in women and 46.5% in men (Odds Ratio [OR] 1.42, 95% CI 1.33-1.52). In IHCA, hospital mortality was 52.0% in women and 48.2% in men (OR 1.11, 95% CI 1.03-1.20).

CONCLUSION

Women admitted to the ICU after cardiac arrest have a higher mortality rate than men. After left-truncation, we found that this sex difference persisted for OHCA. For IHCA we found that the effect of sex was mainly present in the initial phase after the cardiac arrest.

Contextual property detection in Dutch diagnosis descriptions for uncertainty, laterality and temporality

Background

Accurate, coded problem lists are valuable for data reuse, including clinical decision support and research. However, healthcare providers frequently modify coded diagnoses by including or removing common contextual properties in free-text diagnosis descriptions: uncertainty (suspected glaucoma), laterality (left glaucoma) and temporality (glaucoma 2002). These contextual properties could cause a difference in meaning between underlying diagnosis codes and modified descriptions, inhibiting data reuse. We therefore aimed to develop and evaluate an algorithm to identify these contextual properties.

Methods

A rule-based algorithm called UnLaTem (Uncertainty, Laterality, Temporality) was developed using a single-center dataset, including 288,935 diagnosis descriptions, of which 73,280 (25.4%) were modified by healthcare providers. Internal validation of the algorithm was conducted with an independent sample of 980 unique records. A second validation of the algorithm was conducted with 996 records from a Dutch multicenter dataset including 175,210 modified descriptions of five hospitals. Two researchers independently annotated the two validation samples. Performance of the algorithm was determined using means of the recall and precision of the validation samples. The algorithm was applied to the multicenter dataset to determine the actual prevalence of the contextual properties within the modified descriptions per specialty.

Results

For the single-center dataset recall (and precision) for removal of uncertainty, uncertainty, laterality and temporality respectively were 100 (60.0), 99.1 (89.9), 100 (97.3) and 97.6 (97.6). For the multicenter dataset for removal of uncertainty, uncertainty, laterality and temporality it was 57.1 (88.9), 86.3 (88.9), 99.7 (93.5) and 96.8 (90.1). Within the modified descriptions of the multicenter dataset, 1.3% contained removal of uncertainty, 9.9% uncertainty, 31.4% laterality and 9.8% temporality.

Conclusions

We successfully developed a rule-based algorithm named UnLaTem to identify contextual properties in Dutch modified diagnosis descriptions. UnLaTem could be extended with more trigger terms, new rules and the recognition of term order to increase the performance even further. The algorithm’s rules are available as additional file 2. Implementing UnLaTem in Dutch hospital systems can improve precision of information retrieval and extraction from diagnosis descriptions, which can be used for data reuse purposes such as decision support and research.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01477-y.

Number of intensivists per bed is associated with efficiency of Dutch intensive care units

PURPOSE

To measure efficiency in Intensive Care Units (ICUs) and to determine which organizational factors are associated with ICU efficiency, taking confounding factors into account.

MATERIALS AND METHODS

We used data of all consecutive admissions to Dutch ICUs between January 1, 2016 and January 1, 2019 and recorded ICU organizational factors. We calculated efficiency for each ICU by averaging the Standardized Mortality Ratio (SMR) and Standardized Resource Use (SRU) and examined the relationship between various organizational factors and ICU efficiency. We thereby compared the results of linear regression models before and after covariate adjustment using propensity scores.

RESULTS

We included 164,399 admissions from 83 ICUs. ICU efficiency ranged from 0.51-1.42 (average 0.99, 0.15 SD). The unadjusted model as well as the propensity score adjusted model showed a significant association between the ratio of employed intensivists per ICU bed and ICU efficiency. Other organizational factors had no statistically significant association with ICU efficiency after adjustment.

CONCLUSIONS

We found marked variability in efficiency in Dutch ICUs. After applying covariate adjustment using propensity scores, we identified one organizational factor, ratio intensivists per bed, having an association with ICU efficiency.

Evaluation of lexical clarification by patients reading their clinical notes: a quasi-experimental interview study

Background

Patients benefit from access to their medical records. However, clinical notes and letters are often difficult to comprehend for most lay people. Therefore, functionality was implemented in the patient portal of a Dutch university medical centre (UMC) to clarify medical terms in free-text data. The clarifications consisted of synonyms and definitions from a Dutch medical terminology system. We aimed to evaluate to what extent these lexical clarifications match the information needs of the patients. Secondarily, we evaluated how the clarifications and the functionality could be improved.

Methods

We invited participants from the patient panel of the UMC to read their own clinical notes. They marked terms they found difficult and rated the ease of these terms. After the functionality was activated, participants rated the clarifications provided by the functionality, and the functionality itself regarding ease and usefulness. Ratings were on a scale from 0 (very difficult) to 100 (very easy). We calculated the median number of terms not understood per participant, the number of terms with a clarification, the overlap between these numbers (coverage), and the precision and recall.

Results

We included 15 participants from the patient panel. They marked a median of 21 (IQR 19.5–31) terms as difficult in their text files, while only a median of 2 (IQR 1–4) of these terms were clarified by the functionality. The median precision was 6.5% (IQR 2.3–14.25%) and the median recall 8.3% (IQR 4.7–13.5%) per participant. However, participants rated the functionality with median ease of 98 (IQR 93.5–99) and a median usefulness of 79 (IQR 52.5–97). Participants found that many easy terms were unnecessarily clarified, that some clarifications were difficult, and that some clarifications contained mistakes.

Conclusions

Patients found the functionality easy to use and useful. However, in its current form it only helped patients to understand few terms they did not understand, patients found some clarifications to be difficult, and some to be incorrect. This shows that lexical clarification is feasible even when limited terms are available, but needs further development to fully use its potential.

Clinically relevant potential drug-drug interactions in intensive care patients: A large retrospective observational multicenter study

PURPOSE

Potential drug-drug interactions (pDDIs) may harm patients admitted to the Intensive Care Unit (ICU). Due to the patient's critical condition and continuous monitoring on the ICU, not all pDDIs are clinically relevant. Clinical decision support systems (CDSSs) warning for irrelevant pDDIs could result in alert fatigue and overlooking important signals. Therefore, our aim was to describe the frequency of clinically relevant pDDIs (crpDDIs) to enable tailoring of CDSSs to the ICU setting.

MATERIALS & METHODS

In this multicenter retrospective observational study, we used medication administration data to identify pDDIs in ICU admissions from 13 ICUs. Clinical relevance was based on a Delphi study in which intensivists and hospital pharmacists assessed the clinical relevance of pDDIs for the ICU setting.

RESULTS

The mean number of pDDIs per 1000 medication administrations was 70.1, dropping to 31.0 when considering only crpDDIs. Of 103,871 ICU patients, 38% was exposed to a crpDDI. The most frequently occurring crpDDIs involve QT-prolonging agents, digoxin, or NSAIDs.

CONCLUSIONS

Considering clinical relevance of pDDIs in the ICU setting is important, as only half of the detected pDDIs were crpDDIs. Therefore, tailoring CDSSs to the ICU may reduce alert fatigue and improve medication safety in ICU patients.

Nurse Operation Workload (NOW), a new nursing workload model for intensive care units based on time measurements: An observational study

BACKGROUND

Several instruments have been developed to measure nursing workload. The commonly used Nursing Activities Score (NAS) and Therapeutic Intervention Scoring System (TISS) are applied to all types of ICU patients. Former research showed that NAS explained 59 to 81% of actual nursing time, whereas the Therapeutic Intervention Scoring System (TISS) described only 43% of the actual nursing time. In both models the development was not based on time measurements.

OBJECTIVES

The aim of this study was to develop a time-based model which can assess patient related nursing workload more accurately and to evaluate whether patient characteristics influence nursing time and therefore should be included in the model.

DESIGN

Observational study design.

SETTING

All 82 Dutch ICUs participate in the National Intensive Care Evaluation (NICE) quality registry. Fifteen of these ICUs are participating in the newly implemented voluntary nursing capacity module. Seven of these ICUs voluntarily participated in this study.

PARTICIPANTS

The patient(s) that were under the responsibility of a chosen nurse were followed by the observer during the entire shift.

METHODS

Time spent per nursing activity per patient was measured in different shifts in seven Dutch ICUs. Nursing activities were measured using an in-house developed web application. Three different models of varying complexity (1. nursing activities only; 2. nursing activities and case-mix correction; 3. complex model with case-mix correction per nursing activity) were developed to explain the total amount of nursing time per patient. The performance of the three models was assessed in 1000 bootstrap samples using the squared Pearson correlation coefficient (R²), Root Mean Squared Prediction Error (RMSPE), Mean Absolute Prediction Error (MAPE), and prediction bias.

RESULTS

In total 287 unique patients have been observed in 371 shifts. Model one's Pearson's R was 0.89 (95%CI 0.86-0.92), model two with case-mix correction 0.90 (95%CI 0.88-0.93), and the third complex model 0.64 (95%CI 0.56-0.72) compared with the actual patient related nursing workload.

CONCLUSION

The newly developed Nurse Operation Workload (NOW) model outperforms existing models in measuring nursing workload, while it includes a lower number of activities and therewith lowers the registration burden. Case-mix correction does not further improve the performance of this model. The patient related nursing workload measured by the NOW gives insight in the actual nursing time needed by patients and can therefore be used to evaluate the average workload per patient per nurse.

Outcomes of Intensive Care Patients Older Than 90 Years: An 11-Year National Observational Study

BACKGROUND/OBJECTIVES

Many intensive care unit (ICU) physicians are reluctant to admit patients aged 90 years and older, although evidence to support these decisions is scarce. Although the body of evidence on outcomes of patients aged 80 years and older is growing, it does not include patients aged 90 years and older. The aim of this study was to compare the short- and long-term mortality of ICU patients aged 90 years and older in the Netherlands with ICU patients aged 80 to 90 years, that is, octogenarians.

DESIGN

Multicenter national cohort study over an 11-year period (2008-2018), using data of the National Intensive Care Evaluation (NICE) registry and the Dutch insurance claims registry.

SETTING

All 82 ICUs in the Netherlands.

PARTICIPANTS

All patients aged 80 years and older at the time of ICU admission.

MEASUREMENTS

A total of 104,754 patients aged 80 years and older, of whom 9,495 (9%) were 90 years and older, were admitted to Dutch ICUs during the study period.

RESULTS

ICU mortality of the patients aged 90 years and older was lower (13.8% vs 16.1%; P < .001) and hospital mortality was similar (26.1% vs 25.7%; P = .41) compared with octogenarians. After 3 months, mortality was higher for the patients aged 90 years and older (43.1% vs 33.7%; P < .001) and after 1-year mortality was 55.0% vs 42.7%; P < .001.

CONCLUSION

In the Netherlands, mortality rates of patients aged 90 years and older admitted to the ICU are not as disappointing as often assumed. They have a lower ICU mortality and a similar hospital mortality compared with octogenarians. Nevertheless, their longer term mortality is higher compared with octogenarians. However, almost 3 of 4 patients leave the hospital alive, and almost half of the patients aged 90 years and older are still alive 1 year after their ICU admission. J Am Geriatr Soc 68:1842-1846, 2020.

The association between influenza infections in primary care and intensive care admissions for severe acute respiratory infection (SARI): A modelling approach

Background

The burden of severe influenza virus infections is poorly known, for which surveillance of severe acute respiratory infection (SARI) is encouraged. Hospitalized SARI patients are however not always tested for influenza virus infection. Thus, to estimate the impact of influenza circulation we studied how influenza in primary care relates to intensive care unit (ICU) admissions using a modelling approach.

Methods

We used time‐series regression modelling to estimate a) the number of SARI admissions to ICU associated with medically attended influenza infections in primary care; b) how this varies by season; and c) the time lag between SARI and influenza time series. We analysed weekly adult ICU admissions (registry data) and adult influenza incidence (primary care surveillance data) from July 2007 through June 2016.

Results

Depending on the year, 0% to 12% of annual SARI admissions were associated with influenza (0‐554 in absolute numbers; population rate: 0/10 000‐0.39/10 000 inhabitants), up to 27% during influenza epidemics. The average optimal fitting lag was +1 week (SARI trend preceding influenza by 1 week), varying between seasons (−1 to +4) with most seasons showing positive lags.

Conclusion

Up to 12% of yearly SARI admissions to adult ICU are associated with influenza, but with large year‐to‐year variation and higher during influenza epidemics. In most years, SARI increases earlier than medically attended influenza infections in the general population. SARI surveillance could thus complement influenza‐like illness surveillance by providing an indication of the season‐specific burden of severe influenza infections and potential early warning of influenza activity and severity.

Factors Influencing Problem List Use in Electronic Health Records-Application of the Unified Theory of Acceptance and Use of Technology

BACKGROUND

Problem-oriented electronic health record (EHR) systems can help physicians to track a patient's status and progress, and organize clinical documentation, which could help improving quality of clinical data and enable data reuse. The problem list is central in a problem-oriented medical record. However, current problem lists remain incomplete because of the lack of end-user training and inaccurate content of underlying terminologies. This leads to modifications of diagnosis code descriptions and use of free-text notes, limiting reuse of data.

OBJECTIVES

We aimed to investigate factors that influence acceptance and actual use of the problem list, and used these to propose recommendations, to increase the value of problem lists for (re)use.

METHODS

Semistructured interviews were conducted with physicians, heads of medical departments, and data quality experts, who were invited through snowball sampling. The interviews were transcribed and coded. Comments were fitted in constructs of the validated framework unified theory of acceptance user technology (UTAUT), and were discussed in terms of facilitators and barriers.

RESULTS

In total, 24 interviews were conducted. We found large variability in attitudes toward problem list use. Barriers included uncertainty about the responsibility for maintaining the problem list and little perceived benefits. Facilitators included the (re)design of policies, improved (peer-to-peer) training to increase motivation, and positive peer feedback and monitoring. Motivation is best increased through sharing benefits relevant in the care process, such as providing overview, timely generation of discharge or referral letters, and reuse of data. Furthermore, content of the underlying terminology should be improved and the problem list should be better presented in the EHR system.

CONCLUSION

To let physicians accept and use the problem list, policies and guidelines should be redesigned, and prioritized by supervising staff. Additionally, peer-to-peer training on the benefits of using the problem list is needed.

Towards an Adoption Framework for Patient Access to Electronic Health Records: Systematic Literature Mapping Study

Background

Patient access to electronic health records (EHRs) is associated with increased patient engagement and health care quality outcomes. However, the adoption of patient portals and personal health records (PHRs) that facilitate this access is impeded by barriers. The Clinical Adoption Framework (CAF) has been developed to analyze EHR adoption, but this framework does not consider the patient as an end-user.

Objective

We aim to extend the scope of the CAF to patient access to EHRs, develop guidance documentation for the application of the CAF, and assess the interrater reliability.

Methods

We systematically reviewed existing systematic reviews on patients' access to EHRs and PHRs. Results of each review were mapped to one of the 43 CAF categories. Categories were iteratively adapted when needed. We measured the interrater reliability with Cohen’s unweighted kappa and statistics regarding the agreement among reviewers on mapping quotes of the reviews to different CAF categories.

Results

We further defined the framework’s inclusion and exclusion criteria for 33 of the 43 CAF categories and achieved a moderate agreement among the raters, which varied between categories.

Conclusions

In the reviews, categories about people, organization, system quality, system use, and the net benefits of system use were addressed more often than those about international and regional information and communication technology infrastructures, standards, politics, incentive programs, and social trends. Categories that were addressed less might have been underdefined in this study. The guidance documentation we developed can be applied to systematic literature reviews and implementation studies, patient and informal caregiver access to EHRs, and the adoption of PHRs.

Quality indicators for patients with traumatic brain injury in European intensive care units: a CENTER-TBI study

BACKGROUND

The aim of this study is to validate a previously published consensus-based quality indicator set for the management of patients with traumatic brain injury (TBI) at intensive care units (ICUs) in Europe and to study its potential for quality measurement and improvement.

METHODS

Our analysis was based on 2006 adult patients admitted to 54 ICUs between 2014 and 2018, enrolled in the CENTER-TBI study. Indicator scores were calculated as percentage adherence for structure and process indicators and as event rates or median scores for outcome indicators. Feasibility was quantified by the completeness of the variables. Discriminability was determined by the between-centre variation, estimated with a random effect regression model adjusted for case-mix severity and quantified by the median odds ratio (MOR). Statistical uncertainty of outcome indicators was determined by the median number of events per centre, using a cut-off of 10.

RESULTS

A total of 26/42 indicators could be calculated from the CENTER-TBI database. Most quality indicators proved feasible to obtain with more than 70% completeness. Sub-optimal adherence was found for most quality indicators, ranging from 26 to 93% and 20 to 99% for structure and process indicators. Significant (p < 0.001) between-centre variation was found in seven process and five outcome indicators with MORs ranging from 1.51 to 4.14. Statistical uncertainty of outcome indicators was generally high; five out of seven had less than 10 events per centre.

CONCLUSIONS

Overall, nine structures, five processes, but none of the outcome indicators showed potential for quality improvement purposes for TBI patients in the ICU. Future research should focus on implementation efforts and continuous reevaluation of quality indicators.

TRIAL REGISTRATION

The core study was registered with ClinicalTrials.gov, number NCT02210221, registered on August 06, 2014, with Resource Identification Portal (RRID: SCR_015582).

A nationwide overview of 1-year mortality in cardiac arrest patients admitted to intensive care units in the Netherlands between 2010 and 2016

AIM

Worldwide, cardiac arrest (CA) remains a major cause of death. Most post-CA patients are admitted to the intensive care unit (ICU). The aim of this study is to describe mortality rates and possible changes in mortality rates in patients with CA admitted to the ICU in the Netherlands between 2010 and 2016.

METHODS

In this study, we included all adult CA patients registered in the National Intensive Care Evaluation (NICE) registry who were admitted to ICUs in the Netherlands between 2010 and 2016. The primary outcome was 1-year mortality which was analysed by Cox regression. The secondary outcomes were ICU mortality and hospital mortality. Hospital mortality was analysed by binary logistic regression analysis. Patients were stratified by whether they experienced in-hospital cardiac arrest (IHCA) or out-of-hospital cardiac arrest (OHCA). Finally, the outcome over calendar time was assessed for both groups.

RESULTS

We included 26,056 CA patients: 10,618 (40.8%) IHCA patients and 14,482 (55.6%) OHCA patients. The 1-year mortality rate was 57.5%: 59% for IHCA and 56.4% for OHCA, p < 0.01. This mortality rate remained stable between 2010 and 2016 for IHCA (p = 0.31) and declined for OHCA patients (p = 0.01). The hospital mortality rate was 50.3%: 50.5% for IHCA and 50.2% for OHCA, p = 0.66. This mortality rate remained stable between 2010-2016 for IHCA (p = 0.21) and decreased for OHCA patients (p < 0.01). An additional analysis with calendar year as a continuous variable showed a mortality decline of 1.56% per calendar year for 1-year mortality.

CONCLUSION

This nationwide registry cohort study reported a 57.5% 1-year mortality rate for CA patients admitted to the ICU between 2010 and 2016. We reported a decline in 1-year mortality for OHCA patients in these years.

The influence of clinical variables on the risk of developing chronic conditions in ICU survivors

PURPOSE

To assess the association of clinical variables and the development of specified chronic conditions in ICU survivors.

MATERIALS AND METHODS

A retrospective cohort study, combining a national health insurance claims database and a national quality registry for ICUs. Claims data from 2012 to 2014 were combined with clinical data of patients admitted to an ICU during 2013. To assess the association of clinical variables (ICU length of stay, mechanical ventilation, acute physiology score, reason for ICU admission, mean arterial pressure score and glucose score) and the development of chronic conditions (i.e. heart diseases, COPD or asthma, Diabetes mellitus type II, depression and kidney diseases), logistic regression was used.

RESULTS

49,004 ICU patients were included. ICU length of stay was associated with the development of heart diseases, asthma or COPD and depression. The reason for ICU admission was an important risk factor for the development of all chronic conditions with adjusted ORs ranging from 2.05 (CI 1.56; 2.69) for kidney diseases to 5.14 (CI 3.99; 6.62) for depression.

CONCLUSIONS

Clinical variables, especially the reason for ICU admission, are associated with the development of chronic conditions after ICU discharge. Therefore, these clinical variables should be considered when organizing follow-up care for ICU survivors.

Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care

Background

Audit and feedback (A&F) is more effective if it facilitates action planning, but little is known about how best to do this. We developed an electronic A&F intervention with an action implementation toolbox to improve pain management in intensive care units (ICUs); the toolbox contained suggested actions for improvement. A head-to-head randomised trial demonstrated that the toolbox moderately increased the intervention’s effectiveness when compared with A&F only.

Objective

To understand the mechanisms through which A&F with action implementation toolbox facilitates action planning by ICUs to increase A&F effectiveness.

Methods

We extracted all individual actions from action plans developed by ICUs that received A&F with (n = 10) and without (n = 11) toolbox for 6 months and classified them using Clinical Performance Feedback Intervention Theory. We held semi-structured interviews with participants during the trial. We compared the number and type of planned and completed actions between study groups and explored barriers and facilitators to effective action planning.

Results

ICUs with toolbox planned more actions directly aimed at improving practice (p = 0.037) and targeted a wider range of practice determinants compared to ICUs without toolbox. ICUs with toolbox also completed more actions during the study period, but not significantly (p = 0.142). ICUs without toolbox reported more difficulties in identifying what actions they could take. Regardless of the toolbox, all ICUs still experienced barriers relating to the feedback (low controllability, accuracy) and organisational context (competing priorities, resources, cost).

Conclusions

The toolbox helped health professionals to broaden their mindset about actions they could take to change clinical practice. Without the toolbox, professionals tended to focus more on feedback verification and exploring solutions without developing intentions for actual change. All feedback recipients experienced organisational barriers that inhibited eventual completion of actions.

Trial registration

ClinicalTrials.gov, NCT02922101. Registered on 26 September 2016.

Electronic supplementary material

The online version of this article (10.1186/s13012-019-0937-8) contains supplementary material, which is available to authorized users.

Reflecting and Looking to the Future: What Is the Research Agenda for Theory in Health Informatics?

In this chapter, we reflect on the aim and objectives of the textbook and address known gaps in our theory coverage. We reinforce the importance of theory in health informatics and review the varying disciplinary origins of the theories considered in the book. We discuss the question of what makes a good theory and how to know which one is relevant for a given study. We recognize the limitations of the body of theory that we have presented and suggest what might be regarded as "native" theory that is original to health informatics. Finally, we propose topics to form a research agenda for theory in health informatics.

Impact of audit and feedback with action implementation toolbox on improving ICU pain management: cluster-randomised controlled trial

Background

Audit and feedback (A&F) enjoys widespread use, but often achieves only marginal improvements in care. Providing recipients of A&F with suggested actions to overcome barriers (action implementation toolbox) may increase effectiveness.

Objective

To assess the impact of adding an action implementation toolbox to an electronic A&F intervention targeting quality of pain management in intensive care units (ICUs).

Trial design

Two-armed cluster-randomised controlled trial. Randomisation was computer generated, with allocation concealment by a researcher, unaffiliated with the study. Investigators were not blinded to the group assignment of an ICU.

Participants

Twenty-one Dutch ICUs and patients eligible for pain measurement.

Interventions

Feedback-only versus feedback with action implementation toolbox.

Outcome

Proportion of patient-shift observations where pain management was adequate; composed by two process (measuring pain at least once per patient in each shift; re-measuring unacceptable pain scores within 1 hour) and two outcome indicators (acceptable pain scores; unacceptable pain scores normalised within 1 hour).

Results

21 ICUs (feedback-only n=11; feedback-with-toolbox n=10) with a total of 253 530 patient-shift observations were analysed. We found absolute improvement on adequate pain management in the feedback-with-toolbox group (14.8%; 95% CI 14.0% to 15.5%) and the feedback-only group (4.8%; 95% CI 4.2% to 5.5%). Improvement was limited to the two process indicators. The feedback-with-toolbox group achieved larger effects than the feedback-only group both on the composite adequate pain management (p<0.05) and on measuring pain each shift (p<0.001). No important adverse effects have occurred.

Conclusion

Feedback with toolbox improved the number of shifts where patients received adequate pain management compared with feedback alone, but only in process and not outcome indicators.

Trial registration number

NCT02922101.

Dutch ICU survivors have more consultations with general practitioners before and after ICU admission compared to a matched control group from the general population

Background

General Practitioners (GPs) play a key role in the healthcare trajectory of patients. If the patient experiences problems that are typically non-life-threatening, such as the symptoms of post-intensive-care syndrome, the GP will be the first healthcare professional they consult. The primary aim of this study is to gain insight in the frequency of GP consultations during the year before hospital admission and the year after discharge for ICU survivors and a matched control group from the general population. The secondary aim of this study is to gain insight into differences between subgroups of the ICU population with respect to the frequency of GP consultations.

Methods

We conducted a retrospective cohort study, combining a national health insurance claims database and a national quality registry for ICUs. Clinical data of patients admitted to an ICU in 2013 were enriched with claims data from the years 2012, 2013 and 2014. Poisson regression was used to assess the differences in frequency of GP consultations between the ICU population and the control group.

Results

ICU patients have more consultations with GPs during the year before and after admission than individuals in the control group. In the last four weeks before admission, ICU patients have 3.58 (CI 3.37; 3.80) times more GP consultations than the control group, and during the first four weeks after discharge they have 4.98 (CI 4.74; 5.23) times more GP consultations. In the year after hospital discharge ICU survivors have an increased GP consultation rate compared to the year before their hospital admission.

Conclusions

Close to hospital admission and shortly after hospital discharge, the frequency of GP consultations substantially increases in the population of ICU survivors. Even a year after hospital discharge, ICU survivors have increased GP consultation rates. Therefore, GPs should be well informed about the problems ICU patients suffer after discharge, in order to provide suitable follow-up care.

Determinants and outcomes of patient access to medical records: Systematic review of systematic reviews

BACKGROUND

Patient access to electronic health records (EHRs) is associated with several determinants and outcomes, which are interrelated. However, individual studies and the reviews summarizing them have only addressed particular aspects, such as policy, usability or health outcomes of adoption. Therefore, no comprehensive overview exists. Additionally, reviews used different theoretical frameworks, which makes results difficult to compare.

OBJECTIVE

We aimed to systematically review recent systematic reviews on determinants and outcomes of patient access to EHRs to create a comprehensive overview and inform policy-makers and EHR implementers about the available literature, and to identify knowledge gaps in the literature reviews.

METHODS

We searched MEDLINE, EMBASE and PsycINFO for systematic reviews on patient portals, personal health records, and patient access to records that addressed determinants and outcomes of adoption. We synthesized the results from these reviews into the Clinical Adoption Framework (CAF), by mapping quotes from the reviews to categories and dimensions of the CAF, starting with the most recent ones until saturation of the CAF had been reached. The risk of bias in the reviews was assessed using the AMSTAR2 checklist.

RESULTS

We included nineteen reviews from 8871 records that were retrieved until February 19^th, 2018. The reviews had a median of 4 (IQR: 4-4) critical flaws according to the AMSTAR2 checklist. The reviews contained a total of 1054 quotes that were mapped to the CAF. All reviews reported on the dimension 'People' that can affect adoption (e.g. personal characteristics such as age) and the dimension 'HIS use' (health information system use). Most reviews reported the dimensions 'Organisation', 'Implementation', HIS 'System quality', and outcomes of HIS use. However, gaps in knowledge might exist on macro-level determinants and outcomes, such as healthcare standards, funding, and incentives, because few reviews addressed these aspects.

CONCLUSIONS

No review covered all aspects of the CAF and there was a large variety in aspects that were addressed, but all dimensions of the CAF were addressed by at least two reviews. Although reviews had critical flaws according to the AMSTAR2 checklist, almost half of the reviews did use methods to assess bias in primary studies. Implementers can use the synthesized results from this study as a reference for implementation and development when taking quality restrictions into account. Researchers should address the risk of bias in primary studies in future reviews and use a framework such as CAF to make results more comparable and reusable.

Clinical performance comparators in audit and feedback: a review of theory and evidence

BACKGROUND

Audit and feedback (A&F) is a common quality improvement strategy with highly variable effects on patient care. It is unclear how A&F effectiveness can be maximised. Since the core mechanism of action of A&F depends on drawing attention to a discrepancy between actual and desired performance, we aimed to understand current and best practices in the choice of performance comparator.

METHODS

We described current choices for performance comparators by conducting a secondary review of randomised trials of A&F interventions and identifying the associated mechanisms that might have implications for effective A&F by reviewing theories and empirical studies from a recent qualitative evidence synthesis.

RESULTS

We found across 146 trials that feedback recipients' performance was most frequently compared against the performance of others (benchmarks; 60.3%). Other comparators included recipients' own performance over time (trends; 9.6%) and target standards (explicit targets; 11.0%), and 13% of trials used a combination of these options. In studies featuring benchmarks, 42% compared against mean performance. Eight (5.5%) trials provided a rationale for using a specific comparator. We distilled mechanisms of each comparator from 12 behavioural theories, 5 randomised trials, and 42 qualitative A&F studies.

CONCLUSION

Clinical performance comparators in published literature were poorly informed by theory and did not explicitly account for mechanisms reported in qualitative studies. Based on our review, we argue that there is considerable opportunity to improve the design of performance comparators by (1) providing tailored comparisons rather than benchmarking everyone against the mean, (2) limiting the amount of comparators being displayed while providing more comparative information upon request to balance the feedback's credibility and actionability, (3) providing performance trends but not trends alone, and (4) encouraging feedback recipients to set personal, explicit targets guided by relevant information.

Development of a quality indicator set to measure and improve quality of ICU care for patients with traumatic brain injury

Background

We aimed to develop a set of quality indicators for patients with traumatic brain injury (TBI) in intensive care units (ICUs) across Europe and to explore barriers and facilitators for implementation of these quality indicators.

Methods

A preliminary list of 66 quality indicators was developed, based on current guidelines, existing practice variation, and clinical expertise in TBI management at the ICU. Eight TBI experts of the Advisory Committee preselected the quality indicators during a first Delphi round. A larger Europe-wide expert panel was recruited for the next two Delphi rounds. Quality indicator definitions were evaluated on four criteria: validity (better performance on the indicator reflects better processes of care and leads to better patient outcome), feasibility (data are available or easy to obtain), discriminability (variability in clinical practice), and actionability (professionals can act based on the indicator). Experts scored indicators on a 5-point Likert scale delivered by an electronic survey tool.

Results

The expert panel consisted of 50 experts from 18 countries across Europe, mostly intensivists (N = 24, 48%) and neurosurgeons (N = 7, 14%). Experts agreed on a final set of 42 indicators to assess quality of ICU care: 17 structure indicators, 16 process indicators, and 9 outcome indicators. Experts are motivated to implement this finally proposed set (N = 49, 98%) and indicated routine measurement in registries (N = 41, 82%), benchmarking (N = 42, 84%), and quality improvement programs (N = 41, 82%) as future steps. Administrative burden was indicated as the most important barrier for implementation of the indicator set (N = 48, 98%).

Conclusions

This Delphi consensus study gives insight in which quality indicators have the potential to improve quality of TBI care at European ICUs. The proposed quality indicator set is recommended to be used across Europe for registry purposes to gain insight in current ICU practices and outcomes of patients with TBI. This indicator set may become an important tool to support benchmarking and quality improvement programs for patients with TBI in the future.

Electronic supplementary material

The online version of this article (10.1186/s13054-019-2377-x) contains supplementary material, which is available to authorized users.

Impact of Electronic versus Paper-Based Recording before EHR Implementation on Health Care Professionals' Perceptions of EHR Use, Data Quality, and Data Reuse

BACKGROUND

The implementation of an electronic health record (EHR) with structured and standardized recording of patient data can improve data quality and reusability. Whether and how users perceive these advantages may depend on the preimplementation situation.

OBJECTIVE

To determine whether the influence of implementing a structured and standardized EHR on perceived EHR use, data quality, and data reuse differed for users working with paper-based records versus a legacy EHR before implementation.

METHODS

We used an electronic questionnaire to measure users' perception before implementation (2014), expected change, and perceived change after implementation (2016) on three themes. We included all health care professionals in two university hospitals in the Netherlands. Before jointly implementing the same structured and standardized EHR, one hospital used paper-based records and the other a legacy EHR. We compared perceptions before and after implementation for both centers. Additionally, we compared expected benefit with perceived benefit.

RESULTS

We received 7,611 responses (4,537 before and 3,074 after implementation) of which 5,707 (75%) were from professionals reading and recording patient data. A total of 975 (13%) professionals responded to both before and after implementation questionnaires. In the formerly paper-based center staff perceived improvement in all themes after implementation. The legacy EHR center experienced deterioration of perceived EHR use and data reuse, and only one improvement in EHR use. In both centers, for half of the aspects at least 45% of responders experienced results worse than expected preimplementation.

CONCLUSION

Our results indicate that the preimplementation recording practice impacts the perceived effect of the implementation of a structured and standardized EHR. For almost half of the respondents the new EHR did not meet their expectations. Especially legacy EHR centers need to investigate the expectations as these might be different and less clear cut than those in paper-based centers. These expectations need to be addressed appropriately to achieve a successful implementation.