A usability evaluation of the perceived user friendliness, accessibility, and inclusiveness of a personalized digital care pathway tool

OBJECTIVE

This study aimed to acquire insight into the perceived user friendliness, accessibility and inclusiveness of a personalized digital care pathway.

MATERIALS & METHODS

Usability of the tool was tested in an experimental setting. Mixed methods data collection consisted of scenario-based eye tracking tests in a web- or mobile-based prototype of the tool, followed by a questionnaire assessing user friendliness (System Usability Scale; SUS) and a structured interview. Inclusiveness was assessed by subgroup comparisons based on language proficiency, age and education level. Via purposive sampling a heterogeneous population of users (N = 24) was recruited. Eye tracking was used to measure gaze behavior.

RESULTS

Overall, participants were satisfied with the tool (scale 0-10, 7.5; SD = 1.29). User friendliness of the mobile version (68.3; SD = 21.6) was higher than the web version (50.9; SD = 17.3) measured by SUS score (0-100). With regard to accessibility, eye tracking scenarios showed that the menu bar was hard to find (17% mobile, 55% web). In all scenario's, information was found faster in the mobile version than the web version. Attention was easily drawn to images. Regarding inclusiveness of the tool, we found significantly longer completing time of the scenario tasks for low language proficiency (p-value = 0.029) and higher age subgroups (p-value = 0.049). Lower language proficiency scored a significant lower SUS score (p-value = 0.012).

CONCLUSIONS

Overall, user friendliness and accessibility were positively evaluated. Assessment of inclusiveness emphasized the need for tailoring digital tools to those with low language proficiency and/or an older age. Co-creation of digital care tools with users is therefore important to match users' needs, make tools easily understandable and accessible to all users, and ultimately result in better uptake and impact.

Quality improvements of healthcare trajectories by learning from aggregated patient-reported outcomes: a mixed-methods systematic literature review

Background

In healthcare, analysing patient-reported outcome measures (PROMs) on an aggregated level can improve and regulate healthcare for specific patient populations (meso level). This mixed-methods systematic review aimed to summarize and describe the effectiveness of quality improvement methods based on aggregated PROMs. Additionally, it aimed to describe barriers, facilitators and lessons learned when using these quality improvement methods.

Methods

A mixed-methods systematic review was conducted. Embase, MEDLINE, CINAHL and the Cochrane Library were searched for studies that described, implemented or evaluated a quality improvement method based on aggregated PROMs in the curative hospital setting. Quality assessment was conducted via the Mixed Methods Appraisal Tool. Quantitative data were synthesized into a narrative summary of the characteristics and findings. For the qualitative analysis, a thematic synthesis was conducted.

Results

From 2360 unique search records, 13 quantitative and three qualitative studies were included. Four quality improvement methods were identified: benchmarking, plan-do-study-act cycle, dashboards and internal statistical analysis. Five studies reported on the effectiveness of the use of aggregated PROMs, of which four identified no effect and one a positive effect. The qualitative analysis identified the following themes for facilitators and barriers: (1) conceptual (i.e. stakeholders, subjectivity of PROMs, aligning PROMs with clinical data, PROMs versus patient-reported experience measures [PREMs]); (2a) methodological—data collection (i.e. choice, timing, response rate and focus); (2b) methodological—data processing (i.e. representativeness, responsibility, case-mix control, interpretation); (3) practical (i.e. resources).

Conclusion

The results showed little to no effect of quality improvement methods based on aggregated PROMs, but more empirical research is needed to investigate different quality improvement methods. A shared stakeholder vision, selection of PROMs, timing of measurement and feedback, information on interpretation of data, reduction of missing data, and resources for data collection and feedback infrastructure are important to consider when implementing and evaluating quality improvement methods in future research.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12961-022-00893-4.

What is already known on this topic

The aggregated patient-reported outcome measures (PROMs) can be used for analytical and organizational aspects of improving and regulating healthcare, but there is little empirical evidence regarding the effectiveness of aggregated PROMS.

What this study adds

This study adds a detailed overview of the types of quality improvement methods and recommendations for implementation in practice.

How this study might affect research, practice or policy

Researchers and policy-makers should consider the barriers, facilitators and lessons learned for future implementation and evaluation of quality improvement methods, as presented in this manuscript, to further advance this field.

Feasibility of reusing routinely recorded data to monitor the safe preparation and administration of injectable medication: A multicenter cross-sectional study

BACKGROUND

Reusing routinely recorded data from electronic hospital records (EHR) may offer a less-time consuming, and more real time alternative for monitoring compliance by nurses with a protocol for the safe preparation and administration of injectable medication. However, at present it is unknown if the data necessary to calculate the quality indicators (QIs) are recorded in EHRs, or if these data are suitable for automated QI calculation. Therefore, the aim of this study was to determine the feasibility of monitoring compliance by nurses with a protocol for the safe injectable medication preparation and administration by reusing routinely recorded EHR data for the automated calculation of QIs.

METHODS

A cross-sectional study in 12 Dutch hospitals (October 2015-May 2016). The checks included in the currently prevailing national protocol for the safe preparation and administration of injectable medication were translated into 16 data elements required to calculate the QIs. At each hospital, one interview was conducted using a structured questionnaire to decide whether the data elements were available in EHRs. To present these results, descriptive statistics were used.

RESULTS

In total, 20 health-care professionals were interviewed and four different EHR systems were evaluated. The availability of data elements was comparable between the four evaluated EHR systems. Nine of the 16 required data elements were recorded in EHRs, eight in a structured format. The seven missing data elements were mainly related to checks such as 'gather all materials needed' or 'conduct hand hygiene'. Furthermore, changes were identified in the process for the preparation and administration of injectable medication. These changes are mostly related to the increased use of electronic medication administration registration and barcode medication administration systems.

CONCLUSIONS

Reusing EHR data to monitor compliance by nurses with the currently prevailing protocol for the safe preparation and administration of injectable medication is not entirely feasible. A decision should be made on which checks should be recorded in the EHRs and which checks should be audited in order to minimize the registration burden for nurses. Moreover, the currently prevailing protocol should be revised to bring it in line with work-as-done. Our results can be used as guidance for such a revision and also for designing new QIs that can be calculated by reusing routinely recorded EHR data.

Towards a patient journey perspective on causes of unplanned readmissions using a classification framework: results of a systematic review with narrative synthesis

Background

Several literature reviews have been published focusing on the prevalence and/or preventability of hospital readmissions. To our knowledge, none focused on the different causes which have been used to evaluate the preventability of readmissions. Insight into the range of causes is crucial to understand the complex nature of readmissions.

With this review we aim to: 1) evaluate the range of causes of unplanned readmissions in a patient journey, and 2) present a cause classification framework that can support future readmission studies.

Methods

A literature search was conducted in PUBMED and EMBASE using “readmission” and “avoidability” or “preventability” as key terms. Studies that specified causes of unplanned readmissions were included. The causes were classified into eight preliminary root causes: Technical, Organization (integrated care), Organization (hospital department level), Human (care provider), Human (informal caregiver), Patient (self-management), Patient (disease), and Other. The root causes were based on expert opinions and the root cause analysis tool of PRISMA (Prevention and Recovery Information System for Monitoring and Analysis). The range of different causes were analyzed using Microsoft Excel.

Results

Forty-five studies that reported 381 causes of readmissions were included. All studies reported causes related to organization of care at the hospital department level. These causes were often reported as preventable. Twenty-two studies included causes related to patient’s self-management and 19 studies reported causes related to patient’s disease. Studies differed in which causes were seen as preventable or unpreventable. None reported causes related to technical failures and causes due to integrated care issues were reported in 18 studies.

Conclusions

This review showed that causes for readmissions were mainly evaluated from a hospital perspective. However, causes beyond the scope of the hospital can also play a major role in unplanned readmissions. Opinions regarding preventability seem to depend on contextual factors of the readmission. This study presents a cause classification framework that could help future readmission studies to gain insight into a broad range of causes for readmissions in a patient journey.

In conclusion, we aimed to: 1) evaluate the range of causes for unplanned readmissions, and 2) present a cause classification framework for causes related to readmissions.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0822-9) contains supplementary material, which is available to authorized users.

Effect of a cluster randomised team training intervention on adverse perinatal and maternal outcomes: a stepped wedge study

OBJECTIVE

To study the effect of an intervention based on Crew Resource Management team training, including a tool for structured communication, on adverse perinatal and maternal outcomes.

DESIGN

Stepped wedge.

SETTING

The Netherlands.

POPULATION OR SAMPLE

Registry data of 8123 women referred from primary care to a hospital during childbirth, at ≥ 32.0 weeks of singleton gestation and with no congenital abnormalities, in the period 2012-15.

METHODS

Obstetric teams of five hospitals and their surrounding primary-care midwifery practices participated in the intervention. In total, 49 team training sessions were organised for 465 care professionals (75.5% participated). Adverse perinatal and maternal outcomes before, during and after the intervention were analysed using multivariate logistic regression analyses.

MAIN OUTCOME MEASURES

Adverse Outcome Index (AOI-5), a composite measure involving; intrapartum or neonatal death, admission to neonatal intensive care unit, Apgar < 7 at 5 minutes, postpartum haemorrhage and/or perineal tear.

RESULTS

In total, an AOI-5 score was reported in 11.3% of the study population. No significant difference was found in the incidence of the AOI-5 score after the intervention compared with before the intervention (OR 1.07: 95% CI 0.92-1.24).

CONCLUSIONS

We found no effect of the intervention on adverse perinatal and maternal outcomes for women who were referred during childbirth. Team training is appreciated in practice, but evidence on the long-term impact is still limited. Upcoming studies should build on previous research and consider more sensitive outcome measures.

TWEETABLE ABSTRACT

A cluster randomised team training intervention showed no effect on adverse perinatal and maternal outcomes for women referred during childbirth.

Benchmarking the use of blood products in cardiac surgery to stimulate awareness of transfusion behaviour : Results from a four-year longitudinal study

Introduction

Cardiac operations account for a large proportion of the blood transfusions given each year, leading to high costs and an increased risk to patient safety. Therefore, it is important to explore initiatives to reduce transfusion rates. This study aims to provide a benchmark for transfusion practice by inter-hospital comparison of transfusion rates, blood product use and costs related to patients undergoing coronary artery bypass grafting (CABG), valve surgery or combined CABG and valve surgery.

Methods

Between 2010 and 2013, patients from four Dutch hospitals undergoing CABG, valve surgery or combined CABG and valve surgery (n = 11,150) were included by means of a retrospective longitudinal study design.

Results

In CABG surgery the transfusion rate ranged between 43 and 54%, in valve surgery between 54 and 67%, and in combined CABG and valve surgery between 80 and 88%. With the exception of one hospital, the trend in transfusion rate showed a significant decrease over time for all procedures. Hospitals differed significantly in the units of blood products given to each patient, and in the use of specific transfused combinations of blood products, such as red blood cells (RBCs) and a combination of RBCs, fresh frozen plasma (FFP) and platelets.

Conclusion

This study indicates that benchmarking blood product usage stimulates awareness of transfusion behaviour, which may lead to better patient safety and lower costs. Further studies are warranted to improve awareness of transfusion behaviour and increase the standardisation of transfusion practice in cardiac surgery.

Policy evaluation in diabetes prevention and treatment using a population-based macro simulation model: the MICADO model

AIMS

To test a simulation model, the MICADO model, for estimating the long-term effects of interventions in people with and without diabetes.

METHODS

The MICADO model includes micro- and macrovascular diseases in relation to their risk factors. The strengths of this model are its population scope and the possibility to assess parameter uncertainty using probabilistic sensitivity analyses. Outcomes include incidence and prevalence of complications, quality of life, costs and cost-effectiveness. We externally validated MICADO's estimates of micro- and macrovascular complications in a Dutch cohort with diabetes (n = 498,400) by comparing these estimates with national and international empirical data.

RESULTS

For the annual number of people undergoing amputations, MICADO's estimate was 592 (95% interquantile range 291-842), which compared well with the registered number of people with diabetes-related amputations in the Netherlands (728). The incidence of end-stage renal disease estimated using the MICADO model was 247 people (95% interquartile range 120-363), which was also similar to the registered incidence in the Netherlands (277 people). MICADO performed well in the validation of macrovascular outcomes of population-based cohorts, while it had more difficulty in reflecting a highly selected trial population.

CONCLUSIONS

Validation by comparison with independent empirical data showed that the MICADO model simulates the natural course of diabetes and its micro- and macrovascular complications well. As a population-based model, MICADO can be applied for projections as well as scenario analyses to evaluate the long-term (cost-)effectiveness of population-level interventions targeting diabetes and its complications in the Netherlands or similar countries.

Data quality issues impede comparability of hospital treatment delay performance indicators

Aim

To assess the comparability of five performance indicator scores for treatment delay among patients diagnosed with ST-segment elevation myocardial infarction (STEMI) undergoing primary percutaneous coronary intervention in relation to the quality of the underlying data.

Methods

Secondary analyses were performed on data from 1017 patients in seven Dutch hospitals. Data were collected using standardised forms for patients discharged in 2012. Comparability was assessed as the number of occasions the indicator threshold was reached for each hospital.

Results

Hospitals recorded different time points based on different interpretations of the definitions. This led to substantial differences in indicator scores, ranging from 57 to 100 % of the indictor threshold being reached. Some hospitals recorded all the required data elements for calculating the performance indicators but none of the data elements could be retrieved in a fully automated way. Moreover, recording accessibility and completeness of time points varied widely within and between hospitals.

Conclusion

Hospitals use different definitions for treatment delay and vary greatly in the extent to which the necessary data are available, accessible and complete, impeding comparability between hospitals. Indicator developers, users and hospitals providing data should be aware of these issues and aim to improve data quality in order to facilitate comparability of performance indicators.

Adherence to guidelines for the prescription of secondary prevention medication at hospital discharge after acute coronary syndrome: a multicentre study

Background

The prescription of guideline-recommended medication for secondary prevention after acute coronary syndrome has been suboptimal in the past. In the present study, guideline adherence and associated patient, care and hospital characteristics at hospital discharge after acute coronary syndrome were studied.

Methods

Charts of patients with acute coronary syndrome discharged from 13 Dutch hospitals in 2012 were reviewed. Guideline adherence was defined as the prescription of acetylsalicylic acid, P2Y₁₂ receptor inhibitor, statin, beta-blocker and angiotensin-converting enzyme (ACE) inhibitor at discharge, or a documented contraindication. Associated characteristics were identified by means of generalized linear mixed models for binary outcomes.

Results

In total, 2471 patients were included. Complete guideline adherence was achieved in 69.1 % of the patients, ranging from 42.1 to 87.0 % between hospitals. The ACE inhibitor was most often missing (21.2 %). Patients with non-ST-segment elevation myocardial infarction or unstable angina, patients with a history of coronary artery bypass grafting or elderly women were less likely to be discharged with the guideline-recommended medication.

Conclusions

Guideline adherence for secondary prevention medication following acute coronary syndrome was substantial; however, variation between hospitals and patient groups was found. Efforts to increase guideline adherence can focus on underperforming hospitals and undertreated patient groups.

Monitoring guideline adherence in the management of acute coronary syndrome in hospitals: design of a multicentre study

Background

Increasing guideline adherence in the management of acute coronary syndrome (ACS) in hospitals potentially reduces heart failure and mortality. Therefore, an expert panel identified three guideline recommendations as the most important aims for improvement in ACS care, i.e. timely invasive treatment, use of risk scoring instruments and prescription of secondary prevention medication at discharge.

Aims

This study aims to evaluate in-hospital guideline adherence in the care of patients diagnosed with ACS and to identify associated factors.

Methods

The study has a cross-sectional design. Data are collected in 13 hospitals in the Netherlands by means of retrospective chart review of patients discharged in 2012 with a diagnosis of ACS. The primary outcomes will be the percentages of patients receiving timely invasive treatment, with a documented cardiac risk score, and with a prescription of the guideline-recommended discharge medication. In addition, factors associated with guideline adherence will be studied using generalised linear (mixed) models.

Discussion

This study explores guideline adherence in Dutch hospitals in the management of patients diagnosed with ACS, using a data source universally available in hospitals. The results of this study can be informative for professionals involved in ACS care as they facilitate targeted improvement efforts.

The prognosis of heart failure in the general population: The Rotterdam Study

AIMS

To determine the prognosis, cause of death, and its determinants in participants of the population-based Rotterdam Study who were found to have heart failure.

METHODS AND RESULTS

In 5255 Rotterdam Study participants (aged 68.9+/-8.6 years, 3113 women) the presence of heart failure was determined. Data were analysed with Cox's proportional-hazards models. One hundred and eighty-one participants (age 77.3+/-7.9 years, 109 women) had heart failure. Of these 85 (47%) died during the 4.8-8.5 (mean 6.1) years of follow-up. One, 2 and 5 years' survival was 89%, 79%, and 59%, representing an age-adjusted mortality twice that of persons without heart failure (hazard ratio 2.1, 95% CI 1.8-2.7). The hazard ratio for sudden death was even more pronounced: 4.8, (95% CI 2.6-8.7). Diabetes mellitus, impairment of renal function and atrial fibrillation were associated with a poor outcome. A higher blood pressure and body mass index conferred a more favourable prognosis in those with heart failure.

CONCLUSION

Heart failure generally afflicts older subjects in the community, carries a poor prognosis, especially in the presence of concomitant diseases, and confers a fivefold increase in the risk of sudden death.

Usefulness of echocardiography in detecting left ventricular dysfunction in population-based studies (The Rotterdam Study)

In a population-based study, the routine 12-lead electrocardiogram was found to have a high negative predictive value for detecting left ventricular (LV) systolic dysfunction. Withholding echocardiography in persons without major electrocardiographic abnormalities, however, would result in a considerable underestimation of LV systolic dysfunction (sensitivity only 54%); thus, echocardiography remains an essential tool for detecting LV systolic dysfunction in population-based studies.

[Coronary heart disease in the elderly; the ERGO study (Erasmus Rotterdam Health and the Elderly)]

OBJECTIVE

To determine the prevalence of cardiovascular disease among elderly people and to investigate the use of cardiovascular medication in this group.

DESIGN

Cross-sectional study.

SETTING

A quarter in Rotterdam, the Netherlands.

METHODS

As a part of the 'Rotterdam elderly study', the cardiovascular anamnesis of 7.983 responding persons > or = 55 years (3105 men and 4878 women) was obtained by means of a questionnaire.

RESULTS

Twelve and 4% of men and women, respectively, had at some time had a myocardial infarction, and 5 and 4% a stroke. Almost 7% of both sexes had anamnestic symptoms of angina pectoris. Coronary bypass surgery and percutaneous transluminal coronary angioplasty were done in 4% and 1% of men and 1% and 0.5% of women, respectively. Of all participants 43% used at least one drug for cardiovascular purposes. Half of these used two or more, and a quarter used three or more drugs. Among women, diuretics were the most commonly used drug for cardiovascular purposes (21%), whereas among men beta-blocking agents were most common (15%).

CONCLUSION

Cardiovascular diseases are common among elderly men and women. Many of them take at least one drug for cardiovascular purposes.