Protocol for an intervention development and pilot implementation evaluation study of an e-health solution to improve newborn care quality and survival in two low-resource settings, Malawi and Zimbabwe: Neotree

INTRODUCTION

Every year 2.4 million deaths occur worldwide in babies younger than 28 days. Approximately 70% of these deaths occur in low-resource settings because of failure to implement evidence-based interventions. Digital health technologies may offer an implementation solution. Since 2014, we have worked in Bangladesh, Malawi, Zimbabwe and the UK to develop and pilot Neotree: an android app with accompanying data visualisation, linkage and export. Its low-cost hardware and state-of-the-art software are used to improve bedside postnatal care and to provide insights into population health trends, to impact wider policy and practice.

METHODS AND ANALYSIS

This is a mixed methods (1) intervention codevelopment and optimisation and (2) pilot implementation evaluation (including economic evaluation) study. Neotree will be implemented in two hospitals in Zimbabwe, and one in Malawi. Over the 2-year study period clinical and demographic newborn data will be collected via Neotree, in addition to behavioural science informed qualitative and quantitative implementation evaluation and measures of cost, newborn care quality and usability. Neotree clinical decision support algorithms will be optimised according to best available evidence and clinical validation studies.

ETHICS AND DISSEMINATION

This is a Wellcome Trust funded project (215742_Z_19_Z). Research ethics approvals have been obtained: Malawi College of Medicine Research and Ethics Committee (P.01/20/2909; P.02/19/2613); UCL (17123/001, 6681/001, 5019/004); Medical Research Council Zimbabwe (MRCZ/A/2570), BRTI and JREC institutional review boards (AP155/2020; JREC/327/19), Sally Mugabe Hospital Ethics Committee (071119/64; 250418/48). Results will be disseminated via academic publications and public and policy engagement activities. In this study, the care for an estimated 15 000 babies across three sites will be impacted.

TRIAL REGISTRATION NUMBER

NCT0512707; Pre-results.

Overcoming Major Barriers to Build Efficient Decision Support Systems in Pharmacovigilance

Many decision support methods and systems in pharmacovigilance are built without explicitly addressing specific challenges that jeopardize their eventual success. We describe two sets of challenges and appropriate strategies to address them. The first are data-related challenges, which include using extensive multi-source data of poor quality, incomplete information integration, and inefficient data visualization. The second are user-related challenges, which encompass users' overall expectations and their engagement in developing automated solutions. Pharmacovigilance decision support systems will need to rely on advanced methods, such as natural language processing and validated mathematical models, to resolve data-related issues and provide properly contextualized data. However, sophisticated approaches will not provide a complete solution if end-users do not actively participate in their development, which will ensure tools that efficiently complement existing processes without creating unnecessary resistance. Our group has already tackled these issues and applied the proposed strategies in multiple projects.

A transfer learning approach to facilitate ComBat-based harmonization of multicentre radiomic features in new datasets

PURPOSE

To facilitate the demonstration of the prognostic value of radiomics, multicenter radiomics studies are needed. Pooling radiomic features of such data in a statistical analysis is however challenging, as they are sensitive to the variability in scanner models, acquisition protocols and reconstruction settings, which is often unavoidable in a multicentre retrospective analysis. A statistical harmonization strategy called ComBat was utilized in radiomics studies to deal with the "center-effect". The goal of the present work was to integrate a transfer learning (TL) technique within ComBat-and recently developed alternate versions of ComBat with improved flexibility (M-ComBat) and robustness (B-ComBat)-to allow the use of a previously determined harmonization transform to the radiomic feature values of new patients from an already known center.

MATERIAL AND METHODS

The proposed TL approach were incorporated in the four versions of ComBat (standard, B, M, and B-M ComBat). The proposed approach was evaluated using a dataset of 189 locally advanced cervical cancer patients from 3 centers, with magnetic resonance imaging (MRI) and positron emission tomography (PET) images, with the clinical endpoint of predicting local failure. The impact performance of the TL approach was evaluated by comparing the harmonization achieved using only parts of the data to the reference (harmonization achieved using all the available data). It was performed through three different machine learning pipelines.

RESULTS

The proposed TL technique was successful in harmonizing features of new patients from a known center in all versions of ComBat, leading to predictive models reaching similar performance as the ones developed using the features harmonized with all the data available.

CONCLUSION

The proposed TL approach enables applying a previously determined ComBat transform to new, previously unseen data.

Accuracy assessment methods for physiological model selection toward evaluation of closed-loop controlled medical devices

Physiological closed-loop controlled (PCLC) medical devices are complex systems integrating one or more medical devices with a patient’s physiology through closed-loop control algorithms; introducing many failure modes and parameters that impact performance. These control algorithms should be tested through safety and efficacy trials to compare their performance to the standard of care and determine whether there is sufficient evidence of safety for their use in real care setting. With this aim, credible mathematical models have been constructed and used throughout the development and evaluation phases of a PCLC medical device to support the engineering design and improve safety aspects. Uncertainties about the fidelity of these models and ambiguities about the choice of measures for modeling performance need to be addressed before a reliable PCLC evaluation can be achieved. This research develops tools for evaluating the accuracy of physiological models and establishes fundamental measures for predictive capability assessment across different physiological models. As a case study, we built a refined physiological model of blood volume (BV) response by expanding an original model we developed in our prior work. Using experimental data collected from 16 sheep undergoing hemorrhage and fluid resuscitation, first, we compared the calibration performance of the two candidate physiological models, i.e., original and refined, using root-mean-squared error (RMSE), Akiake information criterion (AIC), and a new multi-dimensional approach utilizing normalized features extracted from the fitting error. Compared to the original model, the refined model demonstrated a significant improvement in calibration performance in terms of RMSE (9%, P = 0.03) and multi-dimensional measure (48%, P = 0.02), while a comparable AIC between the two models verified that the enhanced calibration performance in the refined model is not due to data over-fitting. Second, we compared the physiological predictive capability of the two models under three different scenarios: prediction of subject-specific steady-state BV response, subject-specific transient BV response to hemorrhage perturbation, and leave-one-out inter-subject BV response. Results indicated enhanced accuracy and predictive capability for the refined physiological model with significantly larger proportion of measurements that were within the prediction envelope in the transient and leave-one-out prediction scenarios (P < 0.02). All together, this study helps to identify and merge new methods for credibility assessment and physiological model selection, leading to a more efficient process for PCLC medical device evaluation.

Development of safety and usability guideline for clinical information system

Clinical information systems (CISs) that do not consider usability and safety could lead to harmful events. Therefore, we aimed to develop a safety and usability guideline of CISs that is comprehensive for both users and developers. And the guideline was categorized to apply actual clinical workflow and work environment.The guideline components were extracted through a systematic review of the articles published between 2000 and 2015, and existing CIS safety and/or usability design guidelines. The guideline components were categorized according to clinical workflow and types of user interface (UI). The contents of the guideline were evaluated and validated by experts with 3 specialties: medical informatics, patient safety, and human engineering.Total 1276 guideline components were extracted through article and guideline review. Of these, 464 guideline components were categorized according to 5 divisions of the clinical workflow: "Data identification and selection," "Document entry," "Order entry," "Clinical decision support and alert," and "Management". While 521 guideline components were categorized according to 4 divisions of UI: UIs related to information process steps, "Perception," "Recognition," "Control," and "Feedback". We developed a guideline draft with 219 detailed guidance for clinical task and 70 for UI. Overall appropriateness and comprehensiveness were proven to achieve more than 90% in experts' survey. However, there were significant differences among the groups of specialties in the judgment of appropriateness (P < .001) and comprehensiveness (P = .038).We developed and verified a safety and usability guideline for CIS that qualifies the requirements of both clinical workflows and usability issues. The developed guideline can be a practical tool to enhance the usability and safety of CISs. Further validation is required by applying the guideline for designing the actual CIS.

Effects of patients' hospital discharge preferences on uptake of clinical decision support

The Centers for Medicare and Medicaid Services identified unplanned hospital readmissions as a critical healthcare quality and cost problem. Improvements in hospital discharge decision-making and post-discharge care are needed to address the problem. Utilization of clinical decision support (CDS) can improve discharge decision-making but little is known about the empirical significance of two opposing problems that can occur: (1) negligible uptake of CDS by providers or (2) over-reliance on CDS and underuse of other information. This paper reports an experiment where, in addition to electronic medical records (EMR), clinical decision-makers are provided subjective reports by standardized patients, or CDS information, or both. Subjective information, reports of being eager or reluctant for discharge, was obtained during examinations of standardized patients, who are regularly employed in medical education, and in our experiment had been given scripts for the experimental treatments. The CDS tool presents discharge recommendations obtained from econometric analysis of data from de-identified EMR of hospital patients. 38 clinical decision-makers in the experiment, who were third and fourth year medical students, discharged eight simulated patient encounters with an average length of stay 8.1 in the CDS supported group and 8.8 days in the control group. When the recommendation was “Discharge,” CDS uptake of “Discharge” recommendation was 20% higher for eager than reluctant patients. Compared to discharge decisions in the absence of patient reports: (i) odds of discharging reluctant standardized patients were 67% lower in the CDS-assisted group and 40% lower in the control (no-CDS) group; whereas (ii) odds of discharging eager standardized patients were 75% higher in the control group and similar in CDS-assisted group. These findings indicate that participants were neither ignoring nor over-relying on CDS.

A Multivariable Prediction Model to Select Colorectal Surgical Patients for Co-Management

INTRODUCTION

Increased life expectancy leads to older and frailer surgical patients. Co-management between medical and surgical specialities has proven favourable in complex situations. Selection of patients for co-management is full of difficulties. The aim of this study was to develop a clinical decision support tool to select surgical patients for co-management.

MATERIAL AND METHODS

Clinical data was collected from patient electronic health records with an ICD-9 code for colorectal surgery from January 2012 to December 2015 at a hospital in Lisbon. The outcome variable consists in co-management signalling. A dataset from 344 patients was used to develop the prediction model and a second data set from 168 patients was used for external validation.

RESULTS

Using logistic regression modelling the authors built a five variable (age, burden of comorbidities, ASA-PS status, surgical risk and recovery time) predictive referral model for co-management. This model has an area under the curve (AUC) of 0.86 (95% CI: 0.81 - 0.90), a predictive Brier score of 0.11, a sensitivity of 0.80, a specificity of 0.82 and an accuracy of 81.3%.

DISCUSSION

Early referral of high-risk patients may be valuable to guide the decision on the best level of post-operative clinical care. We developed a simple bedside decision tool with a good discriminatory and predictive performance in order to select patients for comanagement.

CONCLUSION

A simple bed-side clinical decision support tool of patients for co-management is viable, leading to potential improvement in early recognition and management of postoperative complications and reducing the 'failure to rescue'. Generalizability to other clinical settings requires adequate customization and validation.

[Clinical history in times of big data; a plea for a standard for the structured recording of the clinical history]

Clinical decision support systems to aid the clinician in making a correct diagnosis will only succeed if data from the clinical history are taken into account. However, currently, very little is known on diagnostic test characteristics of specific symptoms, let alone of a pattern of several symptoms with all their cardinal features. We plead for the nation-wide introduction of a standard for the structured recording of the clinical history. To allow for such structured recording, user interfaces of electronic healthcare records must become far more user-friendly. Furthermore, scribes may be used, or, ideally, a digital scribe, a computer application that records the conversation between healthcare professional and patient and creates an automated summary. So far, to our knowledge, no digital scribe encompassing the entire patient history has been implemented into medical practice. We are currently trying to develop such a digital scribe.

Decision support systems for the prediction of lymph node involvement in early breast cancer

The prediction of lymph node involvement represents an important task which could reduce unnecessary surgery and improve the definition of oncological therapies. An artificial intelligence model able to predict it in pre-operative phase requires the interface among multiple data structures. The trade-off among time consuming, expensive and invasive methodologies is emerging in experimental setups exploited for the analysis of sentinel lymph nodes, where machine learning algorithms represent a key ingredient in recorded data elaboration. The accuracy required for clinical applications is obtainable matching different kind of data. Statistical associations of prognostic factors with symptoms and predictive models implemented also through on-line softwares represent useful diagnostic support tools which translate into patients quality of life improvement and costs reduction.

Unexpected Inequality: Disparate-Impact From Artificial Intelligence in Healthcare Decisions

Systemic discrimination in healthcare plagues marginalized groups. Physicians incorrectly view people of color as having high pain tolerance, leading to undertreatment. Women with disabilities are often undiagnosed because their symptoms are dismissed. Low-income patients have less access to appropriate treatment. These patterns, and others, reflect long-standing disparities that have become engrained in U.S. health systems. As the healthcare industry adopts artificial intelligence and algorithminformed (AI) tools, it is vital that regulators address healthcare discrimination. AI tools are increasingly used to make both clinical and administrative decisions by hospitals, physicians, and insurers--yet there is no framework that specifically places nondiscrimination obligations on AI users. The Food and Drug Administration has limited authority to regulate AI and has not sought to incorporate anti-discrimination principles in its guidance. Section 1557 of the Affordable Care Act has not been used to enforce nondiscrimination in healthcare AI and is under-utilized by the Office of Civil Rights. State level protections by medical licensing boards or malpractice liability are similarly untested and have not yet extended nondiscrimination obligations to AI. This Article discusses the role of each legal obligation on healthcare AI and the ways in which each system can improve to address discrimination. It highlights the ways in which industries can self-regulate to set nondiscrimination standards and concludes by recommending standards and creating a super-regulator to address disparate impact by AI. As the world moves towards automation, it is imperative that ongoing concerns about systemic discrimination are removed to prevent further marginalization in healthcare.

Economic Value of Data and Analytics for Health Care Providers: Hermeneutic Systematic Literature Review

BACKGROUND

The benefits of data and analytics for health care systems and single providers is an increasingly investigated field in digital health literature. Electronic health records (EHR), for example, can improve quality of care. Emerging analytics tools based on artificial intelligence show the potential to assist physicians in day-to-day workflows. Yet, single health care providers also need information regarding the economic impact when deciding on potential adoption of these tools.

OBJECTIVE

This paper examines the question of whether data and analytics provide economic advantages or disadvantages for health care providers. The goal is to provide a comprehensive overview including a variety of technologies beyond computer-based patient records. Ultimately, findings are also intended to determine whether economic barriers for adoption by providers could exist.

METHODS

A systematic literature search of the PubMed and Google Scholar online databases was conducted, following the hermeneutic methodology that encourages iterative search and interpretation cycles. After applying inclusion and exclusion criteria to 165 initially identified studies, 50 were included for qualitative synthesis and topic-based clustering.

RESULTS

The review identified 5 major technology categories, namely EHRs (n=30), computerized clinical decision support (n=8), advanced analytics (n=5), business analytics (n=5), and telemedicine (n=2). Overall, 62% (31/50) of the reviewed studies indicated a positive economic impact for providers either via direct cost or revenue effects or via indirect efficiency or productivity improvements. When differentiating between categories, however, an ambiguous picture emerged for EHR, whereas analytics technologies like computerized clinical decision support and advanced analytics predominantly showed economic benefits.

CONCLUSIONS

The research question of whether data and analytics create economic benefits for health care providers cannot be answered uniformly. The results indicate ambiguous effects for EHRs, here representing data, and mainly positive effects for the significantly less studied analytics field. The mixed results regarding EHRs can create an economic barrier for adoption by providers. This barrier can translate into a bottleneck to positive economic effects of analytics technologies relying on EHR data. Ultimately, more research on economic effects of technologies other than EHRs is needed to generate a more reliable evidence base.

The Potential of Clinical Decision Support Systems for Prevention, Diagnosis, and Monitoring of Allergic Diseases

Clinical decision support systems (CDSS) aid health care professionals (HCP) in evaluating large sets of information and taking informed decisions during their clinical routine. CDSS are becoming particularly important in the perspective of precision medicine, when HCP need to consider growing amounts of data to create precise patient profiles for personalized diagnosis, treatment and outcome monitoring. In allergy care, several CDSS are being developed and investigated, mainly for respiratory allergic diseases. Although the proposed solutions address different stakeholders, the majority aims at facilitating evidence-based and shared decision-making, incorporating guidelines, and real-time clinical data. We offer here an overview on existing tools, new developments and novel concepts and discuss the potential of digital CDSS in improving prevention, diagnosis and monitoring of allergic diseases.

Triage of patients with venous and lymphatic diseases during the COVID-19 pandemic - The Venous and Lymphatic Triage and Acuity Scale (VELTAS) : A consensus document of the International Union of Phlebology (UIP), Australasian College of Phlebology (ACP), American Vein and Lymphatic Society (AVLS), American Venous Forum (AVF), European College of Phlebology (ECoP), European Venous Forum (EVF), Interventional Radiology Society of Australasia (IRSA), Latin American Venous Forum, Pan-American Society of Phlebology and Lymphology and the Venous Association of India (VAI) This consensus document has been co-published in Phlebology [DOI: 10.1177/0268355520930884] and Journal of Vascular Surgery: Venous and Lymphatic Disorders [DOI: 10.1016/j.jvsv.2020.05.002]. The publications are identical except for minor stylistic and spelling differences in keeping with each journal's style. The contribution has been published under a Attribution-Non Commercial-No Derivatives 4.0 International (CC BY-NC-ND 4.0), (https://creativecommons.org/licenses/by-nc-nd/4.0/)

The coronavirus disease 2019 (COVID-19) global pandemic has resulted in diversion of healthcare resources to the management of patients infected with SARS-CoV-2 virus. Elective interventions and surgical procedures in most countries have been postponed and operating room resources have been diverted to manage the pandemic. The Venous and Lymphatic Triage and Acuity Scale was developed to provide an international standard to rationalise and harmonise the management of patients with venous and lymphatic disorders or vascular anomalies. Triage urgency was determined based on clinical assessment of urgency with which a patient would require medical treatment or surgical intervention. Clinical conditions were classified into six categories of: (1) venous thromboembolism (VTE), (2) chronic venous disease, (3) vascular anomalies, (4) venous trauma, (5) venous compression and (6) lymphatic disease. Triage urgency was categorised into four groups and individual conditions were allocated to each class of triage. These included (1) medical emergencies (requiring immediate attendance), example massive pulmonary embolism; (2) urgent (to be seen as soon as possible), example deep vein thrombosis; (3) semi-urgent (to be attended to within 30-90 days), example highly symptomatic chronic venous disease, and (4) discretionary/non-urgent- (to be seen within 6-12 months), example chronic lymphoedema. Venous and Lymphatic Triage and Acuity Scale aims to standardise the triage of patients with venous and lymphatic disease or vascular anomalies by providing an international consensus-based classification of clinical categories and triage urgency. The scale may be used during pandemics such as the current COVID-19 crisis but may also be used as a general framework to classify urgency of the listed conditions.

Managing COVID-19 With a Clinical Decision Support Tool in a Community Health Network: Algorithm Development and Validation

BACKGROUND

The coronavirus disease (COVID-19) pandemic has resulted in significant morbidity and mortality; large numbers of patients require intensive care, which is placing strain on health care systems worldwide. There is an urgent need for a COVID-19 disease severity assessment that can assist in patient triage and resource allocation for patients at risk for severe disease.

OBJECTIVE

The goal of this study was to develop, validate, and scale a clinical decision support system and mobile app to assist in COVID-19 severity assessment, management, and care.

METHODS

Model training data from 701 patients with COVID-19 were collected across practices within the Family Health Centers network at New York University Langone Health. A two-tiered model was developed. Tier 1 uses easily available, nonlaboratory data to help determine whether biomarker-based testing and/or hospitalization is necessary. Tier 2 predicts the probability of mortality using biomarker measurements (C-reactive protein, procalcitonin, D-dimer) and age. Both the Tier 1 and Tier 2 models were validated using two external datasets from hospitals in Wuhan, China, comprising 160 and 375 patients, respectively.

RESULTS

All biomarkers were measured at significantly higher levels in patients who died vs those who were not hospitalized or discharged (P<.001). The Tier 1 and Tier 2 internal validations had areas under the curve (AUCs) of 0.79 (95% CI 0.74-0.84) and 0.95 (95% CI 0.92-0.98), respectively. The Tier 1 and Tier 2 external validations had AUCs of 0.79 (95% CI 0.74-0.84) and 0.97 (95% CI 0.95-0.99), respectively.

CONCLUSIONS

Our results demonstrate the validity of the clinical decision support system and mobile app, which are now ready to assist health care providers in making evidence-based decisions when managing COVID-19 patient care. The deployment of these new capabilities has potential for immediate impact in community clinics and sites, where application of these tools could lead to improvements in patient outcomes and cost containment.

Developing a Clinical Decision Support Tool for Appropriate Antibiotic Prescribing in Australian General Practice: A Simulation Study

Background. Inappropriate antibiotic prescribing can lead to antimicrobial resistance and drug side effects. Tools that assist general practitioners (GPs) in prescribing decisions may help to optimize prescribing. The aim of this study was to explore the use, acceptability, and feasibility of a clinical decision support (CDS) tool that incorporates evidence-based guidelines and consumer information that integrates with the electronic medical record (EMR). Methods. Eight GPs completed an interview and brief survey after participating in 2 simulated consultations. The survey consisted of demographic questions, perception of realism and representativeness of consultations, Post-Study System Usability Questionnaire, and System Usability Scale. Qualitative data were analyzed using framework analysis. Video data were reviewed, with length of consultation and time spent using the CDS tool documented. Results. Survey responses indicated that all GPs thought the consultations were "real" and representative of real-life consultations; 7 of 8 GPs were satisfied with usability of the tool. Key qualitative findings included that the tool assisted with clinical decision making and informed appropriate antibiotic prescribing. Accessibility and ease of use, including content (guideline and patient education resources), layout, and format, were key factors that determined whether GPs said that they would access the tool in everyday practice. Integration of the tool at multiple sites within the EMR facilitated access to guidelines and assisted in ensuring that the tool fit the clinical workflow. Conclusion. Our CDS tool was acceptable to GPs. Key features required for the tool were easy navigation, clear and useful guideline content, ability to fit into the clinical workflow, and incorporation into the EMR. Piloting of the tool in general practices to assess the impact and feasibility of use in real-world consultations will now be undertaken.

National Trends in the Safety Performance of Electronic Health Record Systems From 2009 to 2018

IMPORTANCE

Despite the broad adoption of electronic health record (EHR) systems across the continuum of care, safety problems persist.

OBJECTIVE

To measure the safety performance of operational EHRs in hospitals across the country during a 10-year period.

DESIGN, SETTING, AND PARTICIPANTS

This case series included all US adult hospitals nationwide that used the National Quality Forum Health IT Safety Measure EHR computerized physician order entry safety test administered by the Leapfrog Group between 2009 and 2018. Data were analyzed from July 1, 2018 to December 1, 2019.

EXPOSURE

The Health IT Safety Measure test, which uses simulated medication orders that have either injured or killed patients previously to evaluate how well hospital EHRs could identify medication errors with potential for patient harm.

MAIN OUTCOMES AND MEASURES

Descriptive statistics for performance on the assessment test over time were calculated at the overall test score level, type of decision support category level, and EHR vendor level.

RESULTS

Among 8657 hospital-years observed during the study, mean (SD) scores on the overall test increased from 53.9% (18.3%) in 2009 to 65.6% (15.4%) in 2018. Mean (SD) hospital score for the categories representing basic clinical decision support increased from 69.8% (20.8%) in 2009 to 85.6% (14.9%) in 2018. For the categories representing advanced clinical decision support, the mean (SD) score increased from 29.6% (22.4%) in 2009 to 46.1% (21.6%) in 2018. There was considerable variation in test performance by EHR.

CONCLUSIONS AND RELEVANCE

These findings suggest that despite broad adoption and optimization of EHR systems in hospitals, wide variation in the safety performance of operational EHR systems remains across a large sample of hospitals and EHR vendors. Hospitals using some EHR vendors had significantly higher test scores. Overall, substantial safety risk persists in current hospital EHR systems.

Development, Implementation, and Evaluation of a Personalized Machine Learning Algorithm for Clinical Decision Support: Case Study With Shingles Vaccination

BACKGROUND

Although clinical decision support (CDS) alerts are effective reminders of best practices, their effectiveness is blunted by clinicians who fail to respond to an overabundance of inappropriate alerts. An electronic health record (EHR)-integrated machine learning (ML) algorithm is a potentially powerful tool to increase the signal-to-noise ratio of CDS alerts and positively impact the clinician's interaction with these alerts in general.

OBJECTIVE

This study aimed to describe the development and implementation of an ML-based signal-to-noise optimization system (SmartCDS) to increase the signal of alerts by decreasing the volume of low-value herpes zoster (shingles) vaccination alerts.

METHODS

We built and deployed SmartCDS, which builds personalized user activity profiles to suppress shingles vaccination alerts unlikely to yield a clinician's interaction. We extracted all records of shingles alerts from January 2017 to March 2019 from our EHR system, including 327,737 encounters, 780 providers, and 144,438 patients.

RESULTS

During the 6 weeks of pilot deployment, the SmartCDS system suppressed an average of 43.67% (15,425/35,315) potential shingles alerts (appointments) and maintained stable counts of weekly shingles vaccination orders (326.3 with system active vs 331.3 in the control group; P=.38) and weekly user-alert interactions (1118.3 with system active vs 1166.3 in the control group; P=.20).

CONCLUSIONS

All key statistics remained stable while the system was turned on. Although the results are promising, the characteristics of the system can be subject to future data shifts, which require automated logging and monitoring. We demonstrated that an automated, ML-based method and data architecture to suppress alerts are feasible without detriment to overall order rates. This work is the first alert suppression ML-based model deployed in practice and serves as foundational work in encounter-level customization of alert display to maximize effectiveness.

The impact of a computerised decision support system on antibiotic usage in an English hospital

Background Antimicrobial resistance is correlated with the inappropriate use of antibiotics. Computerised decision support systems may help practitioners to make evidence-based decisions when prescribing antibiotics. Objective This study aimed to evaluate the impact of computerized decision support systems on the volume of antibiotics used. Setting A very large 1200-bed teaching hospital in Birmingham, England. Main outcome measure The primary outcome measure was the defined daily doses/1000 occupied bed-days. Method A retrospective longitudinal study was conducted to examine the impact of computerised decision support systems on the volume of antibiotic use. The study compared two periods: one with computerised decision support systems, which lasted for 2 years versus one without which lasted for 2 years after the withdrawal of computerised decision support systems. Antibiotic use data from June 2012 to June 2016 were analysed (comprising 2 years with computerised decision support systems immediately followed by 2 years where computerised decision support systems had been withdrawn). Regression analysis was applied to assess the change in antibiotic consumption through the period of the study. Result From June 2012 to June 2016, total antibiotic usage increased by 13.1% from 1436 to 1625 defined daily doses/1000 bed-days: this trend of increased antibiotic prescribing was more pronounced following the withdrawal of structured prescribing (computerised decision support systems). There was a difference of means of - 110.14 defined daily doses/1000 bed days of the total usage of antibiotics in the period with and without structured prescribing, and this was statistically significant (p = 0.026). From June 2012 to June 2016, the dominant antibiotic class used was penicillins. The trends for the total consumption of all antibiotics demonstrated an increase of use for all antibiotic classes except for tetracyclines, quinolones, and anti-mycobacterial drugs, whereas aminoglycoside usage remained stable. Conclusion The implementation of computerised decision support systems appears to influence the use of antibiotics by reducing their consumption. Further research is required to determine the specific features of computerised decision support systems, which influence increased higher adoption and uptake of this technology.

Standards-Based Clinical Decision Support Platform to Manage Patients Who Meet Guideline-Based Criteria for Genetic Evaluation of Familial Cancer

PURPOSE

The ubiquitous adoption of electronic health records (EHRs) with family health history (FHH) data provides opportunities for tailoring cancer screening strategies to individuals. We aimed to enable a standards-based clinical decision support (CDS) platform for identifying and managing patients who meet guidelines for genetic evaluation of hereditary cancer.

METHODS

The CDS platform (www.opencds.org) was used to implement algorithms based on the 2018 National Comprehensive Cancer Network guidelines for genetic evaluation of hereditary breast/ovarian and colorectal cancer. The platform was designed to be interfaced with different EHR systems via the Health Level Seven International Fast Healthcare Interoperability Resources standard. The platform was integrated with the Epic EHR and evaluated in a pilot study at an academic health care system.

RESULTS

The CDS platform was executed against a target population of 143,012 patients; 5,245 (3.7%) met criteria for genetic evaluation based on the FHH recorded in the EHR. In a clinical pilot study, genetic counselors attempted to reach out to 71 of the patients. Of those patients, 25 (35%) scheduled an appointment, 10 (14%) declined, 2 (3%) did not need genetic counseling, 7 (10%) said they would consider it in the future, and 27 (38%) were unreachable. To date, 13 (52%) of the scheduled patients completed visits, and 2 (15%) of those were found to have pathogenic variants in cancer predisposition genes.

CONCLUSION

A standards-based CDS platform integrated with EHR systems is a promising population-based approach to identify patients who are appropriate candidates for genetic evaluation of hereditary cancers.

The NASSS framework for ex post theorisation of technology-supported change in healthcare: worked example of the TORPEDO programme

BACKGROUND

Evaluation of health technology programmes should be theoretically informed, interdisciplinary, and generate in-depth explanations. The NASSS (non-adoption, abandonment, scale-up, spread, sustainability) framework was developed to study unfolding technology programmes in real time-and in particular to identify and manage their emergent uncertainties and interdependencies. In this paper, we offer a worked example of how NASSS can also inform ex post (i.e. retrospective) evaluation.

METHODS

We studied the TORPEDO (Treatment of Cardiovascular Risk in Primary Care using Electronic Decision Support) research programme, a multi-faceted computerised quality improvement intervention for cardiovascular disease prevention in Australian general practice. The technology (HealthTracker) had shown promise in a cluster randomised controlled trial (RCT), but its uptake and sustainability in a real-world implementation phase was patchy. To explain this variation, we used NASSS to undertake secondary analysis of the multi-modal TORPEDO dataset (results and process evaluation of the RCT, survey responses, in-depth professional interviews, videotaped consultations) as well as a sample of new, in-depth narrative interviews with TORPEDO researchers.

RESULTS

Ex post analysis revealed multiple areas of complexity whose influence and interdependencies helped explain the wide variation in uptake and sustained use of the HealthTracker technology: the nature of cardiovascular risk in different populations, the material properties and functionality of the technology, how value (financial and non-financial) was distributed across stakeholders in the system, clinicians' experiences and concerns, organisational preconditions and challenges, extra-organisational influences (e.g. policy incentives), and how interactions between all these influences unfolded over time.

CONCLUSION

The NASSS framework can be applied retrospectively to generate a rich, contextualised narrative of technology-supported change efforts and the numerous interacting influences that help explain its successes, failures, and unexpected events. A NASSS-informed ex post analysis can supplement earlier, contemporaneous evaluations to uncover factors that were not apparent or predictable at the time but dynamic and emergent.

Fog Computing Employed Computer Aided Cancer Classification System Using Deep Neural Network in Internet of Things Based Healthcare System

Computer assisted automatic smart pattern analysis of cancer affected pixel structure takes critical role in pre-interventional decision making for oral cancer treatment. Internet of Things (IoT) in healthcare systems is now emerging solution for modern e-healthcare system to provide high quality medical care. In this research work, we proposed a novel method which utilizes a modified vesselness measurement and a Deep Convolutional Neural Network (DCNN) to identify the oral cancer region structure in IoT based smart healthcare system. The robust vesselness filtering scheme handles noise while reserving small structures, while the CNN framework considerably improves classification accuracy by deblurring focused region of interest (ROI) through integrating with multi-dimensional information from feature vector selection step. The marked feature vector points are extracted from each connected component in the region and used as input for training the CNN. During classification, each connected part is individually analysed using the trained DCNN by considering the feature vector values that belong to its region. For a training of 1500 image dataset, an accuracy of 96.8% and sensitivity of 92% is obtained. Hence, the results of this work validate that the proposed algorithm is effective and accurate in terms of classification of oral cancer region in accurate decision making. The developed system can be used in IoT based diagnosis in health care systems, where accuracy and real time diagnosis are essential.

Reducing waste: a guidelines-based approach to reducing inappropriate vitamin D and TSH testing in the inpatient rehabilitation setting

Background

Laboratory overutilisation increases healthcare costs, and can lead to overdiagnosis, overtreatment and negative health outcomes. Discipline-specific guidelines do not support routine testing for Vitamin D and thyroid-stimulating hormone (TSH) in the inpatient rehabilitation setting, yet 94% of patients had Vitamin D and TSH tests on admission to inpatient rehabilitation at our institution. Our objective was to reduce Vitamin D and TSH testing by 25% on admission to inpatient Stroke, Spinal Cord Injury, Acquired Brain Injury and Amputee Rehabilitation units.

Methods

A fishbone framework for root cause analysis revealed potential causes underlying overutilisation of Vitamin D and TSH testing. A series of Plan-Do-Study-Act (PDSA) cycles were introduced to target remediable factors, starting with an academic detailing intervention with key stakeholders that reviewed applicable clinical guidelines for each patient care discipline and the rationale for reducing admission testing. Simultaneously, computerised clinical decision support (CCDS) limited Vitamin D testing to specific criteria. Audit and feedback were used in a subsequent PDSA cycle. Frequency of Vitamin D and TSH testing on admission was the primary outcome measure. The number of electronic admission order caresets containing automatic Vitamin D and/or TSH orders before and after the interventions was the process measure. Rate of Vitamin D supplementation and changes in thyroid-related medication were the balancing measures.

Results

After implementation, 2.9% of patients had admission Vitamin D testing (97% relative reduction) and 53% of patients had admission TSH testing (43% relative reduction). Admission order caresets with prepopulated Vitamin D and TSH orders decreased from 100% (n=6) to 0%. The interventions were successful; similar to previous literature, CCDS was more effective than education and audit and feedback interventions alone. The interventions represent >$9000 annualised savings.

CADDIE2-evaluation of a clinical decision-support system for early detection of systemic inflammatory response syndrome in paediatric intensive care: study protocol for a diagnostic study

INTRODUCTION

Systemic inflammatory response syndrome (SIRS) is one of the most critical indicators determining the clinical outcome of paediatric intensive care patients. Clinical decision support systems (CDSS) can be designed to support clinicians in detection and treatment. However, the use of such systems is highly discussed as they are often associated with accuracy problems and 'alert fatigue'. We designed a CDSS for detection of paediatric SIRS and hypothesise that a high diagnostic accuracy together with an adequate alerting will accelerate the use. Our study will (1) determine the diagnostic accuracy of the CDSS compared with gold standard decisions created by two blinded, experienced paediatricians, and (2) compare the system's diagnostic accuracy with that of routine clinical care decisions compared with the same gold standard.

METHODS AND ANALYSIS

CADDIE2 is a prospective diagnostic accuracy study taking place at the Department of Pediatric Cardiology and Intensive Care Medicine at the Hannover Medical School; it represents the second step towards our vision of cross-institutional and data-driven decision-support for intensive care environments (CADDIE). The study comprises (1) recruitment of up to 300 patients (start date 1 August 2018), (2) creation of gold standard decisions (start date 1 May 2019), (3) routine SIRS assessments by physicians (starts with recruitment), (4) SIRS assessments by a CDSS (start date 1 May 2019), and (5) statistical analysis with a modified approach for determining sensitivity and specificity and comparing the accuracy results of the different diagnostic approaches (planned start date 1 July 2019).

ETHICS AND DISSEMINATION

Ethics approval was obtained at the study centre (Ethics Committee of Hannover Medical School). Results of the main study will be communicated via publication in a peer-reviewed journal.

TRIAL REGISTRATION NUMBER

ClinicalTrials.gov NCT03661450; Pre-results.

The Safer Prescription of Opioids Tool (SPOT): A Novel Clinical Decision Support Digital Health Platform for Opioid Conversion in Palliative and End of Life Care-A Single-Centre Pilot Study

Opioid errors are a leading cause of patient harm. Active failures in opioid dose conversion can contribute to error. Conversion is complex and is currently performed manually using tables of approximate equivalence. Apps that offer opioid dose double-checking are available but there are concerns about their accuracy and clinical validation. This study evaluated a novel opioid dose conversion app, The Safer Prescription of Opioids Tool (SPOT), a CE-marked Class I medical device, as a clinician decision support (CDS) platform. This single-centre prospective clinical utility pilot study followed a mixed methods design. Prescribers completed an initial survey exploring their current opioid prescribing practice. Thereafter prescribers used SPOT for opioid dosage conversions in parallel to their usual clinical practice, then evaluated SPOT through a survey and focus group. SPOT matched the Gold Standard result in 258 of 268 (96.3%) calculations. The 10 instances (3.7%) when SPOT did not match were due to a rounding error. Users had a statistically significant increase in confidence in prescribing opioids after using SPOT. Focus group feedback highlighted benefits in Quality Improvement and Safety when using SPOT. SPOT is a safe, reliable and validated CDS that has potential to reduce harms from opioid dosing errors.

Using Mobile Health to Support Clinical Decision-Making to Improve Maternal and Neonatal Health Outcomes in Ghana: Insights of Frontline Health Worker Information Needs

BACKGROUND

Developing and maintaining resilient health systems in low-resource settings like Ghana requires innovative approaches that adapt technology to context to improve health outcomes. One such innovation was a mobile health (mHealth) clinical decision-making support system (mCDMSS) that utilized text messaging (short message service, SMS) of standard emergency maternal and neonatal protocols via an unstructured supplementary service data (USSD) on request of the health care providers. This mCDMSS was implemented in a cluster randomized controlled trial (CRCT) in the Eastern Region of Ghana.

OBJECTIVE

This study aimed to analyze the pattern of requests made to the USSD by health workers (HWs). We assessed the relationship between requests made to the USSD and types of maternal and neonatal morbidities reported in health facilities (HFs).

METHODS

For clusters in the intervention arm of the CRCT, all requests to the USSD during the 18-month intervention period were extracted from a remote server, and maternal and neonatal health outcomes of interest were obtained from the District Health Information System of Ghana. Chi-square and Fisher exact tests were used to compare the proportion and type of requests made to the USSD by cluster, facility type, and location; whether phones accessing the intervention were shared facility phones or individual-use phones (type-of-phone); or whether protocols were accessed during the day or at night (time-of-day). Trends in requests made were analyzed over 3 6-month periods. The relationship between requests made and the number of cases reported in HFs was assessed using Spearman correlation.

RESULTS

In total, 5329 requests from 72 (97%) participating HFs were made to the intervention. The average number of requests made per cluster was 667. Requests declined from the first to the third 6-month period (44.96% [2396/5329], 39.82% [2122/5329], and 15.22% [811/5329], respectively). Maternal conditions accounted for the majority of requests made (66.35% [3536/5329]). The most frequently accessed maternal conditions were postpartum hemorrhage (25.23% [892/3536]), other conditions (17.82% [630/3536]), and hypertension (16.49% [583/3536]), whereas the most frequently accessed neonatal conditions were prematurity (20.08% [360/1793]), sepsis (15.45% [277/1793]), and resuscitation (13.78% [247/1793]). Requests made to the mCDMSS varied significantly by cluster, type of request (maternal or neonatal), facility type and its location, type-of-phone, and time-of-day at 6-month interval (P<.001 for each variable). Trends in maternal and neonatal requests showed varying significance over each 6-month interval. Only asphyxia and sepsis cases showed significant correlations with the number of requests made (r=0.44 and r=0.79; P<.001 and P=.03, respectively).

CONCLUSIONS

There were variations in the pattern of requests made to the mCDMSS over time. Detailed information regarding the use of the mCDMSS provides insight into the information needs of HWs for decision-making and an opportunity to focus support for HW training and ultimately improved maternal and neonatal health.

Implementing a digital clinical decision support tool for side effects of antipsychotics: a focus group study

BACKGROUND

In medicine, algorithms can inform treatment decisions by combining the most up-to-date evidence about side effect profiles of medications, which are comparable in efficacy. Their use provides opportunities for improved shared clinician-patient decision-making when initiating therapy. We designed a decision support tool (DST) that incorporated the latest evidence regarding antipsychotic side effects. The tool allowed patients to select one side effect commonly associated with antipsychotics that they wished to avoid; the tool then provided a list of suggested medications and ones to avoid.

OBJECTIVE

To explore qualitatively the acceptability and usefulness of the DST from the perspectives of patients and psychiatrists.

METHODS

This qualitative study took place at a mental health and community hospital in Oxford, UK, in 2018. Four patients/carers and four psychiatrists were recruited to two focus groups to explore their perceptions of the tool. Data were thematically analysed.

FINDINGS

Findings demonstrated a high degree of acceptability and potential usability of the DST for patients and psychiatrists. The main themes to emerge relating to the DST were 'prescribing preferences and practices', 'consideration and awareness of side effects', 'app content, layout and accessibility', 'influence on clinical practice' and 'role in decision-making'.

CONCLUSIONS

A proof-of-concept clinical study will incorporate the recommendations produced from the findings into the tool's design.

CLINICAL IMPLICATIONS

Digital DSTs provide opportunities for the most up-to-date information on medication side effects to be used as the basis for shared clinician-patient decision-making. This tool has the potential to improve adherence to psychiatric medication, with benefits to clinical outcomes and healthcare resourcing.

Optimizing Drug-Drug Interaction Alerts Using a Multidimensional Approach

OBJECTIVES

Excessive alerts are a common concern associated with clinical decision support systems that monitor drug-drug interactions (DDIs). To reduce the number of low-value interruptive DDI alerts at our hospital, we implemented an iterative, multidimensional quality improvement effort, which included an interdisciplinary advisory group, alert metrics, and measurement of perceived clinical value.

METHODS

Alert data analysis indicated that DDIs were the most common interruptive medication alert. An interdisciplinary alert advisory group was formed to provide expert advice and oversight for alert refinement and ongoing review of alert data. Alert data were categorized into drug classes and analyzed to identify DDI alerts for refinement. Refinement strategies included alert suppression and modification of alerts to be contextually aware.

RESULTS

On the basis of historical analysis of classified DDI alerts, 26 alert refinements were implemented, representing 47% of all alerts. Alert refinement efforts resulted in the following substantial decreases in the number of interruptive DDI alerts: 40% for all clinicians (22.9-14 per 100 orders) and as high as 82% for attending physicians (6.5-1.2 per 100 orders). Two patient safety events related to alert refinements were reported during the project period.

CONCLUSIONS

Our quality improvement effort refined 47% of all DDI alerts that were firing during historical analysis, significantly reduced the number of DDI alerts in a 54-week period, and established a model for sustained alert refinements.

Value-based medicine in oncology: the importance of perspective in the emerging value frameworks

Recently, professional and healthcare-related entities have launched frameworks designed to assess the value of cancer innovations in multistakeholder decision processes. Among the most visible entities that propose and implement value frameworks in oncology are the European Society of Medical Oncology (ESMO), the American Society of Clinical Oncology (ASCO), the Memorial Sloan Kettering Cancer Center (MSKCC) and the National Comprehensive Cancer Network (NCCN). However, these value frameworks have been criticized for conceptual inconsistencies, inability to include a greater variety of value criteria, and inadequate explanation of the uncertainty approach used in the modeling process. On the other hand, Multi-Criteria Decision Analysis (MCDA) is a set of methods and processes that allow the multiple criteria involved in a decision to be explicitly addressed. This approach allows the identification of relevant decision criteria, gathering of evidence based on scientific literature, attribution of weights to the criteria and scores to the evidence raised, and aggregation of the weighted scores to constitute a global metric of value. The purpose of this article is to review the main features of these value frameworks in oncology and the importance of perspective for framework readiness to support healthcare decision-making based on MCDA methodology.

Development and validation of a novel computer-aided score to predict the risk of in-hospital mortality for acutely ill medical admissions in two acute hospitals using their first electronically recorded blood test results and vital signs: a cross-sectional study

OBJECTIVES

There are no established mortality risk equations specifically for emergency medical patients who are admitted to a general hospital ward. Such risk equations may be useful in supporting the clinical decision-making process. We aim to develop and externally validate a computer-aided risk of mortality (CARM) score by combining the first electronically recorded vital signs and blood test results for emergency medical admissions.

DESIGN

Logistic regression model development and external validation study.

SETTING

Two acute hospitals (Northern Lincolnshire and Goole NHS Foundation Trust Hospital (NH)-model development data; York Hospital (YH)-external validation data).

PARTICIPANTS

Adult (aged ≥16 years) medical admissions discharged over a 24-month period with electronic National Early Warning Score(s) and blood test results recorded on admission.

RESULTS

The risk of in-hospital mortality following emergency medical admission was 5.7% (NH: 1766/30 996) and 6.5% (YH: 1703/26 247). The C-statistic for the CARM score in NH was 0.87 (95% CI 0.86 to 0.88) and was similar in an external hospital setting YH (0.86, 95% CI 0.85 to 0.87) and the calibration slope included 1 (0.97, 95% CI 0.94 to 1.00).

CONCLUSIONS

We have developed a novel, externally validated CARM score with good performance characteristics for estimating the risk of in-hospital mortality following an emergency medical admission using the patient's first, electronically recorded, vital signs and blood test results. Since the CARM score places no additional data collection burden on clinicians and is readily automated, it may now be carefully introduced and evaluated in hospitals with sufficient informatics infrastructure.

Clinical Pharmacology: A Comprehensive Drug Reference

This column discusses the point-of-care tool Clinical Pharmacology. This review is primarily intended for newer health sciences librarians who are learning about drug references and clinical decision-making support systems or health sciences librarians making collection development decisions, although any librarian will find this review useful. A sample search will be provided to highlight the database's unique features as well as a comparison to other resources.

Using dynamic treatment regimes to understand erythropoietin-stimulating agent hyporesponsiveness

BACKGROUND

Erythropoietin-stimulating agent hyporesponsiveness (ESAH) is associated with increased cardiovascular mortality in patients with end-stage renal disease (ESRD) on hemodialysis. Dynamic treatment regimes (DTR), a clinical decision support (CDS) tool that guides the prescription of specific therapies in response to variations in patient states, have been used to guide treatment for chronic illnesses that require frequent monitoring and therapy changes. Our objective is to explore the role of utilizing a DTR to reduce ESAH in pediatric hemodialysis patients.

METHODS

Retrospective analysis of ESRD patients on hemodialysis who received ESAs. Dosing was adjusted using a locally developed protocol designed to target a hemoglobin between 10 and 12 g/dl. Analyzing this protocol as a DTR, we assessed adherence to the protocol over time measuring how the hyporesponse index (ESA dose/hemoglobin value) changed due to varying levels of adherence.

RESULTS

Eighteen patients met study criteria. Median hemoglobin was 11.4 g/dl (range 6.1-15.4), and median weekly ESA dose (darbepoetin-equivalent) was 0.4 mcg/kg/dose (range 0-2.1). Full adherence to the DTR was identified in 266 (71%) of the 4-week periods, with a median average adherence score of 0.80 (range 0.63-0.91). As adherence to the DTR improved, ESAH decreased. During the last 12 weeks, 13 out of 18 patients had lower average ESA/hemoglobin ratio than the first 12 weeks.

CONCLUSIONS

A DTR appears to be well-suited to the treatment of anemia in ESRD and reduces ESAH. Our work shows the potential of DTRs to drive the development and evaluation of clinical practice guidelines.

Protocol for the validation of sensitivity and specificity of the Cow's Milk-related Symptom Score (CoMiSS) against open food challenge in a single-blinded, prospective, multicentre trial in infants

INTRODUCTION

The symptoms of cow's milk protein allergy (CMPA) in infancy can be non-specific which may delay a correct diagnosis and cause adverse clinical outcomes. The diagnosis of non-IgE-mediated CMPA is particularly complex as it involves a 2 to 4 week elimination diet followed by oral food challenge (OFC). The Cow's Milk-related Symptom Score (CoMiSS) is a clinical resource for primary healthcare providers which aims to increase awareness of CMPA symptoms to facilitate an earlier diagnosis. The aim of the present study is to assess if the CoMiSS can be used as a potential diagnostic tool in infants with suspected CMPA.

METHODS AND ANALYSIS

Exclusively formula-fed infants aged 0-6 months presenting with symptoms suggestive of CMPA will be included in this prospective, multicentre trial which will be conducted in 10 centres in China. All infants will commence a 2-week trial of an amino acid-based formula (AAF) while eliminating all cow milk protein from their diets. After the AAF treatment period, infants will undergo an open OFC in hospital with standard cow's milk formula, followed by an open home challenge for another 2 weeks. Clinical symptoms will be documented on standardised symptom scorecards. The CoMiSS will be determined at study entry (CoMiSS 1, before the start of the AAF), after 2 weeks (CoMiSS 2, before the OFC) and after a further period of 2 weeks or when symptoms suggestive of CMPA reappear (CoMiSS 3). Weight and length will be measured at each visit. The difference between CoMiSS 1 and 2 as a predictor of the OFC outcome will also be assessed. The diagnostic accuracy of the baseline CoMiSS will be calculated.

ETHICS AND DISSEMINATION

The study was approved by the Hunan Children's Hospital Medical Ethics Committee, Hunan, China. The findings of this trial will be submitted for publication in a peer-reviewed journal in paediatric nutrition or gastroenterology. Abstracts will be submitted to the relevant national and international conferences.

TRIAL REGISTRATION NUMBER

NCT03004729; Pre-results.

Comparison of antibiotic dosing recommendations for neonatal sepsis from established reference sources

Background Incorrect dosing is the most frequent prescribing error in neonatology, with antibiotics being the most frequently prescribed medicines. Computer physician order entry and clinical decision support systems can create consistency contributing to a reduction of medication errors. Although evidence-based dosing recommendations should be included in such systems, the evidence is not always available and subsequently, dosing recommendations mentioned in guidelines and textbooks are often based on expert opinion. Objective To compare dosage recommendations for antibiotics in neonates with sepsis provided by eight commonly used and well-established international reference sources. Setting An expert team from our Dutch tertiary care neonatal intensive care unit selected eight well-established international reference sources. Method Daily doses of the seven most frequently used antibiotics in the treatment of neonatal sepsis, classified by categories for birth weight and gestational age, were identified from eight well-respected reference sources in neonatology/pediatric infectious diseases. Main outcome measure Standardized average daily dosage. Results A substantial variation in dosage recommendations of antibiotics for neonatal sepsis between the reference sources was shown. Dosage recommendations of ampicillin, ceftazidime, meropenem and vancomycin varied more than recommendations for benzylpenicillin, cefotaxime and gentamicin. One reference source showed a larger variation in dosage recommendations in comparison to the average recommended daily dosage, compared to the other reference sources. Conclusion Antibiotic dosage recommendations for neonates with sepsis can be derived from important reference sources and guidelines. Further exploration to overcome variation in dosage recommendations is necessary to obtain standardized dosage regimens.

Clinical Impact of 21-Gene Recurrence Score Test Within the Veterans Health Administration: Utilization and Receipt of Guideline-Concordant Care

INTRODUCTION

Ensuring guideline-concordant cancer care is a Department of Veterans Affairs (VA) priority, especially as the number of breast cancer patients at VA medical centers (VAMCs) grows. We assessed the utilization and clinical impact of the 21-gene Recurrence Score test, which predicts 10-year risk of breast cancer recurrence and the likelihood of chemotherapy benefit, on veterans newly diagnosed with breast cancer.

PATIENTS AND METHODS

We conducted a retrospective cohort study using 2011-2012 VA Central Cancer Registry, chart review, and laboratory test data. Independent variables assessed included patient and site-of-care characteristics. The outcome of interest was whether newly diagnosed, eligible (node negative, hormone-receptor positive, human epidermal growth factor receptor 2 [HER2] negative) veterans underwent the 21-gene test. We performed descriptive statistics on all patients and multivariate logistic regression to determine associations. We correlated treatments received with test results.

RESULTS

Among 328 eligible veterans, 82 (25%) had the 21-gene test; 100 eligible veterans (30%) sought care at a VAMC where no tests were ordered. Receiving care at a VAMC that had women's health services (odds ratio [OR], 1.84, 95% confidence interval [CI], 1.05-3.22) and having tumor characteristics meeting the National Comprehensive Cancer Network 2010 test criteria (OR, 3.06, 95% CI, 1.69-5.57) were positive predictors of testing; increasing age (OR, 0.93, 95% CI, 0.91-0.96 per year) and fee-based care (OR, 0.46, 95% CI, 0.26-0.82) were negative predictors. The majority of tested patients received guideline-concordant care.

CONCLUSION

Site of care and tumor characteristics were important predictors of test uptake. Facilitating delivery of guideline-concordant cancer care requires improved laboratory informatics and clinical decision support.

Use of an electronic consultation system in primary care: a qualitative interview study

BACKGROUND

The level of demand on primary care continues to increase. Electronic or e-consultations enable patients to consult their GP online and have been promoted as having potential to improve access and efficiency.

AIM

To evaluate whether an e-consultation system improves the ability of practice staff to manage workload and access.

DESIGN AND SETTING

A qualitative interview study in general practices in the West of England that piloted an e-consultation system for 15 months during 2015 and 2016.

METHOD

Practices were purposefully sampled by location and level of e-consultation use. Clinical, administrative, and management staff were recruited at each practice. Interviews were transcribed and analysed thematically.

RESULTS

Twenty-three interviews were carried out across six general practices. Routine e-consultations offered benefits for the practice because they could be completed without direct contact between GP and patient. However, most e-consultations resulted in GPs needing to follow up with a telephone or face-to-face appointment because the e-consultation did not contain sufficient information to inform clinical decision making. This was perceived as adding to the workload and providing some patients with an alternative route into the appointment system. Although this was seen as offering some patient benefit, there appeared to be fewer benefits for the practices.

CONCLUSION

The experiences of the practices in this study demonstrate that the technology, in its current form, fell short of providing an effective platform for clinicians to consult with patients and did not justify their financial investment in the system. The study also highlights the challenges of remote consultations, which lack the facility for real time interactions.

Rheumatologists' guideline adherence in rheumatoid arthritis: a randomised controlled study on electronic decision support, education and feedback

OBJECTIVES

To assess the effects of education, feedback and a computerised decision support system (CDSS) versus education and feedback alone on rheumatologists' rheumatoid arthritis (RA) guideline adherence.

METHODS

A single-centre, randomised controlled pilot study was performed among clinicians (rheumatologists, residents and physician assistants; n=20) working at the study centre, with a 1:1 randomisation of included clinicians. A standardized sum score (SSS) on guideline adherence was used as the primary outcome (patient level). The SSS was calculated from 13 dichotomous indicators on quality of RA monitoring, treatment and follow-up. The randomised controlled design was combined with a before-after design in the control group to assess the effect education and feedback alone.

RESULTS

Twenty clinicians (mean age 44.3±10.9 years; 55% female) and 990 patients (mean age 62 ± 13 years; 69% female; 72% rheumatoid factor and/or anti-CCP positive) were included. Addition of CDSS to education and feedback did not result in significant better quality of RA care than education and feedback alone (SSS difference 0.02; 95%-CI -0.04 to 0.08; p=0.60). However, before/after comparison showed that education and feedback alone resulted in a significant increase in the SSS from 0.58 to 0.64 (difference 0.06; 95%-CI 0.02 to 0.11; p<0.01).

CONCLUSIONS

Our results suggest that CDSS did not have added value with regard to guideline adherence, whereas education and feedback can lead to a small but significant improvement of guideline adherence.

Pharmacogenomics-Based Point-of-Care Clinical Decision Support Significantly Alters Drug Prescribing

Changes in behavior are necessary to apply genomic discoveries to practice. We prospectively studied medication changes made by providers representing eight different medicine specialty clinics whose patients had submitted to preemptive pharmacogenomic genotyping. An institutional clinical decision support (CDS) system provided pharmacogenomic results using traffic light alerts: green = genomically favorable, yellow = genomic caution, red = high risk. The influence of pharmacogenomic alerts on prescribing behaviors was the primary endpoint. In all, 2,279 outpatient encounters were analyzed. Independent of other potential prescribing mediators, medications with high pharmacogenomic risk were changed significantly more often than prescription drugs lacking pharmacogenomic information (odds ratio (OR) = 26.2 (9.0-75.3), P < 0.0001). Medications with cautionary pharmacogenomic information were also changed more frequently (OR = 2.4 (1.7-3.5), P < 0.0001). No pharmacogenomically high-risk medications were prescribed during the entire study when physicians consulted the CDS tool. Pharmacogenomic information improved prescribing in patterns aimed at reducing patient risk, demonstrating that enhanced prescription decision-making is achievable through clinical integration of genomic medicine.

Electronic clinical decision support tool for the evaluation of cardiovascular risk in general practice: A pilot study

BACKGROUND

Cardiovascular disease (CVD) is a major cause of death in Australia. Electronic medical record (EMR)-based clinical decision support (CDS) tools have the potential to support absolute CVD risk (ACVDR) evaluation and management. The objective of this study was to test the acceptability and feasibility of the Treat to Target CVD (T3CVD), an EMR-based CDS tool, for the evaluation of ACVDR in general practice.

METHODS

Five general practitioners (GPs) piloted the T3CVD tool in their clinic. Interviews with the clinicians explored the acceptability and feasibility of the T3CVD tool.

RESULTS

The T3CVD tool was acceptable and, in the small pilot, was shown to have the capacity to support GPs in ACVDR assessment and management, and to encourage patient participation and motivation. Technical and structural factors important to ensure feasibility of the tool were identified.

DISCUSSION

With further development, the T3CVD tool has the potential to improve ACVDR assessment and management in primary care.

Management of Central Venous Access Device-Associated Skin Impairment: An Evidence-Based Algorithm

Patients relying on central venous access devices (CVADs) for treatment are frequently complex. Many have multiple comorbid conditions, including renal impairment, nutritional deficiencies, hematologic disorders, or cancer. These conditions can impair the skin surrounding the CVAD insertion site, resulting in an increased likelihood of skin damage when standard CVAD management practices are employed. Supported by the World Congress of Vascular Access (WoCoVA), developed an evidence- and consensus-based algorithm to improve CVAD-associated skin impairment (CASI) identification and diagnosis, guide clinical decision-making, and improve clinician confidence in managing CASI. A scoping review of relevant literature surrounding CASI management was undertaken March 2014, and results were distributed to an international advisory panel. A CASI algorithm was developed by an international advisory panel of clinicians with expertise in wounds, vascular access, pediatrics, geriatric care, home care, intensive care, infection control and acute care, using a 2-phase, modified Delphi technique. The algorithm focuses on identification and treatment of skin injury, exit site infection, noninfectious exudate, and skin irritation/contact dermatitis. It comprised 3 domains: assessment, skin protection, and patient comfort. External validation of the algorithm was achieved by prospective pre- and posttest design, using clinical scenarios and self-reported clinician confidence (Likert scale), and incorporating algorithm feasibility and face validity endpoints. The CASI algorithm was found to significantly increase participants' confidence in the assessment and management of skin injury (P = .002), skin irritation/contact dermatitis (P = .001), and noninfectious exudate (P < .01). A majority of participants reported the algorithm as easy to understand (24/25; 96%), containing all necessary information (24/25; 96%). Twenty-four of 25 (96%) stated that they would recommend the tool to guide management of CASI.

Answering medical questions at the point of care: a cross-sectional study comparing rapid decisions based on PubMed and Epistemonikos searches with evidence-based recommendations developed with the GRADE approach

INTRODUCTION

Using the best current evidence to inform clinical decisions remains a challenge for clinicians. Given the scarcity of trustworthy clinical practice guidelines providing recommendations to answer clinicians' daily questions, clinical decision support systems (ie, assistance in question identification and answering) emerge as an attractive alternative. The trustworthiness of the recommendations achieved by such systems is unknown.

OBJECTIVE

To evaluate the trustworthiness of a question identification and answering system that delivers timely recommendations.

DESIGN

Cross-sectional study.

METHODS

We compared the responses to 100 clinical questions related to inpatient management provided by two rapid response methods with 'Gold Standard' recommendations. One of the rapid methods was based on PubMed and the other on Epistemonikos database. We defined our 'Gold Standard' as trustworthy published evidence-based recommendations or, when unavailable, recommendations developed locally by a panel of six clinicians following the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach. Recommendations provided by the rapid strategies were classified as potentially misleading or reasonable. We also determined if the potentially misleading recommendations could have been avoided with the appropriate implementation of searching and evidence summary tools.

RESULTS

We were able to answer all of the 100 questions with both rapid methods. Of the 200 recommendations obtained, 6.5% (95% CI 3% to 9.9%) were classified as potentially misleading and 93.5% (95% CI 90% to 96.9%) as reasonable. 6 of the 13 potentially misleading recommendations could have been avoided by the appropriate usage of the Epistemonikos matrix tool or by constructing summary of findings tables. No significant differences were observed between the evaluated rapid response methods.

CONCLUSION

A question answering service based on the GRADE approach proved feasible to implement and provided appropriate guidance for most identified questions. Our approach could help stakeholders in charge of managing resources and defining policies for patient care to improve evidence-based decision-making in an efficient and feasible manner.

A nonparametric Bayesian method of translating machine learning scores to probabilities in clinical decision support

BACKGROUND

Probabilistic assessments of clinical care are essential for quality care. Yet, machine learning, which supports this care process has been limited to categorical results. To maximize its usefulness, it is important to find novel approaches that calibrate the ML output with a likelihood scale. Current state-of-the-art calibration methods are generally accurate and applicable to many ML models, but improved granularity and accuracy of such methods would increase the information available for clinical decision making. This novel non-parametric Bayesian approach is demonstrated on a variety of data sets, including simulated classifier outputs, biomedical data sets from the University of California, Irvine (UCI) Machine Learning Repository, and a clinical data set built to determine suicide risk from the language of emergency department patients.

RESULTS

The method is first demonstrated on support-vector machine (SVM) models, which generally produce well-behaved, well understood scores. The method produces calibrations that are comparable to the state-of-the-art Bayesian Binning in Quantiles (BBQ) method when the SVM models are able to effectively separate cases and controls. However, as the SVM models' ability to discriminate classes decreases, our approach yields more granular and dynamic calibrated probabilities comparing to the BBQ method. Improvements in granularity and range are even more dramatic when the discrimination between the classes is artificially degraded by replacing the SVM model with an ad hoc k-means classifier.

CONCLUSIONS

The method allows both clinicians and patients to have a more nuanced view of the output of an ML model, allowing better decision making. The method is demonstrated on simulated data, various biomedical data sets and a clinical data set, to which diverse ML methods are applied. Trivially extending the method to (non-ML) clinical scores is also discussed.

Healthcare provider education to support integration of pharmacogenomics in practice: the eMERGE Network experience

Ten organizations within the Electronic Medical Records and Genomics Network developed programs to implement pharmacogenomic sequencing and clinical decision support into clinical settings. Recognizing the importance of informed prescribers, a variety of strategies were used to incorporate provider education to support implementation. Education experiences with pharmacogenomics are described within the context of each organization's prior involvement, including the scope and scale of implementation specific to their Electronic Medical Records and Genomics projects. We describe common and distinct education strategies, provide exemplars and share challenges. Lessons learned inform future perspectives. Future pharmacogenomics clinical implementation initiatives need to include funding toward implementing provider education and evaluating outcomes.

Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system

BACKGROUND

Although alert fatigue is blamed for high override rates in contemporary clinical decision support systems, the concept of alert fatigue is poorly defined. We tested hypotheses arising from two possible alert fatigue mechanisms: (A) cognitive overload associated with amount of work, complexity of work, and effort distinguishing informative from uninformative alerts, and (B) desensitization from repeated exposure to the same alert over time.

METHODS

Retrospective cohort study using electronic health record data (both drug alerts and clinical practice reminders) from January 2010 through June 2013 from 112 ambulatory primary care clinicians. The cognitive overload hypotheses were that alert acceptance would be lower with higher workload (number of encounters, number of patients), higher work complexity (patient comorbidity, alerts per encounter), and more alerts low in informational value (repeated alerts for the same patient in the same year). The desensitization hypothesis was that, for newly deployed alerts, acceptance rates would decline after an initial peak.

RESULTS

On average, one-quarter of drug alerts received by a primary care clinician, and one-third of clinical reminders, were repeats for the same patient within the same year. Alert acceptance was associated with work complexity and repeated alerts, but not with the amount of work. Likelihood of reminder acceptance dropped by 30% for each additional reminder received per encounter, and by 10% for each five percentage point increase in proportion of repeated reminders. The newly deployed reminders did not show a pattern of declining response rates over time, which would have been consistent with desensitization. Interestingly, nurse practitioners were 4 times as likely to accept drug alerts as physicians.

CONCLUSIONS

Clinicians became less likely to accept alerts as they received more of them, particularly more repeated alerts. There was no evidence of an effect of workload per se, or of desensitization over time for a newly deployed alert. Reducing within-patient repeats may be a promising target for reducing alert overrides and alert fatigue.

The effect of a clinical decision-making mHealth support system on maternal and neonatal mortality and morbidity in Ghana: study protocol for a cluster randomized controlled trial

BACKGROUND

Mobile health (mHealth) presents one of the potential solutions to maximize health worker impact and efficiency in an effort to reach the Sustainable Development Goals 3.1 and 3.2, particularly in sub-Saharan African countries. Poor-quality clinical decision-making is known to be associated with poor pregnancy and birth outcomes. This study aims to assess the effect of a clinical decision-making support system (CDMSS) directed at frontline health care providers on neonatal and maternal health outcomes.

METHODS/DESIGN

A cluster randomized controlled trial will be conducted in 16 eligible districts (clusters) in the Eastern Region of Ghana to assess the effect of an mHealth CDMSS for maternal and neonatal health care services on maternal and neonatal outcomes. The CDMSS intervention consists of an Unstructured Supplementary Service Data (USSD)-based text messaging of standard emergency obstetric and neonatal protocols to providers on their request. The primary outcome of the intervention is the incidence of institutional neonatal mortality. Outcomes will be assessed through an analysis of data on maternal and neonatal morbidity and mortality extracted from the District Health Information Management System-2 (DHIMS-2) and health facility-based records. The quality of maternal and neonatal health care will be assessed in two purposively selected clusters from each study arm.

DISCUSSION

In this trial the effect of a mobile CDMSS on institutional maternal and neonatal health outcomes will be evaluated to generate evidence-based recommendations for the use of mobile CDMSS in Ghana and other West African countries.

TRIAL REGISTRATION

ClinicalTrials.gov, identifier: NCT02468310 . Registered on 7 September 2015; Pan African Clinical Trials Registry, identifier: PACTR20151200109073 . Registered on 9 December 2015 retrospectively from trial start date.