A task-based support architecture for developing point-of-care clinical decision support systems for the emergency department

Using Theories, Models, and Frameworks to Inform Implementation Cycles of Computerized Clinical Decision Support Systems in Tertiary Health Care Settings: Scoping Review

BACKGROUND

Computerized clinical decision support systems (CDSSs) are essential components of modern health system service delivery, particularly within acute care settings such as hospitals. Theories, models, and frameworks may assist in facilitating the implementation processes associated with CDSS innovation and its use within these care settings. These processes include context assessments to identify key determinants, implementation plans for adoption, promoting ongoing uptake, adherence, and long-term evaluation. However, there has been no prior review synthesizing the literature regarding the theories, models, and frameworks that have informed the implementation and adoption of CDSSs within hospitals.

OBJECTIVE

This scoping review aims to identify the theory, model, and framework approaches that have been used to facilitate the implementation and adoption of CDSSs in tertiary health care settings, including hospitals. The rationales reported for selecting these approaches, including the limitations and strengths, are described.

METHODS

A total of 5 electronic databases were searched (CINAHL via EBSCOhost, PubMed, Scopus, PsycINFO, and Embase) to identify studies that implemented or adopted a CDSS in a tertiary health care setting using an implementation theory, model, or framework. No date or language limits were applied. A narrative synthesis was conducted using full-text publications and abstracts. Implementation phases were classified according to the "Active Implementation Framework stages": exploration (feasibility and organizational readiness), installation (organizational preparation), initial implementation (initiating implementation, ie, training), full implementation (sustainment), and nontranslational effectiveness studies.

RESULTS

A total of 81 records (42 full text and 39 abstracts) were included. Full-text studies and abstracts are reported separately. For full-text studies, models (18/42, 43%), followed by determinants frameworks (14/42,33%), were most frequently used to guide adoption and evaluation strategies. Most studies (36/42, 86%) did not list the limitations associated with applying a specific theory, model, or framework.

CONCLUSIONS

Models and related quality improvement methods were most frequently used to inform CDSS adoption. Models were not typically combined with each other or with theory to inform full-cycle implementation strategies. The findings highlight a gap in the application of implementation methods including theories, models, and frameworks to facilitate full-cycle implementation strategies for hospital CDSSs.

Development of a machine learning-based clinical decision support system to predict clinical deterioration in patients visiting the emergency department

This study aimed to develop a machine learning-based clinical decision support system for emergency departments based on the decision-making framework of physicians. We extracted 27 fixed and 93 observation features using data on vital signs, mental status, laboratory results, and electrocardiograms during emergency department stay. Outcomes included intubation, admission to the intensive care unit, inotrope or vasopressor administration, and in-hospital cardiac arrest. eXtreme gradient boosting algorithm was used to learn and predict each outcome. Specificity, sensitivity, precision, F1 score, area under the receiver operating characteristic curve (AUROC), and area under the precision-recall curve were assessed. We analyzed 303,345 patients with 4,787,121 input data, resampled into 24,148,958 1 h-units. The models displayed a discriminative ability to predict outcomes (AUROC > 0.9), and the model with lagging 6 and leading 0 displayed the highest value. The AUROC curve of in-hospital cardiac arrest had the smallest change, with increased lagging for all outcomes. With inotropic use, intubation, and intensive care unit admission, the range of AUROC curve change with the leading 6 was the highest according to different amounts of previous information (lagging). In this study, a human-centered approach to emulate the clinical decision-making process of emergency physicians has been adopted to enhance the use of the system. Machine learning-based clinical decision support systems customized according to clinical situations can help improve the quality of care.

Ontologies Applied in Clinical Decision Support System Rules: Systematic Review

BACKGROUND

Clinical decision support systems (CDSSs) are important for the quality and safety of health care delivery. Although CDSS rules guide CDSS behavior, they are not routinely shared and reused.

OBJECTIVE

Ontologies have the potential to promote the reuse of CDSS rules. Therefore, we systematically screened the literature to elaborate on the current status of ontologies applied in CDSS rules, such as rule management, which uses captured CDSS rule usage data and user feedback data to tailor CDSS services to be more accurate, and maintenance, which updates CDSS rules. Through this systematic literature review, we aim to identify the frontiers of ontologies used in CDSS rules.

METHODS

The literature search was focused on the intersection of ontologies; clinical decision support; and rules in PubMed, the Association for Computing Machinery (ACM) Digital Library, and the Nursing & Allied Health Database. Grounded theory and PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines were followed. One author initiated the screening and literature review, while 2 authors validated the processes and results independently. The inclusion and exclusion criteria were developed and refined iteratively.

RESULTS

CDSSs were primarily used to manage chronic conditions, alerts for medication prescriptions, reminders for immunizations and preventive services, diagnoses, and treatment recommendations among 81 included publications. The CDSS rules were presented in Semantic Web Rule Language, Jess, or Jena formats. Despite the fact that ontologies have been used to provide medical knowledge, CDSS rules, and terminologies, they have not been used in CDSS rule management or to facilitate the reuse of CDSS rules.

CONCLUSIONS

Ontologies have been used to organize and represent medical knowledge, controlled vocabularies, and the content of CDSS rules. So far, there has been little reuse of CDSS rules. More work is needed to improve the reusability and interoperability of CDSS rules. This review identified and described the ontologies that, despite their limitations, enable Semantic Web technologies and their applications in CDSS rules.

Grading and assessment of clinical predictive tools for paediatric head injury: a new evidence-based approach

BACKGROUND

Many clinical predictive tools have been developed to diagnose traumatic brain injury among children and guide the use of computed tomography in the emergency department. It is not always feasible to compare tools due to the diversity of their development methodologies, clinical variables, target populations, and predictive performances. The objectives of this study are to grade and assess paediatric head injury predictive tools, using a new evidence-based approach, and to provide emergency clinicians with standardised objective information on predictive tools to support their search for and selection of effective tools.

METHODS

Paediatric head injury predictive tools were identified through a focused review of literature. Based on the critical appraisal of published evidence about predictive performance, usability, potential effect, and post-implementation impact, tools were evaluated using a new framework for grading and assessment of predictive tools (GRASP). A comprehensive analysis was conducted to explain why certain tools were more successful.

RESULTS

Fourteen tools were identified and evaluated. The highest-grade tool is PECARN; the only tool evaluated in post-implementation impact studies. PECARN and CHALICE were evaluated for their potential effect on healthcare, while the remaining 12 tools were only evaluated for predictive performance. Three tools; CATCH, NEXUS II, and Palchak, were externally validated. Three tools; Haydel, Atabaki, and Buchanich, were only internally validated. The remaining six tools; Da Dalt, Greenes, Klemetti, Quayle, Dietrich, and Güzel did not show sufficient internal validity for use in clinical practice.

CONCLUSIONS

The GRASP framework provides clinicians with a high-level, evidence-based, comprehensive, yet simple and feasible approach to grade, compare, and select effective predictive tools. Comparing the three main tools which were assigned the highest grades; PECARN, CHALICE and CATCH, to the remaining 11, we find that the quality of tools' development studies, the experience and credibility of their authors, and the support by well-funded research programs were correlated with the tools' evidence-based assigned grades, and were more influential, than the sole high predictive performance, on the wide acceptance and successful implementation of the tools. Tools' simplicity and feasibility, in terms of resources needed, technical requirements, and training, are also crucial factors for their success.

Machine learning in critical care: state-of-the-art and a sepsis case study

BACKGROUND

Like other scientific fields, such as cosmology, high-energy physics, or even the life sciences, medicine and healthcare face the challenge of an extremely quick transformation into data-driven sciences. This challenge entails the daunting task of extracting usable knowledge from these data using algorithmic methods. In the medical context this may for instance realized through the design of medical decision support systems for diagnosis, prognosis and patient management. The intensive care unit (ICU), and by extension the whole area of critical care, is becoming one of the most data-driven clinical environments.

RESULTS

The increasing availability of complex and heterogeneous data at the point of patient attention in critical care environments makes the development of fresh approaches to data analysis almost compulsory. Computational Intelligence (CI) and Machine Learning (ML) methods can provide such approaches and have already shown their usefulness in addressing problems in this context. The current study has a dual goal: it is first a review of the state-of-the-art on the use and application of such methods in the field of critical care. Such review is presented from the viewpoint of the different subfields of critical care, but also from the viewpoint of the different available ML and CI techniques. The second goal is presenting a collection of results that illustrate the breath of possibilities opened by ML and CI methods using a single problem, the investigation of septic shock at the ICU.

CONCLUSION

We have presented a structured state-of-the-art that illustrates the broad-ranging ways in which ML and CI methods can make a difference in problems affecting the manifold areas of critical care. The potential of ML and CI has been illustrated in detail through an example concerning the sepsis pathology. The new definitions of sepsis and the relevance of using the systemic inflammatory response syndrome (SIRS) in its diagnosis have been considered. Conditional independence models have been used to address this problem, showing that SIRS depends on both organ dysfunction measured through the Sequential Organ Failure (SOFA) score and the ICU outcome, thus concluding that SIRS should still be considered in the study of the pathophysiology of Sepsis. Current assessment of the risk of dead at the ICU lacks specificity. ML and CI techniques are shown to improve the assessment using both indicators already in place and other clinical variables that are routinely measured. Kernel methods in particular are shown to provide the best performance balance while being amenable to representation through graphical models, which increases their interpretability and, with it, their likelihood to be accepted in medical practice.

Interplay between Clinical Guidelines and Organizational Workflow Systems

Background: Implementing a decision-support system within a healthcare organization requires integration of clinical domain knowledge with resource constraints. Computer-interpretable guidelines (CIG) are excellent instruments for addressing clinical aspects while business process management (BPM) languages and Workflow (Wf) engines manage the logistic organizational constraints.

Objectives: Our objective is the orchestra -tion of all the relevant factors needed for a successful execution of patient’s care pathways, especially when spanning the contin -uum of care, from acute to community or home care.

Methods: We considered three strategies for integrating CIGs with organizational work-flows: extending the CIG or BPM languages and their engines, or creating an interplay between them. We used the interplay approach to implement a set of use cases arising from a CIG implementation in the domain of Atrial Fibrillation. To provide a more scalable and standards-based solution, we explored the use of Cross-Enterprise Document Workflow Integration Profile.

Results: We describe our proof-of-concept implementation of five use cases. We utilized the Personal Health Record of the MobiGuide project to implement a loosely-coupled approach between the Activiti BPM engine and the Picard CIG engine. Changes in the PHR were detected by polling. IHE profiles were used to develop workflow documents that orchestrate cross-enterprise execution of cardioversion.

Conclusions: Interplay between CIG and BPM engines can support orchestration of care flows within organizational settings.

Comprehensive mitigation framework for concurrent application of multiple clinical practice guidelines

In this work we propose a comprehensive framework based on first-order logic (FOL) for mitigating (identifying and addressing) interactions between multiple clinical practice guidelines (CPGs) applied to a multi-morbid patient while also considering patient preferences related to the prescribed treatment. With this framework we respond to two fundamental challenges associated with clinical decision support: (1) concurrent application of multiple CPGs and (2) incorporation of patient preferences into the decision making process. We significantly expand our earlier research by (1) proposing a revised and improved mitigation-oriented representation of CPGs and secondary medical knowledge for addressing adverse interactions and incorporating patient preferences and (2) introducing a new mitigation algorithm. Specifically, actionable graphs representing CPGs allow for parallel and temporal activities (decisions and actions). Revision operators representing secondary medical knowledge support temporal interactions and complex revisions across multiple actionable graphs. The mitigation algorithm uses the actionable graphs, revision operators and available (and possibly incomplete) patient information represented in FOL. It relies on a depth-first search strategy to find a valid sequence of revisions and uses theorem proving and model finding techniques to identify applicable revision operators and to establish a management scenario for a given patient if one exists. The management scenario defines a safe (interaction-free) and preferred set of activities together with possible patient states. We illustrate the use of our framework with a clinical case study describing two patients who suffer from chronic kidney disease, hypertension, and atrial fibrillation, and who are managed according to CPGs for these diseases. While in this paper we are primarily concerned with the methodological aspects of mitigation, we also briefly discuss a high-level proof of concept implementation of the proposed framework in the form of a clinical decision support system (CDSS). The proposed mitigation CDSS "insulates" clinicians from the complexities of the FOL representations and provides semantically meaningful summaries of mitigation results. Ultimately we plan to implement the mitigation CDSS within our MET (Mobile Emergency Triage) decision support environment.

CURB-65 Performance Among Admitted and Discharged Emergency Department Patients With Community-acquired Pneumonia

OBJECTIVES

Pneumonia severity tools were primarily developed in cohorts of hospitalized patients, limiting their applicability to the emergency department (ED). We describe current community ED admission practices and examine the accuracy of the CURB-65 to predict 30-day mortality for patients, either discharged or admitted with community-acquired pneumonia (CAP).

METHODS

A retrospective, observational study of adult CAP encounters in 14 community EDs within an integrated healthcare system. We calculated CURB-65 scores for all encounters and described the use of hospitalization, stratified by each score (0-5). We then used each score as a cutoff to calculate sensitivity, specificity, positive predictive value, negative predictive value (NPV), positive likelihood ratios, and negative likelihood ratios for predicting 30-day mortality.

RESULTS

The sample included 21,183 ED encounters for CAP (7,952 discharged and 13,231 admitted). The C-statistic describing the accuracy of CURB-65 for predicting 30-day mortality in the full sample was 0.761 (95% confidence interval [CI], 0.747-0.774). The C-statistic was 0.864 (95% CI, 0.821-0.906) among patients discharged from the ED compared with 0.689 (95% CI, 0.672-0.705) among patients who were admitted. Among all ED encounters a CURB-65 threshold of ≥1 was 92.8% sensitive and 38.0% specific for predicting mortality, with a 99.9% NPV. Among all encounters, 62.5% were admitted, including 36.2% of those at lowest risk (CURB-65 = 0).

CONCLUSIONS

CURB-65 had very good accuracy for predicting 30-day mortality among patients discharged from the ED. This severity tool may help ED providers risk stratify patients to assist with disposition decisions and identify unwarranted variation in patient care.

Is There a Consensus when Physicians Evaluate the Relevance of Retrieved Systematic Reviews?

BACKGROUND

A significant challenge associated with practicing evidence-based medicine is to provide physicians with relevant clinical information when it is needed. At the same time it appears that the notion of relevance is subjective and its perception is affected by a number of contextual factors.

OBJECTIVES

To assess to what extent physicians agree on the relevance of evidence in the form of systematic reviews for a common set of patient cases, and to identify possible contextual factors that influence their perception of relevance.

METHODS

A web-based survey was used where pediatric emergency physicians from multiple academic centers across Canada were asked to evaluate the relevance of systematic reviews retrieved automatically for 14 written case vignettes (paper patients). The vignettes were derived from prospective data describing pediatric patients with asthma exacerbations presenting at the emergency department. To limit the cognitive burden on respondents, the number of reviews associated with each vignette was limited to three.

RESULTS

Twenty-two academic emergency physicians with varying years of clinical practice completed the survey. There was no consensus in their evaluation of relevance of the retrieved reviews and physicians' assessments ranged from very relevant to irrelevant evidence, with the majority of evaluations being somewhere in the middle. This indicates that the study participants did not share a notion of relevance uniformly. Further analysis of commentaries provided by the physicians allowed identifying three possible contextual factors: expected specificity of evidence (acute vs chronic condition), the terminology used in the systematic reviews, and the micro environment of clinical setting.

CONCLUSIONS

There is no consensus among physicians with regards to what constitutes relevant clinical evidence for a given patient case. Subsequently, this finding suggests that evidence retrieval systems should allow for deep customization with regards to physician's preferences and contextual factors, including differences in the micro environment of each clinical setting.

eHealth: past and future perspectives

eHealth is an umbrella term incorporating any area that combines healthcare and technology to improve efficiencies and reduce costs. The ultimate goal of eHealth is to rationalize treatment selection to improve patient safety and outcomes. Telemedicine, first used in the 1920s, is the oldest form of eHealth. The introduction of broadband Internet, followed by wireless technologies, has allowed an explosion of mHealth applications within this field. Wearable technologies, such as smartwatches, are now being used for diagnostics and patient monitoring. Challenges remain to develop reusable Clinical Decision Support systems that will streamline the flow of data from clinical laboratories to point of care. This review explores the history of eHealth, and describes some of the remaining integration and implementation challenges.

From bed to bench: bridging from informatics practice to theory: an exploratory analysis

BACKGROUND

In 2009, Applied Clinical Informatics (ACI)--focused on applications in clinical informatics--was launched as a companion journal to Methods of Information in Medicine (MIM). Both journals are official journals of the International Medical Informatics Association.

OBJECTIVES

To explore which congruencies and interdependencies exist in publications from theory to practice and from practice to theory and to determine existing gaps. Major topics discussed in ACI and MIM were analyzed. We explored if the intention of publishing companion journals to provide an information bridge from informatics theory to informatics practice and vice versa could be supported by this model. In this manuscript we will report on congruencies and interdependences from practice to theory and on major topics in MIM.

METHODS

Retrospective, prolective observational study on recent publications of ACI and MIM. All publications of the years 2012 and 2013 were indexed and analyzed.

RESULTS

Hundred and ninety-six publications were analyzed (ACI 87, MIM 109). In MIM publications, modelling aspects as well as methodological and evaluation approaches for the analysis of data, information, and knowledge in biomedicine and health care were frequently raised - and often discussed from an interdisciplinary point of view. Important themes were ambient-assisted living, anatomic spatial relations, biomedical informatics as scientific discipline, boosting, coding, computerized physician order entry, data analysis, grid and cloud computing, health care systems and services, health-enabling technologies, health information search, health information systems, imaging, knowledge-based decision support, patient records, signal analysis, and web science. Congruencies between journals could be found in themes, but with a different focus on content. Interdependencies from practice to theory, found in these publications, were only limited.

CONCLUSIONS

Bridging from informatics theory to practice and vice versa remains a major component of successful research and practice as well as a major challenge.

Comparing predictions made by a prediction model, clinical score, and physicians: pediatric asthma exacerbations in the emergency department

BACKGROUND

Asthma exacerbations are one of the most common medical reasons for children to be brought to the hospital emergency department (ED). Various prediction models have been proposed to support diagnosis of exacerbations and evaluation of their severity.

OBJECTIVES

First, to evaluate prediction models constructed from data using machine learning techniques and to select the best performing model. Second, to compare predictions from the selected model with predictions from the Pediatric Respiratory Assessment Measure (PRAM) score, and predictions made by ED physicians.

DESIGN

A two-phase study conducted in the ED of an academic pediatric hospital. In phase 1 data collected prospectively using paper forms was used to construct and evaluate five prediction models, and the best performing model was selected. In phase 2 data collected prospectively using a mobile system was used to compare the predictions of the selected prediction model with those from PRAM and ED physicians.

MEASUREMENTS

Area under the receiver operating characteristic curve and accuracy in phase 1; accuracy, sensitivity, specificity, positive and negative predictive values in phase 2.

RESULTS

In phase 1 prediction models were derived from a data set of 240 patients and evaluated using 10-fold cross validation. A naive Bayes (NB) model demonstrated the best performance and it was selected for phase 2. Evaluation in phase 2 was conducted on data from 82 patients. Predictions made by the NB model were less accurate than the PRAM score and physicians (accuracy of 70.7%, 73.2% and 78.0% respectively), however, according to McNemar's test it is not possible to conclude that the differences between predictions are statistically significant.

CONCLUSION

Both the PRAM score and the NB model were less accurate than physicians. The NB model can handle incomplete patient data and as such may complement the PRAM score. However, it requires further research to improve its accuracy.