Computer-assisted diagnostic decision support: history, challenges, and possible paths forward

Assessing the Economic Value of Clinical Artificial Intelligence: Challenges and Opportunities

OBJECTIVES

Clinical artificial intelligence (AI) is a novel technology, and few economic evaluations have focused on it to date. Before its wider implementation, it is important to highlight the aspects of AI that challenge traditional health technology assessment methods.

METHODS

We used an existing broad value framework to assess potential ways AI can provide good value for money. We also developed a rubric of how economic evaluations of AI should vary depending on the case of its use.

RESULTS

We found that the measurement of core elements of value-health outcomes and cost-are complicated by AI because its generalizability across different populations is often unclear and because its use may necessitate reconfigured clinical processes. Clinicians' productivity may improve when AI is used. If poorly implemented though, AI may also cause clinicians' workload to increase. Some AI has been found to exacerbate health disparities. Nevertheless, AI may promote equity by expanding access to medical care and, when properly trained, providing unbiased diagnoses and prognoses. The approach to assessment of AI should vary based on its use case: AI that creates new clinical possibilities can improve outcomes, but regulation and evidence collection may be difficult; AI that extends clinical expertise can reduce disparities and lower costs but may result in overuse; and AI that automates clinicians' work can improve productivity but may reduce skills.

CONCLUSIONS

The potential uses of clinical AI create challenges for health technology assessment methods originally developed for pharmaceuticals and medical devices. Health economists should be prepared to examine data collection and methods used to train AI, as these may impact its future value.

A Research Agenda for Diagnostic Excellence in Critical Care Medicine

Diagnosing critically ill patients in the intensive care unit is difficult. As a result, diagnostic errors in the intensive care unit are common and have been shown to cause harm. Research to improve diagnosis in critical care medicine has accelerated in past years. However, much work remains to fully elucidate the diagnostic process in critical care. To achieve diagnostic excellence, interdisciplinary research is needed, adopting a balanced strategy of continued biomedical discovery while addressing the complex care delivery systems underpinning the diagnosis of critical illness.

Are Regulations Safe? Reflections From Developing a Digital Cancer Decision-Support Tool

PURPOSE

Informatics solutions to early diagnosis of cancer in primary care are increasingly prevalent, but it is not clear whether existing and planned standards and regulations sufficiently address patients' safety nor whether these standards are fit for purpose. We use a patient safety perspective to reflect on the development of a computerized cancer risk assessment tool embedded within a UK primary care electronic health record system.

METHODS

We developed a computerized version of the CAncer Prevention in ExetER studies risk assessment tool, in compliance with the European Union's Medical Device Regulations. The process of building this tool afforded an opportunity to reflect on clinical concerns and whether current regulations for medical devices are fit for purpose. We identified concerns for patient safety and developed nine practical recommendations to mitigate these concerns.

RESULTS

We noted that medical device regulations (1) were initially created for hardware devices rather than software, (2) offer one-shot approval rather than supporting iterative innovation and learning, (3) are biased toward loss-transfer approaches that attempt to manage the fallout of harm instead of mitigating hazards becoming harmful, and (4) are biased toward known hazards, despite unknown hazards being an expected consequence of health care as a complex adaptive system. Our nine recommendations focus on embedding less-reductionist and stronger system perspectives into regulations and standards.

CONCLUSION

Our intention is to share our experience to support research-led collaborative development of health informatics solutions in cancer. We argue that regulations in the European Union do not sufficiently address the complexity of healthcare information systems with consequences for patient safety. Future standards and regulations should continue to follow a system-based approach to risk, safety, and accident avoidance.

Isolating red flags to enhance diagnosis (I-RED): An experimental vignette study

OBJECTIVE

To investigate effects of a cognitive intervention based on isolation of red flags (I-RED) on diagnostic accuracy of 'do-not-miss diagnoses.'

DESIGN

A 2 × 2 randomized case vignette-based experiment with manipulation of I-RED strategy between subjects and case complexity within subjects.

SETTING

Two university-based residency programs.

PARTICIPANTS

One-hundred and nine pediatric residents from all levels of training.

INTERVENTIONS

Participants were randomly assigned to the I-RED vs. control group, and within each group, they were further randomized to the order in which they saw simple and complex cases. The I-RED strategy involved an instruction to look for a constellation of symptoms, signs, clinical data or circumstances that should heighten suspicion for a serious condition.

MAIN OUTCOME MEASURES

Primary outcome was diagnostic accuracy, scored as 1 if any of the three differentials given by participants included the correct diagnosis, and 0 if not. We analyzed effects of I-RED strategy on diagnostic accuracy using logistic regression.

RESULTS

I-RED strategy did not yield statistically higher diagnostic accuracy compared to controls (62 vs. 48%, respectively; odd ratio = 2.07 [95% confidence interval, 0.78-5.5], P = 0.14) although participants reported higher decision confidence compared to controls (7.00 vs. 5.77 on a scale of 1 to 10, P < 0.02) in simple but not complex cases. I-RED strategy significantly shortened time to decision (460 vs. 657 s, P < 0.001) and increased the number of red flags generated (3.04 vs. 2.09, P < 0.001).

CONCLUSIONS

A cognitive strategy of prompting red flag isolation prior to differential diagnosis did not improve diagnostic accuracy of 'do-not-miss diagnoses.' Given the paucity of evidence-based solutions to reduce diagnostic error and the intervention's potential effect on confidence, findings warrant additional exploration.

Exploring healthcare professionals' understanding and experiences of artificial intelligence technology use in the delivery of healthcare: An integrative review

BACKGROUND

The integration of artificial intelligence (AI) into our digital healthcare system is seen as a significant strategy to contain Australia's rising healthcare costs, support clinical decision making, manage chronic disease burden and support our ageing population. With the increasing roll-out of 'digital hospitals', electronic medical records, new data capture and analysis technologies, as well as a digitally enabled health consumer, the Australian healthcare workforce is required to become digitally literate to manage the significant changes in the healthcare landscape. To ensure that new innovations such as AI are inclusive of clinicians, an understanding of how the technology will impact the healthcare professions is imperative.

METHOD

In order to explore the complex phenomenon of healthcare professionals' understanding and experiences of AI use in the delivery of healthcare, an integrative review inclusive of quantitative and qualitative studies was undertaken in June 2018.

RESULTS

One study met all inclusion criteria. This study was an observational study which used a questionnaire to measure healthcare professional's intrinsic motivation in adoption behaviour when using an artificially intelligent medical diagnosis support system (AIMDSS).

DISCUSSION

The study found that healthcare professionals were less likely to use AI in the delivery of healthcare if they did not trust the technology or understand how it was used to improve patient outcomes or the delivery of care which is specific to the healthcare setting. The perception that AI would replace them in the healthcare setting was not evident. This may be due to the fact that AI is not yet at the forefront of technology use in healthcare setting. More research is needed to examine the experiences and perceptions of healthcare professionals using AI in the delivery of healthcare.

Disease vocabulary size as a surrogate marker for physicians' disease knowledge volume

Objective

Recognizing what physicians know and do not know about a particular disease is one of the keys to designing clinical decision support systems, since these systems can fulfill complementary role by recognizing this boundary. To our knowledge, however, no study has attempted to quantify how many diseases physicians actually know and thus the boundary is unclear. This study explores a method to solve this problem by investigating whether the vocabulary assessment techniques developed in the linguistics field can be applied to assess physicians’ knowledge.

Methods

The test design required us to pay special attention to disease knowledge assessment. First, to avoid imposing unnecessary burdens on the physicians, we chose a self-assessment questionnaire that was straightforward to fill out. Second, to prevent overestimation, we used a “pseudo-word” approach: fictitious diseases were included in the questionnaire, and positive responses to them were penalized. Third, we used paper-based tests, rather than computer-based ones, to further prevent participants from cheating by using a search engine. Fourth, we selectively used borderline diseases, i.e., diseases that physicians might or might not know about, rather than well-known or little-known diseases, in the questionnaire.

Results

We collected 102 valid answers from 109 physicians who attended the seminars we conducted. On the basis of these answers, we estimated that the average physician knew of 2008 diseases (95% confidence interval: (1939, 2071)). This preliminary estimation agrees with the guideline for the national license examination in Japan, suggesting that this vocabulary assessment was able to evaluate physicians’ knowledge. The survey included physicians with various backgrounds, but there were no significant differences between subgroups. Other implication for researches on clinical decision support and limitation of the sampling method adopted in this study are also discussed, toward more rigorous estimation in future surveys.

The impact of electronic health records on diagnosis

Diagnostic error may be the largest unaddressed patient safety concern in the United States, responsible for an estimated 40,000-80,000 deaths annually. With the electronic health record (EHR) now in near universal use, the goal of this narrative review is to synthesize evidence and opinion regarding the impact of the EHR and health care information technology (health IT) on the diagnostic process and its outcomes. We consider the many ways in which the EHR and health IT facilitate diagnosis and improve the diagnostic process, and conversely the major ways in which it is problematic, including the unintended consequences that contribute to diagnostic error and sometimes patient deaths. We conclude with a summary of suggestions for improving the safety and safe use of these resources for diagnosis in the future.

A Learning Health Care System Using Computer-Aided Diagnosis

Physicians intuitively apply pattern recognition when evaluating a patient. Rational diagnosis making requires that clinical patterns be put in the context of disease prior probability, yet physicians often exhibit flawed probabilistic reasoning. Difficulties in making a diagnosis are reflected in the high rates of deadly and costly diagnostic errors. Introduced 6 decades ago, computerized diagnosis support systems are still not widely used by internists. These systems cannot efficiently recognize patterns and are unable to consider the base rate of potential diagnoses. We review the limitations of current computer-aided diagnosis support systems. We then portray future diagnosis support systems and provide a conceptual framework for their development. We argue for capturing physician knowledge using a novel knowledge representation model of the clinical picture. This model (based on structured patient presentation patterns) holds not only symptoms and signs but also their temporal and semantic interrelations. We call for the collection of crowdsourced, automatically deidentified, structured patient patterns as means to support distributed knowledge accumulation and maintenance. In this approach, each structured patient pattern adds to a self-growing and -maintaining knowledge base, sharing the experience of physicians worldwide. Besides supporting diagnosis by relating the symptoms and signs with the final diagnosis recorded, the collective pattern map can also provide disease base-rate estimates and real-time surveillance for early detection of outbreaks. We explain how health care in resource-limited settings can benefit from using this approach and how it can be applied to provide feedback-rich medical education for both students and practitioners.

Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision

OBJECTIVE

The objective of this review is to summarize the state of the art of clinical decision support (CDS) circa 1990, review progress in the 25 year interval from that time, and provide a vision of what CDS might look like 25 years hence, or circa 2040.

METHOD

Informal review of the medical literature with iterative review and discussion among the authors to arrive at six axes (data, knowledge, inference, architecture and technology, implementation and integration, and users) to frame the review and discussion of selected barriers and facilitators to the effective use of CDS.

RESULT

In each of the six axes, significant progress has been made. Key advances in structuring and encoding standardized data with an increased availability of data, development of knowledge bases for CDS, and improvement of capabilities to share knowledge artifacts, explosion of methods analyzing and inferring from clinical data, evolution of information technologies and architectures to facilitate the broad application of CDS, improvement of methods to implement CDS and integrate CDS into the clinical workflow, and increasing sophistication of the end-user, all have played a role in improving the effective use of CDS in healthcare delivery.

CONCLUSION

CDS has evolved dramatically over the past 25 years and will likely evolve just as dramatically or more so over the next 25 years. Increasingly, the clinical encounter between a clinician and a patient will be supported by a wide variety of cognitive aides to support diagnosis, treatment, care-coordination, surveillance and prevention, and health maintenance or wellness.

Understanding complex clinical reasoning in infectious diseases for improving clinical decision support design

Background

Clinical experts’ cognitive mechanisms for managing complexity have implications for the design of future innovative healthcare systems. The purpose of the study is to examine the constituents of decision complexity and explore the cognitive strategies clinicians use to control and adapt to their information environment.

Methods

We used Cognitive Task Analysis (CTA) methods to interview 10 Infectious Disease (ID) experts at the University of Utah and Salt Lake City Veterans Administration Medical Center. Participants were asked to recall a complex, critical and vivid antibiotic-prescribing incident using the Critical Decision Method (CDM), a type of Cognitive Task Analysis (CTA). Using the four iterations of the Critical Decision Method, questions were posed to fully explore the incident, focusing in depth on the clinical components underlying the complexity. Probes were included to assess cognitive and decision strategies used by participants.

Results

The following three themes emerged as the constituents of decision complexity experienced by the Infectious Diseases experts: 1) the overall clinical picture does not match the pattern, 2) a lack of comprehension of the situation and 3) dealing with social and emotional pressures such as fear and anxiety. All these factors contribute to decision complexity. These factors almost always occurred together, creating unexpected events and uncertainty in clinical reasoning. Five themes emerged in the analyses of how experts deal with the complexity. Expert clinicians frequently used 1) watchful waiting instead of over- prescribing antibiotics, engaged in 2) theory of mind to project and simulate other practitioners’ perspectives, reduced very complex cases into simple 3) heuristics, employed 4) anticipatory thinking to plan and re-plan events and consulted with peers to share knowledge, solicit opinions and 5) seek help on patient cases.

Conclusion

The cognitive strategies to deal with decision complexity found in this study have important implications for design future decision support systems for the management of complex patients.

Electronic supplementary material

The online version of this article (doi:10.1186/s12911-015-0221-z) contains supplementary material, which is available to authorized users.

Trending autoregulatory indices during treatment for traumatic brain injury

Our goal is to use automatic data monitoring for reliable prediction of episodes of intracranial hypertension in patients with traumatic brain injury. Here we test the validity of our method on retrospective patient data. We developed the Continuous Hemodynamic Autoregulatory Monitor (CHARM), that siphons and stores signals from existing monitors in the surgical intensive care unit (SICU), efficiently compresses them, and standardizes the search for statistical relationships between any proposed index and adverse events. CHARM uses an automated event detector to reliably locate episodes of elevated intracranial pressure (ICP), while eliminating artifacts within retrospective patient data. A graphical user interface allowed data scanning, selection of criteria for events, and calculating indices. The pressure reactivity index (PRx), defined as the least square linear regression slope of intracranial pressure versus arterial BP, was calculated for a single case that spanned 259 h. CHARM collected continuous records of ABP, ICP, ECG, SpO2, and ventilation from 29 patients with TBI over an 18-month period. Analysis of a single patient showed that PRx data distribution in the single hours immediately prior to all 16 intracranial hypertensive events, significantly differed from that in the 243 h that did not precede such events (p < 0.0001). The PRx index, however, lacked sufficient resolution as a real-time predictor of IH in this patient. CHARM streamlines the search for reliable predictors of intracranial hypertension. We report statistical evidence supporting the predictive potential of the pressure reactivity index.

Patient clustering with uncoded text in electronic medical records

We propose a mixture model for text data designed to capture underlying structure in the history of present illness section of electronic medical records data. Additionally, we propose a method to induce bias that leads to more homogeneous sets of diagnoses for patients in each cluster. We apply our model to a collection of electronic records from an emergency department and compare our results to three other relevant models in order to assess performance. Results using standard metrics demonstrate that patient clusters from our model are more homogeneous when compared to others, and qualitative analyses suggest that our approach leads to interpretable patient sub-populations when applied to real data. Finally, we demonstrate an example of our patient clustering model to identify adverse drug events.

Diagnostic Reasoning and Decision Making in the Context of Health Information Technology

Diagnostic reasoning and medical decision making have been focal areas of research in the fields of medical education, cognition, and artificial intelligence in medicine. Drawing on several decades worth of research, we propose an integrated summary of prior research on diagnostic reasoning and decision making—in terms of both historical development and theoretical shifts. We also characterize the changes in research and theory resulting from the incorporation and adoption of health information technology in the clinical work place. In this paper, we differentiate between the various forms of diagnostic reasoning and trace the evolution of the various models of reasoning, including knowledge-based, exemplar-based, and visual strategies. We also discuss the effect of clinical expertise on reasoning processes. Within the medical decision-making research, we delineate the various approaches highlighting decision-making errors that arise due to the nature of heuristics and biases and other factors. Although there has been significant progress in our understanding, there is still a need for greater theoretical integration of disparate empirical phenomena. Specifically, there is a need to reconcile the various characterizations of reasoning and to evaluate the similarity and differences in the context of current health care practice. Finally, we discuss the role of human factors research in the study of clinical environments and also in relation to devising approaches and methodologies for understanding, evaluating, and supporting the diagnostic reasoning and decision processes.

Using network theory to identify the causes of disease outbreaks of unknown origin

The identification of undiagnosed disease outbreaks is critical for mobilizing efforts to prevent widespread transmission of novel virulent pathogens. Recent developments in online surveillance systems allow for the rapid communication of the earliest reports of emerging infectious diseases and tracking of their spread. The efficacy of these programs, however, is inhibited by the anecdotal nature of informal reporting and uncertainty of pathogen identity in the early stages of emergence. We developed theory to connect disease outbreaks of known aetiology in a network using an array of properties including symptoms, seasonality and case-fatality ratio. We tested the method with 125 reports of outbreaks of 10 known infectious diseases causing encephalitis in South Asia, and showed that different diseases frequently form distinct clusters within the networks. The approach correctly identified unknown disease outbreaks with an average sensitivity of 76 per cent and specificity of 88 per cent. Outbreaks of some diseases, such as Nipah virus encephalitis, were well identified (sensitivity = 100%, positive predictive values = 80%), whereas others (e.g. Chandipura encephalitis) were more difficult to distinguish. These results suggest that unknown outbreaks in resource-poor settings could be evaluated in real time, potentially leading to more rapid responses and reducing the risk of an outbreak becoming a pandemic.

Factors of accepting pain management decision support systems by nurse anesthetists

BACKGROUND

Pain management is a critical but complex issue for the relief of acute pain, particularly for postoperative pain and severe pain in cancer patients. It also plays important roles in promoting quality of care. The introduction of pain management decision support systems (PM-DSS) is considered a potential solution for addressing the complex problems encountered in pain management. This study aims to investigate factors affecting acceptance of PM-DSS from a nurse anesthetist perspective.

METHODS

A questionnaire survey was conducted to collect data from nurse anesthetists in a case hospital. A total of 113 questionnaires were distributed, and 101 complete copies were returned, indicating a valid response rate of 89.3%. Collected data were analyzed by structure equation modeling using the partial least square tool.

RESULTS

The results show that perceived information quality (γ=.451, p<.001), computer self-efficacy (γ=.315, p<.01), and organizational structure (γ=.210, p<.05), both significantly impact nurse anesthetists' perceived usefulness of PM-DSS. Information quality (γ=.267, p<.05) significantly impacts nurse anesthetists' perceptions of PM-DSS ease of use. Furthermore, both perceived ease of use (β=.436, p<.001, R(2)=.487) and perceived usefulness (β=.443, p<.001, R(2)=.646) significantly affected nurse anesthetists' PM-DSS acceptance (R2=.640). Thus, the critical role of information quality in the development of clinical decision support system is demonstrated.

CONCLUSIONS

The findings of this study enable hospital managers to understand the important considerations for nurse anesthetists in accepting PM-DSS, particularly for the issues related to the improvement of information quality, perceived usefulness and perceived ease of use of the system. In addition, the results also provide useful suggestions for designers and implementers of PM-DSS in improving system development.

Cognitive Aspect of Diagnostic Errors

Diagnostic errors can result in tangible harm to patients. Despite our advances in medicine, the mental processes required to make a diagnosis exhibits shortcomings, causing diagnostic errors. Cognitive factors are found to be an important cause of diagnostic errors. With new understanding from psychology and social sciences, clinical medicine is now beginning to appreciate that our clinical reasoning can take the form of analytical reasoning or heuristics. Different factors like cognitive biases and affective influences can also impel unwary clinicians to make diagnostic errors. Various strategies have been proposed to reduce the effect of cognitive biases and affective influences when clinicians make diagnoses; however evidence for the efficacy of these methods is still sparse. This paper aims to introduce the reader to the cognitive aspect of diagnostic errors, in the hope that clinicians can use this knowledge to improve diagnostic accuracy and patient outcomes. Keywords: Affective influence, Analytical, Diagnostic errors, Heuristics, Reflective practice

Seen through their eyes: residents' reflections on the cognitive and contextual components of diagnostic errors in medicine

PURPOSE

Diagnostic errors in medicine are common and costly. Cognitive bias causes are increasingly recognized contributors to diagnostic error but remain difficult targets for medical educators and patient safety experts. The authors explored the cognitive and contextual components of diagnostic errors described by internal medicine resident physicians through the use of an educational intervention.

METHOD

Forty-one internal medicine residents at University of Pennsylvania participated in an educational intervention in 2010 that comprised reflective writing and facilitated small-group discussion about experiences with diagnostic error from cognitive bias. Narratives and discussion were transcribed and analyzed iteratively to identify types of cognitive bias and contextual factors present.

RESULTS

All residents described a personal experience with a case of diagnostic error that contained at least one cognitive bias and one contextual factor that may have influenced the outcome. The most common cognitive biases identified by the residents were anchoring bias (36; 88%), availability bias (31; 76%), and framing effect (23; 56%). Prominent contextual factors included caring for patients on a subspecialty service (31; 76%), complex illness (26; 63%), and time pressures (22; 54%). Eighty-five percent of residents described at least one strategy to avoid a similar error in the future.

CONCLUSIONS

Residents can easily recall diagnostic errors, analyze the errors for cognitive bias, and richly describe their context. The use of reflective writing and narrative discussion is an educational strategy to teach recognition, analysis, and cognitive-bias-avoidance strategies for diagnostic error in residency education.

Impact of a computer-based diagnostic decision support tool on the differential diagnoses of medicine residents

BACKGROUND

Computer-based medical diagnostic decision support systems have been used for decades, initially as stand-alone applications. More recent versions have been tested for their effectiveness in enhancing the diagnostic ability of clinicians.

OBJECTIVE

To determine if viewing a rank-ordered list of diagnostic possibilities from a medical diagnostic decision support system improves residents' differential diagnoses or management plans.

METHOD

Twenty first-year internal medicine residents at Massachusetts General Hospital viewed 3 deidentified case descriptions of real patients. All residents completed a web-based questionnaire, entering the differential diagnosis and management plan before and after seeing the diagnostic decision support system's suggested list of diseases. In all 3 exercises, the actual case diagnosis was first on the system's list. Each resident served as his or her own control (pretest/posttest).

RESULTS

For all 3 cases, a substantial percentage of residents changed their primary considered diagnosis after reviewing the system's suggested diagnoses, and a number of residents who had not initially listed a "further action" (laboratory test, imaging study, or referral) added or changed their management options after using the system. Many residents (20% to 65% depending on the case) improved their differential diagnosis from before to after viewing the system's suggestions. The average time to complete all 3 cases was 15.4 minutes. Most residents thought that viewing the medical diagnostic decision support system's list of suggestions was helpful.

CONCLUSION

Viewing a rank-ordered list of diagnostic possibilities from a diagnostic decision support tool had a significant beneficial effect on the quality of first-year medicine residents' differential diagnoses and management plans.

Expert systems in clinical microbiology

This review aims to discuss expert systems in general and how they may be used in medicine as a whole and clinical microbiology in particular (with the aid of interpretive reading). It considers rule-based systems, pattern-based systems, and data mining and introduces neural nets. A variety of noncommercial systems is described, and the central role played by the EUCAST is stressed. The need for expert rules in the environment of reset EUCAST breakpoints is also questioned. Commercial automated systems with on-board expert systems are considered, with emphasis being placed on the "big three": Vitek 2, BD Phoenix, and MicroScan. By necessity and in places, the review becomes a general review of automated system performances for the detection of specific resistance mechanisms rather than focusing solely on expert systems. Published performance evaluations of each system are drawn together and commented on critically.

Checklists to reduce diagnostic errors

Diagnostic errors are common and can often be traced to physicians' cognitive biases and failed heuristics (mental shortcuts). A great deal is known about how these faulty thinking processes lead to error, but little is known about how to prevent them. Faulty thinking plagues other high-risk, high-reliability professions, such as airline pilots and nuclear plant operators, but these professions have reduced errors by using checklists. Recently, checklists have gained acceptance in medical settings, such as operating rooms and intensive care units. This article extends the checklist concept to diagnosis and describes three types of checklists: (1) a general checklist that prompts physicians to optimize their cognitive approach, (2) a differential diagnosis checklist to help physicians avoid the most common cause of diagnostic error--failure to consider the correct diagnosis as a possibility, and (3) a checklist of common pitfalls and cognitive forcing functions to improve evaluation of selected diseases. These checklists were developed informally and have not been subjected to rigorous evaluation. The purpose of this article is to argue for the further investigation and revision of these initial attempts to apply checklists to the diagnostic process. The basic idea behind checklists is to provide an alternative to reliance on intuition and memory in clinical problem solving. This kind of solution is demanded by the complexity of diagnostic reasoning, which often involves sense-making under conditions of great uncertainty and limited time.

Governance for clinical decision support: case studies and recommended practices from leading institutions

OBJECTIVE

Clinical decision support (CDS) is a powerful tool for improving healthcare quality and ensuring patient safety; however, effective implementation of CDS requires effective clinical and technical governance structures. The authors sought to determine the range and variety of these governance structures and identify a set of recommended practices through observational study.

DESIGN

Three site visits were conducted at institutions across the USA to learn about CDS capabilities and processes from clinical, technical, and organizational perspectives. Based on the results of these visits, written questionnaires were sent to the three institutions visited and two additional sites. Together, these five organizations encompass a variety of academic and community hospitals as well as small and large ambulatory practices. These organizations use both commercially available and internally developed clinical information systems.

MEASUREMENTS

Characteristics of clinical information systems and CDS systems used at each site as well as governance structures and content management approaches were identified through extensive field interviews and follow-up surveys.

RESULTS

Six recommended practices were identified in the area of governance, and four were identified in the area of content management. Key similarities and differences between the organizations studied were also highlighted.

CONCLUSION

Each of the five sites studied contributed to the recommended practices presented in this paper for CDS governance. Since these strategies appear to be useful at a diverse range of institutions, they should be considered by any future implementers of decision support.

Approaching semantic interoperability in Health Level Seven

'Semantic Interoperability' is a driving objective behind many of Health Level Seven's standards. The objective in this paper is to take a step back, and consider what semantic interoperability means, assess whether or not it has been achieved, and, if not, determine what concrete next steps can be taken to get closer. A framework for measuring semantic interoperability is proposed, using a technique called the 'Single Logical Information Model' framework, which relies on an operational definition of semantic interoperability and an understanding that interoperability improves incrementally. Whether semantic interoperability tomorrow will enable one computer to talk to another, much as one person can talk to another person, is a matter for speculation. It is assumed, however, that what gets measured gets improved, and in that spirit this framework is offered as a means to improvement.