Cross-vendor evaluation of key user-defined clinical decision support capabilities: a scenario-based assessment of certified electronic health records with guidelines for future development

OBJECTIVE

Clinical decision support (CDS) is essential for delivery of high-quality, cost-effective, and safe healthcare. The authors sought to evaluate the CDS capabilities across electronic health record (EHR) systems.

METHODS

We evaluated the CDS implementation capabilities of 8 Office of the National Coordinator for Health Information Technology Authorized Certification Body (ONC-ACB)-certified EHRs. Within each EHR, the authors attempted to implement 3 user-defined rules that utilized the various data and logic elements expected of typical EHRs and that represented clinically important evidenced-based care. The rules were: 1) if a patient has amiodarone on his or her active medication list and does not have a thyroid-stimulating hormone (TSH) result recorded in the last 12 months, suggest ordering a TSH; 2) if a patient has a hemoglobin A1c result >7% and does not have diabetes on his or her problem list, suggest adding diabetes to the problem list; and 3) if a patient has coronary artery disease on his or her problem list and does not have aspirin on the active medication list, suggest ordering aspirin.

RESULTS

Most evaluated EHRs lacked some CDS capabilities; 5 EHRs were able to implement all 3 rules, and the remaining 3 EHRs were unable to implement any of the rules. One of these did not allow users to customize CDS rules at all. The most frequently found shortcomings included the inability to use laboratory test results in rules, limit rules by time, use advanced Boolean logic, perform actions from the alert interface, and adequately test rules.

CONCLUSION

Significant improvements in the EHR certification and implementation procedures are necessary.

Feasibility and impact of a computerised clinical decision support system on investigation and initial management of new onset chest pain: a mixed methods study

BACKGROUND

Clinical decision support systems (CDSS) can modify clinician behaviour, yet the factors influencing their effect remain poorly understood. This study assesses the feasibility and acceptability of a CDSS supporting diagnostic and treatment decisions for patients with suspected stable angina.

METHODS

Intervention The Optimising Management of Angina (OMA) programme includes a CDSS guiding investigation and medication decisions for clinicians managing patients with new onset stable angina, based on English national guidelines, introduced through an educational intervention. Design and participants A mixed methods study i. A study of outcomes among patients presenting with suspected angina in three chest pain clinics in England before and after introduction of the OMA programme. ii. Observations of clinic processes, interviews and a focus group with health professionals at two chest pain clinics after delivery of the OMA programme.

OUTCOMES

Medication and cardiovascular imaging investigations undertaken within six months of presentation, and concordance of these with the recommendations of the CDSS. Thematic analysis of qualitative data to understand how the CDSS was used.

RESULTS

Data were analysed for 285 patients attending chest pain clinics: 106 before and 179 after delivery of the OMA programme. 40 consultations were observed, 5 clinicians interviewed, and a focus group held after the intervention. The proportion of patients appropriate for diagnostic investigation who received one was 50 % (95 CI 34-66 %) of those before OMA and 59 % (95 CI 48-70 %) of those after OMA. Despite high use of the CDSS (84 % of consultations), observations and interviews revealed difficulty with data entry into the CDSS, and structural and practical barriers to its use. In the majority of cases the CDSS was not used to guide real-time decision making, only being consulted after the patient had left the room.

CONCLUSIONS

The OMA CDSS for the management of chest pain is not feasible in its current form. The CDSS was not used to support decisions about the care of individual patients. A range of barriers to the use of the CDSS were identified, some are easily removed, such as insufficient capture of cardiovascular risk, while others are more deeply embedded in current practice, such as unavailability of some investigations or no prescribing privileges for nurses.

Referral criteria and clinical decision support: radiological protection aspects for justification

Advanced imaging technology has opened new horizons for medical diagnostics and improved patient care. However, many procedures are unjustified and do not provide a net benefit. An area of particular concern is the unnecessary use of radiation when clinical evaluation or other imaging modalities could provide an accurate diagnosis. Referral criteria for medical imaging are consensus statements based on the best-available evidence to assist the decision-making process when choosing the best imaging procedure for a given patient. Although they are advisory rather than compulsory, physicians should have good reasons for deviation from these criteria. Voluntary use of referral criteria has shown limited success compared with integration into clinical decision support systems. These systems support good medical practice, can improve health service delivery, and foster safer, more efficient, fair, cost-effective care, thus contributing to the strengthening of health systems. Justification of procedures and optimisation of protection, the two pillars of radiological protection in health care, are implicit in the notion of good medical practice. However, some health professionals are not familiar with these principles, and have low awareness of radiological protection aspects of justification. A stronger collaboration between radiation protection and healthcare communities could contribute to improve the radiation protection culture in medical practice.

Development of an online, publicly accessible naive Bayesian decision support tool for mammographic mass lesions based on the American College of Radiology (ACR) BI-RADS lexicon

PURPOSE

To develop and validate a decision support tool for mammographic mass lesions based on a standardized descriptor terminology (BI-RADS lexicon) to reduce variability of practice.

MATERIALS AND METHODS

We used separate training data (1,276 lesions, 138 malignant) and validation data (1,177 lesions, 175 malignant). We created naïve Bayes (NB) classifiers from the training data with tenfold cross-validation. Our "inclusive model" comprised BI-RADS categories, BI-RADS descriptors, and age as predictive variables; our "descriptor model" comprised BI-RADS descriptors and age. The resulting NB classifiers were applied to the validation data. We evaluated and compared classifier performance with ROC-analysis.

RESULTS

In the training data, the inclusive model yields an AUC of 0.959; the descriptor model yields an AUC of 0.910 (P < 0.001). The inclusive model is superior to the clinical performance (BI-RADS categories alone, P < 0.001); the descriptor model performs similarly. When applied to the validation data, the inclusive model yields an AUC of 0.935; the descriptor model yields an AUC of 0.876 (P < 0.001). Again, the inclusive model is superior to the clinical performance (P < 0.001); the descriptor model performs similarly.

CONCLUSION

We consider our classifier a step towards a more uniform interpretation of combinations of BI-RADS descriptors. We provide our classifier at www.ebm-radiology.com/nbmm/index.html .

KEY POINTS

• We provide a decision support tool for mammographic masses at www.ebm-radiology.com/nbmm/index.html . • Our tool may reduce variability of practice in BI-RADS category assignment. • A formal analysis of BI-RADS descriptors may enhance radiologists' diagnostic performance.

A decision support tool for appropriate glucose-lowering therapy in patients with type 2 diabetes

BACKGROUND

Optimal glucose-lowering therapy in type 2 diabetes mellitus requires a patient-specific approach. Although a good framework, current guidelines are insufficiently detailed to address the different phenotypes and individual needs of patients seen in daily practice. We developed a patient-specific decision support tool based on a systematic analysis of expert opinion.

MATERIALS AND METHODS

Based on the American Diabetes Association (ADA)/European Association for the Study of Diabetes (EASD) 2012 position statement, a panel of 12 European experts rated the appropriateness (RAND/UCLA Appropriateness Method) of treatment strategies for 930 clinical scenarios, which were permutations of clinical variables considered relevant to treatment choice. These included current treatment, hemoglobin A1c difference from individualized target, risk of hypoglycemia, body mass index, life expectancy, and comorbidities. Treatment options included addition of a second or third agent, drug switches, and replacement by monotherapies if the patient was metformin-intolerant. Treatment costs were not considered. Appropriateness (appropriate, inappropriate, uncertain) was based on the median score and expert agreement. The panel recommendations were embedded in an online decision support tool (DiaScope(®); Novo Nordisk Health Care AG, Zürich, Switzerland).

RESULTS

Treatment appropriateness was associated with (combinations of) the patient variables mentioned above. As second-line agents, dipeptidyl peptidase-4 inhibitors were considered appropriate in all scenarios, followed by glucagon-like peptide-1 receptor agonists (50%), insulins (33%), and sulfonylureas (25%), but not pioglitazone (0%). Ratings of third-line combinations followed a similar pattern. Disagreement was highest for regimens including pioglitazone, sulfonylureas, or insulins and was partly due to differences in panelists' opinions and in drug availability and reimbursement across European countries (although costs were disregarded in the rating process).

CONCLUSIONS

A novel decision support tool based on the ADA/EASD 2012 position statement and a systematic analysis of expert opinion has been developed to help healthcare professionals to individualize glucose-lowering therapy in daily clinical situations.

Improving appropriateness of acid-suppressive medication use via computerized clinical decision support

As part of the Choosing Wisely Campaign, the Society of Hospital Medicine identified reducing inappropriate use of acid-suppressive medication for stress ulcer prophylaxis as 1 of 5 key opportunities to improve the value of care for hospitalized patients. We designed a computerized clinical decision support intervention to reduce use of acid-suppressive medication for stress ulcer prophylaxis in hospitalized patients outside of the intensive care unit at an academic medical center. Using quasiexperimental interrupted time series analysis, we found that the decision support intervention resulted in a significant reduction in use of acid-suppressive medication with stress ulcer prophylaxis selected as the only indication, a nonsignificant reduction in overall use, and no change in use on discharge. We found low rates of use of acid-suppressive medication for the purpose of stress ulcer prophylaxis even before the intervention, and continuing preadmission medication was the most commonly selected indication throughout the study. Our results suggest that attention should be focused on both the inpatient and outpatient settings when designing future initiatives to improve the appropriateness of acid-suppressive medication use.

Semantic Interoperability in Clinical Decision Support Systems: A Systematic Review

The interoperability of Clinical Decision Support (CDS) systems with other health information systems has become one of the main limitations to their broad adoption. Semantic interoperability must be granted in order to share CDS modules across different health information systems. Currently, numerous standards for different purposes are available to enable the interoperability of CDS systems. We performed a literature review to identify and provide an overview of the available standards that enable CDS interoperability in the areas of clinical information, decision logic, terminology, and web service interfaces.

Impact of data quality assessment on development of clinical predictive models

Data quality plays a very important role in predicting clinical outcomes. Data quality is multi dimensional and most relevant studies consider just one or two dimensions. In this study a systematic data quality assessment is performed using four data dimensions. The results demonstrate that performance of predictive models improves when the quality of the data is assessed and addressed systematically.

A mobile platform for emergency care

The paper describes the adaptation of a mobile platform initially developed for designing and administering questionnaires to a new context supporting checklists in emergency care. We took part in the checklists formalization process together with the domain experts and recognized that some tasks would highly benefit from the inherent features offered by the mobile technology. Thus we exploited the robustness of the model already designed for navigating among questionnaires and implemented additional functionalities that improved the usability of the mobile application, making it suitable for the paramedic staff and the volunteers that manage emergency cases.

An Ontology-Based Clinical Decision Support System for the Management of Patients with Multiple Chronic Disorders

Decision support systems, as means of disseminating clinical practice guidelines, are powerful software that may lead to an improvement of medical practices. However, they are not always efficient and may suffer from limitations among which are lack of flexibility and weaknesses in the integration of several clinical practice guidelines (CPGs) for the management of patients with multiple chronic disorders. We propose a framework based on an ontological modeling of CPG contents as rules. The ontology provides the required flexibility to adapt patient data and enable the provision of appropriate recommendations expressed at various levels of abstraction. To solve decisional conflicts that occur when combining multiple sources of recommendations, we proposed a method based on the subsumption graph of the patient profiles corresponding to the rules. A prototype CDSS implementing this approach has been developed. Results are given on a clinical case to illustrate the assets of ontological reasoning in increasing the number of issued recommendations and thereby the reliability of decision support.

INITIATE: An Intelligent Adaptive Alert Environment

Exposure to a large volume of alerts generated by medical Alert Generating Systems (AGS) such as drug-drug interaction softwares or clinical decision support systems over-whelms users and causes alert fatigue in them. Some of alert fatigue effects are ignoring crucial alerts and longer response times. A common approach to avoid alert fatigue is to devise mechanisms in AGS to stop them from generating alerts that are deemed irrelevant. In this paper, we present a novel framework called INITIATE: an INtellIgent adapTIve AlerT Environment to avoid alert fatigue by managing alerts generated by one or more AGS. We have identified and categories the lifecycle of different alerts and have developed alert management logic as per the alerts' lifecycle. Our framework incorporates an ontology that represents the alert management strategy and an alert management engine that executes this strategy. Our alert management framework offers the following features: (1) Adaptability based on users' feedback; (2) Personalization and aggregation of messages; and (3) Connection to Electronic Medical Records by implementing a HL7 Clinical Document Architecture parser.

Utility of Arden Syntax for Representation of Fuzzy Logic in Clinical Quality Measures

BACKGROUND

Prior work has established that fuzzy logic is prevalent in clinical practice guidelines and that Arden Syntax is suitable for representing clinical quality measures (CQMs). Approved since then, Arden Syntax v2.9 (2012) has formal constructs for fuzzy logic even as new formalisms are proposed to represent quality logic.

OBJECTIVES

Determine the prevalence of fuzzy logic in CQMs and assess the utility of a contemporary version of Arden Syntax for representing them.

METHODS

Linguistic variables were tabulated in the 329 Assessing Care of the Vulnerable Elderly (ACOVE-3) CQMs, and these logic statements were encoded in Arden Syntax.

RESULTS

In a total of 392 CQMs, linguistic variables occurred in 30.6%, and Arden Syntax could be used to represent these formally.

CONCLUSIONS

Fuzzy logic occurs commonly in CQMs, and Arden Syntax offers particular utility for the representations of these constructs.

Development of Hypertension Management Mobile Application based on Clinical Practice Guidelines

This study aims to develop and evaluate a mobile application for hypertension management based on Clinical Practice Guidelines. The application was developed according to Web-Roadmap methodology. In planning phase, we defined the tasks and product of each phase, selected clinical practice guidelines and extracted intervention items for hypertension management. In analysis phase, we analysed intervention items and made data dictionary, rules, use-case diagram, hypertension management ontology and tailored recommendations for the application. In design phase, we developed an entity-relations diagram, algorithm, and user interface and coded them in the implementation phase. In evaluation phase, first, the knowledge-base was evaluated for its accuracy by experts and they proposed three more detailed recommendations, which were added to the application. Second, mobile heuristics were evaluated. The evaluators pointed out 33 usability-related problems on mobile heuristics items. Out of these, three problems were solved by reflecting evaluators' comments.

Use of a Proven Framework for Computer Decision Support within the Intermountain Healthcare Network

Hospitalized patients in the U.S. do not always receive optimal care. In light of this, Computerized Decision Support (CDS) has been recommended to for the improvement of patient care. A number of methodologies, standards, and frameworks have been developed to facilitate the development and interoperability of computerized clinical guidelines and CDS logic. In addition, Health Information Exchange using Service-Oriented Architecture holds some promise to help realize that goal. We have used a framework at Intermountain Healthcare that employs familiar programming languages and technology to develop over 40 CDS applications during the past 13 years, which clinicians are dependent on each day. This paper describes the framework, technology, and CDS application development methods, while providing three distinct examples of applications that illustrate the need and use of the framework for patient care improvement. The main limitation of this framework is its dependence on point-to-point interfaces to access patient data. We look forward to the use of validated and accessible Service-Oriented Architecture to facilitate patient data access across diverse databases.

Virtual Oncological Networks--IT Support for an Evidence-based, Oncological Health Care Management

An interdisciplinary and intersectoral coordinated therapy management along Clinical Practice Guidelines can ensure that all patients receive adequate diagnostic, treatment, and supportive services that lead most likely to optimal outcomes. Within the research project "Virtual Oncological Networks", guideline-compliant pathways are defined and enacted within a Health Care Management Platform to support treatment planning and ongoing care of oncological diseases.

A Conceptual Framework for Decision-making Support in Uncertainty- and Risk-based Diagnosis of Rare Clinical Cases by Specialist Physicians

Mitigating uncertainty and risks faced by specialist physicians in analysis of rare clinical cases is something desired by anyone who needs health services. The number of clinical cases never seen by these experts, with little documentation, may introduce errors in decision-making. Such errors negatively affect well-being of patients, increase procedure costs, rework, health insurance premiums, and impair the reputation of specialists and medical systems involved. In this context, IT and Clinical Decision Support Systems (CDSS) play a fundamental role, supporting decision-making process, making it more efficient and effective, reducing a number of avoidable medical errors and enhancing quality of treatment given to patients. An investigation has been initiated to look into characteristics and solution requirements of this problem, model it, propose a general solution in terms of a conceptual risk-based, automated framework to support rare-case medical diagnostics and validate it by means of case studies. A preliminary validation study of the proposed framework has been carried out by interviews conducted with experts who are practicing professionals, academics, and researchers in health care. This paper summarizes the investigation and its positive results. These results motivate continuation of research towards development of the conceptual framework and of a software tool that implements the proposed model.

Clinical Decision Support using a Terminology Server to improve Patient Safety

Clinical Decision Support Systems (CDSS) are software applications that support clinicians in making healthcare decisions providing relevant information for individual patients about their specific conditions. The lack of integration between CDSS and Electronic Health Record (EHR) has been identified as a significant barrier to CDSS development and adoption. Andalusia Healthcare Public System (AHPS) provides an interoperable health information infrastructure based on a Service Oriented Architecture (SOA) that eases CDSS implementation. This paper details the deployment of a CDSS jointly with the deployment of a Terminology Server (TS) within the AHPS infrastructure. It also explains a case study about the application of decision support to thromboembolism patients and its potential impact on improving patient safety. We will apply the inSPECt tool proposal to evaluate the appropriateness of alerts in this scenario.

Physicians' Attitudes Towards the Advice of a Guideline-Based Decision Support System: A Case Study With OncoDoc2 in the Management of Breast Cancer Patients

When wrongly used, guideline-based clinical decision support systems (CDSSs) may generate inappropriate propositions that do not match the recommendations provided by clinical practice guidelines (CPGs). The user may decide to comply with or react to the CDSS, and her decision may finally comply or not with CPGs. OncoDoc2 is a guideline-based CDSS for breast cancer management. We collected 394 decisions made by multidisciplinary meeting physicians in three hospitals where the CDSS was evaluated. We observed a global CPG compliance of 86.8% and a global CDSS compliance of 75.4%. Non-CPG compliance was observed in case of a negative reactance to the CDSS, when users did not follow a correct CDSS proposition (8.6% of decisions). Because of errors in patient data entry, OncoDoc2 delivered non-recommended propositions in 21.3% of decisions, leading to compliances with CDSS and CPGs of respectively 21.4% and 65.5%, whereas both compliances exceeded 90% when CDSS advices included CPG recommendations. Automation bias, when users followed an incorrect CDSS proposition explained the remaining non-compliance with CPGs (4.6% of decisions). Securing the use of CDSSs is of major importance to warranty patient safety and benefit of their potential to improve care.

Service oriented architecture for clinical decision support: a systematic review and future directions

The use of a service-oriented architecture (SOA) has been identified as a promising approach for improving health care by facilitating reliable clinical decision support (CDS). A review of the literature through October 2013 identified 44 articles on this topic. The review suggests that SOA related technologies such as Business Process Model and Notation (BPMN) and Service Component Architecture (SCA) have not been generally adopted to impact health IT systems' performance for better care solutions. Additionally, technologies such as Enterprise Service Bus (ESB) and architectural approaches like Service Choreography have not been generally exploited among researchers and developers. Based on the experience of other industries and our observation of the evolution of SOA, we found that the greater use of these approaches have the potential to significantly impact SOA implementations for CDS.

A Service Oriented Architecture Approach to Achieve Interoperability between Immunization Information Systems in Iran

Clinical decision support (CDS) systems can support vaccine forecasting and immunization reminders; however, immunization decision-making requires data from fragmented, independent systems. Interoperability and accurate data exchange between immunization information systems (IIS) is an essential factor to utilize Immunization CDS systems. Service oriented architecture (SOA) and Health Level 7 (HL7) are dominant standards for web-based exchange of clinical information. We implemented a system based on SOA and HL7 v3 to support immunization CDS in Iran. We evaluated system performance by exchanging 1500 immunization records for roughly 400 infants between two IISs. System turnaround time is less than a minute for synchronous operation calls and the retrieved immunization history of infants were always identical in different systems. CDS generated reports were accordant to immunization guidelines and the calculations for next visit times were accurate. Interoperability is rare or nonexistent between IIS. Since inter-state data exchange is rare in United States, this approach could be a good prototype to achieve interoperability of immunization information.

Clinical decision-making tools for exam selection, reporting and dose tracking

Although many efforts have been made to reduce the radiation dose associated with individual medical imaging examinations to "as low as reasonably achievable," efforts to ensure such examinations are performed only when medically indicated and appropriate are equally if not more important. Variations in the use of ionizing radiation for medical imaging are concerning, regardless of whether they occur on a local, regional or national basis. Such variations among practices can be reduced with the use of decision support tools at the time of order entry. These tools help reduce radiation exposure among practices through the appropriate use of medical imaging. Similarly, adoption of best practices among imaging facilities can be promoted through tracking the radiation exposure among imaging patients. Practices can benchmark their aggregate radiation exposures for medical imaging through the use of dose index registries. However several variables must be considered when contemplating individual patient dose tracking. The specific dose measures and the variation among them introduced by variations in body habitus must be understood. Moreover the uncertainties in risk estimation from dose metrics related to age, gender and life expectancy must also be taken into account.

Measuring agreement between decision support reminders: the cloud vs. the local expert

BACKGROUND

A cloud-based clinical decision support system (CDSS) was implemented to remotely provide evidence-based guideline reminders in support of preventative health. Following implementation, we measured the agreement between preventive care reminders generated by an existing, local CDSS and the new, cloud-based CDSS operating on the same patient visit data.

METHODS

Electronic health record data for the same set of patients seen in primary care were sent to both the cloud-based web service and local CDSS. The clinical reminders returned by both services were captured for analysis. Cohen's Kappa coefficient was calculated to compare the two sets of reminders. Kappa statistics were further adjusted for prevalence and bias due to the potential effects of bias in the CDS logic and prevalence in the relative small sample of patients.

RESULTS

The cloud-based CDSS generated 965 clinical reminders for 405 patient visits over 3 months. The local CDSS returned 889 reminders for the same patient visit data. When adjusted for prevalence and bias, observed agreement varied by reminder from 0.33 (95% CI 0.24 - 0.42) to 0.99 (95% CI 0.97 - 1.00) and demonstrated almost perfect agreement for 7 of the 11 reminders.

CONCLUSIONS

Preventive care reminders delivered by two disparate CDS systems show substantial agreement. Subtle differences in rule logic and terminology mapping appear to account for much of the discordance. Cloud-based CDSS therefore show promise, opening the door for future development and implementation in support of health care providers with limited resources for knowledge management of complex logic and rules.

Development of a personalized decision aid for breast cancer risk reduction and management

BACKGROUND

Breast cancer risk reduction has the potential to decrease the incidence of the disease, yet remains underused. We report on the development a web-based tool that provides automated risk assessment and personalized decision support designed for collaborative use between patients and clinicians.

METHODS

Under Institutional Review Board approval, we evaluated the decision tool through a patient focus group, usability testing, and provider interviews (including breast specialists, primary care physicians, genetic counselors). This included demonstrations and data collection at two scientific conferences (2009 International Shared Decision Making Conference, 2009 San Antonio Breast Cancer Symposium).

RESULTS

Overall, the evaluations were favorable. The patient focus group evaluations and usability testing (N = 34) provided qualitative feedback about format and design; 88% of these participants found the tool useful and 94% found it easy to use. 91% of the providers (N = 23) indicated that they would use the tool in their clinical setting.

CONCLUSION

BreastHealthDecisions.org represents a new approach to breast cancer prevention care and a framework for high quality preventive healthcare. The ability to integrate risk assessment and decision support in real time will allow for informed, value-driven, and patient-centered breast cancer prevention decisions. The tool is being further evaluated in the clinical setting.

Clinical guideline-driven personalized self-management diary for paediatric cancer survivors

Shared Decision Making (SDM) is the process of patients and clinicians working together to manage medical treatment using the existing knowledge base. This paper presents the YouCan framework, a system for summarizing and presenting the necessary knowledge for SDM related to pediatric cancer follow-up management. Knowledge modelling of a Clinical Practice Guideline produced a customized ontology, which was then passed through a pellet reasoned to produce a customized patient diary that summarizes a patient's oncological history as well as the potential issues they may face in follow-up.