A method for structuring complex clinical knowledge and its representational formalisms to support composite knowledge interoperability in healthcare

INTRODUCTION

The use and interoperability of clinical knowledge starts with the quality of the formalism utilized to express medical expertise. However, a crucial challenge is that existing formalisms are often suboptimal, lacking the fidelity to represent complex knowledge thoroughly and concisely. Often this leads to difficulties when seeking to unambiguously capture, share, and implement the knowledge for care improvement in clinical information systems used by providers and patients.

OBJECTIVES

To provide a systematic method to address some of the complexities of knowledge composition and interoperability related to standards-based representational formalisms of medical knowledge.

METHODS

Several cross-industry (Healthcare, Linguistics, System Engineering, Standards Development, and Knowledge Engineering) frameworks were synthesized into a proposed reference knowledge framework. The framework utilizes IEEE 42010, the MetaObject Facility, the Semantic Triangle, an Ontology Framework, and the Domain and Comprehensibility Appropriateness criteria. The steps taken were: 1) identify foundational cross-industry frameworks, 2) select architecture description method, 3) define life cycle viewpoints, 4) define representation and knowledge viewpoints, 5) define relationships between neighboring viewpoints, and 6) establish characteristic definitions of the relationships between components. System engineering principles applied included separation of concerns, cohesion, and loose coupling.

RESULTS

A "Multilayer Metamodel for Representation and Knowledge" (M*R/K) reference framework was defined. It provides a standard vocabulary for organizing and articulating medical knowledge curation perspectives, concepts, and relationships across the artifacts created during the life cycle of language creation, authoring medical knowledge, and knowledge implementation in clinical information systems such as electronic health records (EHR).

CONCLUSION

M*R/K provides a systematic means to address some of the complexities of knowledge composition and interoperability related to medical knowledge representations used in diverse standards. The framework may be used to guide the development, assessment, and coordinated use of knowledge representation formalisms. M*R/K could promote the alignment and aggregated use of distinct domain-specific languages in composite knowledge artifacts such as clinical practice guidelines (CPGs).

A survey of computable biomedical knowledge repositories

Introduction

While data repositories are well-established in clinical and research enterprises, knowledge repositories with shareable computable biomedical knowledge (CBK) are relatively new entities to the digital health ecosystem. Trustworthy knowledge repositories are necessary for learning health systems, but the policies, standards, and practices to promote trustworthy CBK artifacts and methods to share, and safely and effectively use them are not well studied.

Methods

We conducted an online survey of 24 organizations in the United States known to be involved in the development or deployment of CBK. The aim of the survey was to assess the current policies and practices governing these repositories and to identify best practices. Descriptive statistics methods were applied to data from 13 responding organizations, to identify common practices and policies instantiating the TRUST principles of Transparency, Responsibility, User Focus, Sustainability, and Technology.

Results

All 13 respondents indicated to different degrees adherence to policies that convey TRUST. Transparency is conveyed by having policies pertaining to provenance, credentialed contributors, and provision of metadata. Repositories provide knowledge in machine-readable formats, include implementation guidelines, and adhere to standards to convey Responsibility. Repositories report having Technology functions that enable end-users to verify, search, and filter for knowledge products. Less common TRUST practices are User Focused procedures that enable consumers to know about user licensing requirements or query the use of knowledge artifacts. Related to Sustainability, less than a majority post describe their sustainability plans. Few organizations publicly describe whether patients play any role in their decision-making.

Conclusion

It is essential that knowledge repositories identify and apply a baseline set of criteria to lay a robust foundation for their trustworthiness leading to optimum uptake, and safe, reliable, and effective use to promote sharing of CBK. Identifying current practices suggests a set of desiderata for the CBK ecosystem in its continued evolution.

Summary of fourth annual MCBK public meeting: Mobilizing computable biomedical knowledge-metadata and trust

The exponential growth of biomedical knowledge in computable formats challenges organizations to consider mobilizing artifacts in findable, accessible, interoperable, reusable, and trustable (FAIR+T) ways1. There is a growing need to apply biomedical knowledge artifacts to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations lack the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are insufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community formed in 2016 to address these needs. This report summarizes the main outputs of the Fourth Annual MCBK public meeting, which was held virtually July 20 to July 21, 2021 and convened over 100 participants spanning diverse domains to frame and address important dimensions for mobilizing CBK.

Summary of third annual MCBK public meeting: Mobilizing computable biomedical knowledge-Accelerating the second knowledge revolution

The volume of biomedical knowledge is growing exponentially and much of this knowledge is represented in computer executable formats, such as models, algorithms, and programmatic code. There is a growing need to apply this knowledge to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations do not yet have the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are not sufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community was formed in 2016 to address these needs. This report summarizes the main outputs of the third annual MCBK public meeting, which was held virtually from June 30 to July 1, 2020 and brought together over 200 participants from various domains to frame and address important dimensions for mobilizing CBK.

Commercial Interests in Continuing Medical Education: Where Do Electronic Health Record Vendors Fit?

The Accreditation Council for Continuing Medical Education (ACCME) will not accredit an organization that it defines as a commercial interest, that is an entity that produces, markets, resells, or distributes health care goods or services consumed by, or used on, patients. Thus, commercial interests are not eligible to be accredited organizations offering continuing medical education (CME) credit to physicians. This decision is based on the concern that commercial interests may use CME events to market their products or services to physicians, who then might inappropriately prescribe or administer those products or services to patients. Studies have shown that CME events supported by pharmaceutical companies, for example, have influenced physicians' prescribing behaviors.Currently, however, the ACCME does not recognize electronic health record (EHR) vendors, which are part of a multi-billion-dollar business, as commercial interests, and it accredits them to provide or directly influence CME events. Like pharmaceutical company-sponsored CME events, EHR vendor activities, which inherently only focus on use of the sponsoring vendor's EHR system despite its potential intrinsic limitations, can lead to physician reciprocity. Such events also may inappropriately influence EHR system purchases, upgrades, and implementation decisions. These actions can negatively influence patient safety and care. Thus, the authors of this Perspective call on the ACCME to recognize EHR vendors as commercial interests and remove them from the list of accredited CME providers.

Summary of second annual MCBK public meeting: Mobilizing Computable Biomedical Knowledge-A movement to accelerate translation of knowledge into action

The volume of biomedical knowledge is growing exponentially and much of this knowledge is represented in computer executable formats, such as models, algorithms and programmatic code. There is a growing need to apply this knowledge to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations do not yet have the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are not sufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community formed in 2016 to address these needs. This report summarizes the main outputs of the Second Annual MCBK public meeting, which was held at the National Institutes of Health on July 18-19, 2019 and brought together over 150 participants from various domains to frame and address important dimensions for mobilizing CBK.

Building and maintaining trust in clinical decision support: Recommendations from the Patient-Centered CDS Learning Network

Knowledge artifacts in digital repositories for clinical decision support (CDS) can promote the use of CDS in clinical practice. However, stakeholders will benefit from knowing which they can trust before adopting artifacts from knowledge repositories. We discuss our investigation into trust for knowledge artifacts and repositories by the Patient-Centered CDS Learning Network's Trust Framework Working Group (TFWG). The TFWG identified 12 actors (eg, vendors, clinicians, and policy makers) within a CDS ecosystem who each may play a meaningful role in prioritizing, authoring, implementing, or evaluating CDS and developed 33 recommendations distributed across nine "trust attributes." The trust attributes and recommendations represent a range of considerations such as the "Competency" of knowledge artifact engineers and the "Organizational Capacity" of institutions that develop and implement CDS. The TFWG findings highlight an initial effort to make trust explicit and embedded within CDS knowledge artifacts and repositories and thus more broadly accepted and used.

Melatonin profiles during the third trimester of pregnancy and health status in the offspring among day and night workers: A case series

Successful pregnancy requires adaptation in maternal physiology. During intrauterine life the mother's circadian timing system supports successful birth and postnatal development. Maternal melatonin is important to transmit circadian timing and day length to the fetus. This study aims to describe the third trimester of pregnancy among day (n = 5) and night (n = 3) workers by assessing their melatonin levels in a natural environment. Additionally, we describe the worker's metabolic profiles and compare the health status of the newborns between groups of day and night working mothers. Our results indicate an occurrence of assisted delivery (cesarean and forceps) among night workers. Moreover, the newborns of night workers showed lower Apgar index and breastfeeding difficulty indicating a worse condition to deal with the immediate outside the womb environment. Additionally, there was lower night-time melatonin production among pregnant night workers compared to day workers. These findings may be related to light-induced suppression of melatonin that occurs during night work. We conclude that night work and consequent exposure to light at unconventional times might compromise the success of pregnancy and the health of the newborn. Further studies need to be carried out to monitor pregnancy and newborn health in pregnant night workers.

•

There was lower night-time melatonin production among pregnant night workers compared to day workers.

•

Night work might compromise the success of pregnancy.

•

Night work during pregnancy might compromise the health of the newborn.

Evaluating Consensus Among Physicians in Medical Knowledge Base Construction

This study evaluates inter-author variability in knowledge base construction. Seven board-certified internists independently profiled “acute perinephric abscess”, using as reference material a set of 109 peer-reviewed articles. Each participant created a list of findings associated with the disease, estimated the predictive value and sensitivity of each finding, and assessed the pertinence of each article for making each judgment. Agreement in finding selection was significantly different from chance: seven, six, and five participants selected the same finding 78.6, 9.8, and 1.6 times more often than predicted by chance. Findings with the highest sensitivity were most likely to be included by all participants. The selection of supporting evidence from the medical literature was significantly related to each physician’s agreement with the majority. The study shows that, with appropriate guidance, physicians can reproducibly extract information from the medical literature, and thus established a foundation for multi-author knowledge base construction.

Probabilistic Diagnosis Using a Reformulation of the INTERNIST-1/QMR Knowledge Base

In Part I of this two-part series, we report the design of a probabilistic reformulation of the Quick Medical Reference (QMR) diagnostic decision-support tool. We describe a two-level multiply connected belief-network representation of the QMR knowledge base of internal medicine. In the belief-network representation of the QMR knowledge base, we use probabilities derived from the QMR disease profiles, from QMR imports of findings, and from National Center for Health Statistics hospital-discharge statistics.

We use a stochastic simulation algorithm for inference on the belief network. This algorithm computes estimates of the posterior marginal probabilities of diseases given a set of findings. In Part II of the series, we compare the performance of QMR to that of our probabilistic system on cases abstracted from continuing medical education materials from Scientific American Medicine. In addition, we analyze empirically several components of the probabilistic model and simulation algorithm.

Probabilistic Diagnosis Using a Reformulation of the INTERNIST-1/QMR Knowledge Base

We have developed a probabilistic reformulation of the Quick Medical Reference (QMR) system. In Part I of this two-part series, we described a two-level, multiply connected belief-network representation of the QMR knowledge base and a simulation algorithm to perform probabilistic inference on the reformulated knowledge base. In Part II of this series, we report on an evaluation of the probabilistic QMR, in which we compare the performance of QMR to that of our probabilistic system on cases abstracted from continuing medical education materials from Scientific American Medicine. In addition, we analyze empirically several components of the probabilistic model and simulation algorithm.

How Stakeholder Assessment of E-Prescribing Can Help Determine Incentives to Facilitate Management of Care: A Delphi Study

BACKGROUND

Little research has been conducted on the quality, benefits, costs, and financial considerations associated with health information technology (HIT), particularly informatics technologies such as e-prescribing, from the perspective of all of its stakeholders.

OBJECTIVES

To (a) identify the stakeholders involved in e-prescribing and (b) identify and rank order the positives and negatives of e-prescribing from the perspective of stakeholders in order to create a framework for payers, integrated delivery systems, policymakers and legislators, and those who influence public policy to assist them in the development of incentives and payment mechanisms that result in the better management of care.

METHODS

The Delphi method was used to enlist a panel of experts in e-prescribing, informatics, and/or HIT who have published in the field. This panel was presented with the results of initial research and an online survey of questions that sought to prioritize the quality, benefit, cost, and financial effects of e-prescribing from the perspective of each stakeholder. Eleven experts completed the first survey, which contained a list of stakeholders and positives and negatives associated with e-prescribing. Nine of the 11 experts completed the second survey, and 7 experts completed the final survey. From the results of these 3 surveys, a framework was presented to 5 framework experts, who were representatives from payers, integrated delivery systems, policymakers and legislators, and those who influence public policy. These framework experts were interviewed regarding the usefulness of the framework from their perspectives.

RESULTS

The experts added stakeholders and many positives and negatives to the initial list and rank ordered the positives and negatives of e-prescribing from the perspective of each stakeholder. The aggregate results were summarized by stakeholder category. The positives and negatives were categorized as health, finance, effort, time, management, or data concerns. The framework experts evaluated the framework and found it useful.

CONCLUSIONS

Positives and negatives associated with e-prescribing in the areas of quality, benefits, costs, and financial considerations can be rank ordered from the perspective of each stakeholder. The experts agreed on some points but disagreed on others. For example, they agreed that the main negative of e-prescribing from the perspective of pharmacists and pharmacies was its implementation costs but differed on the importance of providing faster information transfer. A framework was created that could be successfully used by payers, integrated delivery systems, policymakers and legislators, and those who influence public policy for the development of incentives and payment mechanisms.

DISCLOSURES

This research was supported by the National Library of Medicine of the National Institutes of Health under Award Number T15LM007088. The authors declare no conflicts of interest in the research. Study concept and design were contributed by DeMuro, Ash, Middleton, and Fletcher. DeMuro took the lead in data collection, along with Ash, and data interpretation was performed by DeMuro, Ash, Madison, Middleton, and Fletcher. The manuscript was written primarily by DeMuro, along with Ash and Middleton, and revised by DeMuro, Madison, and Ash, along with Middleton and Fletcher.

Crossing the health IT chasm: considerations and policy recommendations to overcome current challenges and enable value-based care

While great progress has been made in digitizing the US health care system, today's health information technology (IT) infrastructure remains largely a collection of systems that are not designed to support a transition to value-based care. In addition, the pursuit of value-based care, in which we deliver better care with better outcomes at lower cost, places new demands on the health care system that our IT infrastructure needs to be able to support. Provider organizations pursuing new models of health care delivery and payment are finding that their electronic systems lack the capabilities needed to succeed. The result is a chasm between the current health IT ecosystem and the health IT ecosystem that is desperately needed.In this paper, we identify a set of focal goals and associated near-term achievable actions that are critical to pursue in order to enable the health IT ecosystem to meet the acute needs of modern health care delivery. These ideas emerged from discussions that occurred during the 2015 American Medical Informatics Association Policy Invitational Meeting. To illustrate the chasm and motivate our recommendations, we created a vignette from the multistakeholder perspectives of a patient, his provider, and researchers/innovators. It describes an idealized scenario in which each stakeholder's needs are supported by an integrated health IT environment. We identify the gaps preventing such a reality today and present associated policy recommendations that serve as a blueprint for critical actions that would enable us to cross the current health IT chasm by leveraging systems and information to routinely deliver high-value care.

Diurnal preference, mood and the response to morning light in relation to polymorphisms in the human clock gene PER3

PER3 gene polymorphisms have been associated with differences in human sleep-wake phenotypes, and sensitivity to light. The aims of this study were to assess: i) the frequency of allelic variants at two PER3 polymorphic sites (rs57875989 length polymorphism: PER3 ⁴, PER3 ⁵; rs228697 SNP: PER3 ^C, PER3 ^G) in relation to sleep-wake timing; ii) the effect of morning light on behavioural/circadian variables in PER3 ⁴ /PER3 ⁴ and PER3 ⁵ /PER3 ⁵ homozygotes. 786 Caucasian subjects living in Northern Italy donated buccal DNA and completed diurnal preference, sleep quality/timing and sleepiness/mood questionnaires. 19 PER3 ⁴ /PER3 ⁴ and 11 PER3 ⁵ /PER3 ⁵ homozygotes underwent morning light administration, whilst monitoring sleep-wake patterns and the urinary 6-sulphatoxymelatonin (aMT6s) rhythm. No significant relationship was observed between the length polymorphism and diurnal preference. By contrast, a significant association was observed between the PER3 ^G variant and morningness (OR = 2.10), and between the PER3 ^G-PER3 ⁴ haplotype and morningness (OR = 2.19), for which a mechanistic hypothesis is suggested. No significant differences were observed in sleep timing/aMT6s rhythms between PER3 ⁵ /PER3 ⁵ and PER3 ⁴ /PER3 ⁴ subjects at baseline. After light administration, PER3 ⁴ /PER3 ⁴ subjects advanced their aMT6s acrophase (p < 0.05), and showed a trend of advanced sleep-wake timing. In conclusion, significant associations were observed between PER3 polymorphic variants/their combinations and both diurnal preference and the response to light.

Sleep and circadian rhythm disturbance in bipolar disorder

BACKGROUND

Subjective reports of insomnia and hypersomnia are common in bipolar disorder (BD). It is unclear to what extent these relate to underlying circadian rhythm disturbance (CRD). In this study we aimed to objectively assess sleep and circadian rhythm in a cohort of patients with BD compared to matched controls.

METHOD

Forty-six patients with BD and 42 controls had comprehensive sleep/circadian rhythm assessment with respiratory sleep studies, prolonged accelerometry over 3 weeks, sleep questionnaires and diaries, melatonin levels, alongside mood, psychosocial functioning and quality of life (QoL) questionnaires.

RESULTS

Twenty-three (50%) patients with BD had abnormal sleep, of whom 12 (52%) had CRD and 29% had obstructive sleep apnoea. Patients with abnormal sleep had lower 24-h melatonin secretion compared to controls and patients with normal sleep. Abnormal sleep/CRD in BD was associated with impaired functioning and worse QoL.

CONCLUSIONS

BD is associated with high rates of abnormal sleep and CRD. The association between these disorders, mood and functioning, and the direction of causality, warrants further investigation.

A Quality, Benefit, Cost, and Financial Framework for Health Information Technology, E-Prescribing: A Delphi Study

BACKGROUND

Little research has been conducted about the quality, benefits, costs, and financial considerations associated with health information technology (HIT), particularly informatics technologies, such as e-prescribing, from the perspective of all its stakeholders.

OBJECTIVES

This research effort sought to identify the stakeholders involved in e-prescribing and to identify and rank-order the positives and the negatives from the perspective of the stakeholders to create a framework to assist in the development of incentives and payment mechanisms which result in better managed care.

METHODS

The Delphi method was employed by enlisting a panel of experts. They were presented with the results of initial research in an online survey of questions which sought to prioritize the quality, benefit, cost, and financial effects of e-prescribing from the perspective of each stakeholder. From the results of this study, a framework was presented to framework experts.

RESULTS

The experts added stakeholders and positives and negatives to the initial lists and rank-ordered the positives and negatives of e-prescribing from the perspective of each stakeholder. The aggregate results were summarized by category of stakeholder. The framework experts evaluated the framework.

CONCLUSIONS

Positives and negatives can be rank-ordered from the perspective of each stakeholder. A useful framework was created.

Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision

OBJECTIVE

The objective of this review is to summarize the state of the art of clinical decision support (CDS) circa 1990, review progress in the 25 year interval from that time, and provide a vision of what CDS might look like 25 years hence, or circa 2040.

METHOD

Informal review of the medical literature with iterative review and discussion among the authors to arrive at six axes (data, knowledge, inference, architecture and technology, implementation and integration, and users) to frame the review and discussion of selected barriers and facilitators to the effective use of CDS.

RESULT

In each of the six axes, significant progress has been made. Key advances in structuring and encoding standardized data with an increased availability of data, development of knowledge bases for CDS, and improvement of capabilities to share knowledge artifacts, explosion of methods analyzing and inferring from clinical data, evolution of information technologies and architectures to facilitate the broad application of CDS, improvement of methods to implement CDS and integrate CDS into the clinical workflow, and increasing sophistication of the end-user, all have played a role in improving the effective use of CDS in healthcare delivery.

CONCLUSION

CDS has evolved dramatically over the past 25 years and will likely evolve just as dramatically or more so over the next 25 years. Increasingly, the clinical encounter between a clinician and a patient will be supported by a wide variety of cognitive aides to support diagnosis, treatment, care-coordination, surveillance and prevention, and health maintenance or wellness.

The Chief Clinical Informatics Officer (CCIO): AMIA Task Force Report on CCIO Knowledge, Education, and Skillset Requirements

INTRODUCTION

The emerging operational role of the "Chief Clinical Informatics Officer" (CCIO) remains heterogeneous with individuals deriving from a variety of clinical settings and backgrounds. The CCIO is defined in title, responsibility, and scope of practice by local organizations. The term encompasses the more commonly used Chief Medical Informatics Officer (CMIO) and Chief Nursing Informatics Officer (CNIO) as well as the rarely used Chief Pharmacy Informatics Officer (CPIO) and Chief Dental Informatics Officer (CDIO).

BACKGROUND

The American Medical Informatics Association (AMIA) identified a need to better delineate the knowledge, education, skillsets, and operational scope of the CCIO in an attempt to address the challenges surrounding the professional development and the hiring processes of CCIOs.

DISCUSSION

An AMIA task force developed knowledge, education, and operational skillset recommendations for CCIOs focusing on the common core aspect and describing individual differences based on Clinical Informatics focus. The task force concluded that while the role of the CCIO currently is diverse, a growing body of Clinical Informatics and increasing certification efforts are resulting in increased homogeneity. The task force advised that 1.) To achieve a predictable and desirable skillset, the CCIO must complete clearly defined and specified Clinical Informatics education and training. 2.) Future education and training must reflect the changing body of knowledge and must be guided by changing day-to-day informatics challenges.

CONCLUSION

A better defined and specified education and skillset for all CCIO positions will motivate the CCIO workforce and empower them to perform the job of a 21st century CCIO. Formally educated and trained CCIOs will provide a competitive advantage to their respective enterprise by fully utilizing the power of Informatics science.

Lessons learned from implementing service-oriented clinical decision support at four sites: A qualitative study

OBJECTIVE

To identify challenges, lessons learned and best practices for service-oriented clinical decision support, based on the results of the Clinical Decision Support Consortium, a multi-site study which developed, implemented and evaluated clinical decision support services in a diverse range of electronic health records.

METHODS

Ethnographic investigation using the rapid assessment process, a procedure for agile qualitative data collection and analysis, including clinical observation, system demonstrations and analysis and 91 interviews.

RESULTS

We identified challenges and lessons learned in eight dimensions: (1) hardware and software computing infrastructure, (2) clinical content, (3) human-computer interface, (4) people, (5) workflow and communication, (6) internal organizational policies, procedures, environment and culture, (7) external rules, regulations, and pressures and (8) system measurement and monitoring. Key challenges included performance issues (particularly related to data retrieval), differences in terminologies used across sites, workflow variability and the need for a legal framework.

DISCUSSION

Based on the challenges and lessons learned, we identified eight best practices for developers and implementers of service-oriented clinical decision support: (1) optimize performance, or make asynchronous calls, (2) be liberal in what you accept (particularly for terminology), (3) foster clinical transparency, (4) develop a legal framework, (5) support a flexible front-end, (6) dedicate human resources, (7) support peer-to-peer communication, (8) improve standards.

CONCLUSION

The Clinical Decision Support Consortium successfully developed a clinical decision support service and implemented it in four different electronic health records and four diverse clinical sites; however, the process was arduous. The lessons identified by the Consortium may be useful for other developers and implementers of clinical decision support services.

The Number Needed to Remind: a Measure for Assessing CDS Effectiveness

BACKGROUND

Clinical decision support (CDS) is associated with improvement in quality and efficiency in healthcare delivery. The appropriate way to evaluate its effectiveness remains uncertain.

METHODS

We analyzed data from our electronic health record (EHR) measuring the display frequency of eight reminders for Coronary Artery disease and Type 2 Diabetes and their associated performance according to a predefined methodology. We propose two key performance indicators to measure their impact on a target population: the reminder performance (RP), and the number needed to remind (NNR), to evaluate the impact that Clinical decision support reminders have on the adherence to guideline derived CDS interventions on the entire patient population, and individual providers receiving the interventions.

RESULTS

Data were available for 116,027 patients and a total of 1,982,735 reminders were displayed to a subset of 65,516 patients during the study period from January 1 to December 31, 2010. The evaluation framework assessed provider acknowledgement of the CDS intervention, and the presence of the expected performance event while accounting for patients' exposure to the CDS reminders. The total RP was 2.7% while the average NNR was 3.1 for all the reminders under study.

CONCLUSIONS

The proposed framework to asses of CDS performance provides a novel approach to improve the design and evaluation of CDS interventions. The application of this methodology represents an indicator to understand the impact of CDS interventions and subsequent patient outcomes. Further research is required to evaluate the impact of these systems on the quality of care.

An open platform for personal health record apps with platform-level privacy protection

One of the main barriers to the adoption of Personal Health Records (PHR) systems is their closed nature. It has been argued in the literature that this barrier can be overcome by introducing an open market of substitutable PHR apps. The requirements introduced by such an open market on the underlying platform have also been derived. In this paper, we argue that MyPHRMachines, a cloud-based PHR platform recently developed by the authors, satisfies these requirements better than its alternatives. The MyPHRMachines platform leverages Virtual Machines as flexible and secure execution sandboxes for health apps. MyPHRMachines does not prevent pushing hospital- or patient-generated data to one of its instances, nor does it prevent patients from sharing data with their trusted caregivers. External software developers have minimal barriers to contribute innovative apps to the platform, since apps are only required to avoid pushing patient data outside a MyPHRMachines cloud. We demonstrate the potential of MyPHRMachines by presenting two externally contributed apps. Both apps provide functionality going beyond the state-of-the-art in their application domain, while they did not require any specific MyPHRMachines platform extension.

Circadian phase assessment by ambulatory monitoring in humans: correlation with dim light melatonin onset

The increased prevalence of circadian disruptions due to abnormal coupling between internal and external time makes the detection of circadian phase in humans by ambulatory recordings a compelling need. Here, we propose an accurate practical procedure to estimate circadian phase with the least possible burden for the subject, that is, without the restraints of a constant routine protocol or laboratory techniques such as melatonin quantification, both of which are standard procedures. In this validation study, subjects (N = 13) wore ambulatory monitoring devices, kept daily sleep diaries and went about their daily routine for 10 days. The devices measured skin temperature at wrist level (WT), motor activity and body position on the arm, and light exposure by means of a sensor placed on the chest. Dim light melatonin onset (DLMO) was used to compare and evaluate the accuracy of the ambulatory variables in assessing circadian phase. An evening increase in WT: WTOnset (WTOn) and "WT increase onset" (WTiO) was found to anticipate the evening increase in melatonin, while decreases in motor activity (Activity Offset or AcOff), body position (Position Offset (POff)), integrative TAP (a combination of WT, activity and body position) (TAPOffset or TAPOff) and an increase in declared sleep propensity were phase delayed with respect to DLMO. The phase markers obtained from subjective sleep (R = 0.811), WT (R = 0.756) and the composite variable TAP (R = 0.720) were highly and significantly correlated with DLMO. The findings strongly support a new method to calculate circadian phase based on WT (WTiO) that accurately predicts and shows a temporal association with DLMO. WTiO is especially recommended due to its simplicity and applicability to clinical use under conditions where knowing endogenous circadian phase is important, such as in cancer chronotherapy and light therapy.

A qualitative study of the activities performed by people involved in clinical decision support: recommended practices for success

Objective

To describe the activities performed by people involved in clinical decision support (CDS) at leading sites.

Materials and methods

We conducted ethnographic observations at seven diverse sites with a history of excellence in CDS using the Rapid Assessment Process and analyzed the data using a series of card sorts, informed by Linstone's Multiple Perspectives Model.

Results

We identified 18 activities and grouped them into four areas. Area 1: Fostering relationships across the organization, with activities (a) training and support, (b) visibility/presence on the floor, (c) liaising between people, (d) administration and leadership, (e) project management, (f) cheerleading/buy-in/sponsorship, (g) preparing for CDS implementation. Area 2: Assembling the system with activities (a) providing technical support, (b) CDS content development, (c) purchasing products from vendors (d) knowledge management, (e) system integration. Area 3: Using CDS to achieve the organization's goals with activities (a) reporting, (b) requirements-gathering/specifications, (c) monitoring CDS, (d) linking CDS to goals, (e) managing data. Area 4: Participation in external policy and standards activities (this area consists of only a single activity). We also identified a set of recommendations associated with these 18 activities.

Discussion

All 18 activities we identified were performed at all sites, although the way they were organized into roles differed substantially. We consider these activities critical to the success of a CDS program.

Conclusions

A series of activities are performed by sites strong in CDS, and sites adopting CDS should ensure they incorporate these activities into their efforts.

Evaluating standard terminologies for encoding allergy information

OBJECTIVE

Allergy documentation and exchange are vital to ensuring patient safety. This study aims to analyze and compare various existing standard terminologies for representing allergy information.

METHODS

Five terminologies were identified, including the Systemized Nomenclature of Medical Clinical Terms (SNOMED CT), National Drug File-Reference Terminology (NDF-RT), Medication Dictionary for Regulatory Activities (MedDRA), Unique Ingredient Identifier (UNII), and RxNorm. A qualitative analysis was conducted to compare desirable characteristics of each terminology, including content coverage, concept orientation, formal definitions, multiple granularities, vocabulary structure, subset capability, and maintainability. A quantitative analysis was also performed to compare the content coverage of each terminology for (1) common food, drug, and environmental allergens and (2) descriptive concepts for common drug allergies, adverse reactions (AR), and no known allergies.

RESULTS

Our qualitative results show that SNOMED CT fulfilled the greatest number of desirable characteristics, followed by NDF-RT, RxNorm, UNII, and MedDRA. Our quantitative results demonstrate that RxNorm had the highest concept coverage for representing drug allergens, followed by UNII, SNOMED CT, NDF-RT, and MedDRA. For food and environmental allergens, UNII demonstrated the highest concept coverage, followed by SNOMED CT. For representing descriptive allergy concepts and adverse reactions, SNOMED CT and NDF-RT showed the highest coverage. Only SNOMED CT was capable of representing unique concepts for encoding no known allergies.

CONCLUSIONS

The proper terminology for encoding a patient's allergy is complex, as multiple elements need to be captured to form a fully structured clinical finding. Our results suggest that while gaps still exist, a combination of SNOMED CT and RxNorm can satisfy most criteria for encoding common allergies and provide sufficient content coverage.

A highly scalable, interoperable clinical decision support service

OBJECTIVE

To create a clinical decision support (CDS) system that is shareable across healthcare delivery systems and settings over large geographic regions.

MATERIALS AND METHODS

The enterprise clinical rules service (ECRS) realizes nine design principles through a series of enterprise java beans and leverages off-the-shelf rules management systems in order to provide consistent, maintainable, and scalable decision support in a variety of settings.

RESULTS

The ECRS is deployed at Partners HealthCare System (PHS) and is in use for a series of trials by members of the CDS consortium, including internally developed systems at PHS, the Regenstrief Institute, and vendor-based systems deployed at locations in Oregon and New Jersey. Performance measures indicate that the ECRS provides sub-second response time when measured apart from services required to retrieve data and assemble the continuity of care document used as input.

DISCUSSION

We consider related work, design decisions, comparisons with emerging national standards, and discuss uses and limitations of the ECRS.

CONCLUSIONS

ECRS design, implementation, and use in CDS consortium trials indicate that it provides the flexibility and modularity needed for broad use and performs adequately. Future work will investigate additional CDS patterns, alternative methods of data passing, and further optimizations in ECRS performance.

Criteria for assessing high-priority drug-drug interactions for clinical decision support in electronic health records

BACKGROUND

High override rates for drug-drug interaction (DDI) alerts in electronic health records (EHRs) result in the potentially dangerous consequence of providers ignoring clinically significant alerts. Lack of uniformity of criteria for determining the severity or validity of these interactions often results in discrepancies in how these are evaluated. The purpose of this study was to identify a set of criteria for assessing DDIs that should be used for the generation of clinical decision support (CDS) alerts in EHRs.

METHODS

We conducted a 20-year systematic literature review of MEDLINE and EMBASE to identify characteristics of high-priority DDIs. These criteria were validated by an expert panel consisting of medication knowledge base vendors, EHR vendors, in-house knowledge base developers from academic medical centers, and both federal and private agencies involved in the regulation of medication use.

RESULTS

Forty-four articles met the inclusion criteria for assessing characteristics of high-priority DDIs. The panel considered five criteria to be most important when assessing an interaction- Severity, Probability, Clinical Implications of the interaction, Patient characteristics, and the Evidence supporting the interaction. In addition, the panel identified barriers and considerations for being able to utilize these criteria in medication knowledge bases used by EHRs.

CONCLUSIONS

A multi-dimensional approach is needed to understanding the importance of an interaction for inclusion in medication knowledge bases for the purpose of CDS alerting. The criteria identified in this study can serve as a first step towards a uniform approach in assessing which interactions are critical and warrant interruption of a provider's workflow.

Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA

In response to mounting evidence that use of electronic medical record systems may cause unintended consequences, and even patient harm, the AMIA Board of Directors convened a Task Force on Usability to examine evidence from the literature and make recommendations. This task force was composed of representatives from both academic settings and vendors of electronic health record (EHR) systems. After a careful review of the literature and of vendor experiences with EHR design and implementation, the task force developed 10 recommendations in four areas: (1) human factors health information technology (IT) research, (2) health IT policy, (3) industry recommendations, and (4) recommendations for the clinician end-user of EHR software. These AMIA recommendations are intended to stimulate informed debate, provide a plan to increase understanding of the impact of usability on the effective use of health IT, and lead to safer and higher quality care with the adoption of useful and usable EHR systems.

A pilot study of distributed knowledge management and clinical decision support in the cloud

OBJECTIVE

Implement and perform pilot testing of web-based clinical decision support services using a novel framework for creating and managing clinical knowledge in a distributed fashion using the cloud. The pilot sought to (1) develop and test connectivity to an external clinical decision support (CDS) service, (2) assess the exchange of data to and knowledge from the external CDS service, and (3) capture lessons to guide expansion to more practice sites and users.

MATERIALS AND METHODS

The Clinical Decision Support Consortium created a repository of shared CDS knowledge for managing hypertension, diabetes, and coronary artery disease in a community cloud hosted by Partners HealthCare. A limited data set for primary care patients at a separate health system was securely transmitted to a CDS rules engine hosted in the cloud. Preventive care reminders triggered by the limited data set were returned for display to clinician end users for review and display. During a pilot study, we (1) monitored connectivity and system performance, (2) studied the exchange of data and decision support reminders between the two health systems, and (3) captured lessons.

RESULTS

During the six month pilot study, there were 1339 patient encounters in which information was successfully exchanged. Preventive care reminders were displayed during 57% of patient visits, most often reminding physicians to monitor blood pressure for hypertensive patients (29%) and order eye exams for patients with diabetes (28%). Lessons learned were grouped into five themes: performance, governance, semantic interoperability, ongoing adjustments, and usability.

DISCUSSION

Remote, asynchronous cloud-based decision support performed reasonably well, although issues concerning governance, semantic interoperability, and usability remain key challenges for successful adoption and use of cloud-based CDS that will require collaboration between biomedical informatics and computer science disciplines.

CONCLUSION

Decision support in the cloud is feasible and may be a reasonable path toward achieving better support of clinical decision-making across the widest range of health care providers.

Design of decision support interventions for medication prescribing

OBJECTIVE

Describe optimal design attributes of clinical decision support (CDS) interventions for medication prescribing, emphasizing perceptual, cognitive and functional characteristics that improve human-computer interaction (HCI) and patient safety.

METHODS

Findings from published reports on success, failures and lessons learned during implementation of CDS systems were reviewed and interpreted with regard to HCI and software usability principles. We then formulated design recommendations for CDS alerts that would reduce unnecessary workflow interruptions and allow clinicians to make informed decisions quickly, accurately and without extraneous cognitive and interactive effort.

RESULTS

Excessive alerting that tends to distract clinicians rather than provide effective CDS can be reduced by designing only high severity alerts as interruptive dialog boxes and less severe warnings without explicit response requirement, by curating system knowledge bases to suppress warnings with low clinical utility and by integrating contextual patient data into the decision logic. Recommended design principles include parsimonious and consistent use of color and language, minimalist approach to the layout of information and controls, the use of font attributes to convey hierarchy and visual prominence of important data over supporting information, the inclusion of relevant patient data in the context of the alert and allowing clinicians to respond with one or two clicks.

CONCLUSION

Although HCI and usability principles are well established and robust, CDS and EHR system interfaces rarely conform to the best known design conventions and are seldom conceived and designed well enough to be truly versatile and dependable tools. These relatively novel interventions still require careful monitoring, research and analysis of its track record to mature. Clarity and specificity of alert content and optimal perceptual and cognitive attributes, for example, are essential for providing effective decision support to clinicians.

Structured representation for core elements of common clinical decision support interventions to facilitate knowledge sharing

At present, there are no widely accepted, standard approaches for representing computer-based clinical decision support (CDS) intervention types and their structural components. This study aimed to identify key requirements for the representation of five widely utilized CDS intervention types: alerts and reminders, order sets, infobuttons, documentation templates/forms, and relevant data presentation. An XML schema was proposed for representing these interventions and their core structural elements (e.g., general metadata, applicable clinical scenarios, CDS inputs, CDS outputs, and CDS logic) in a shareable manner. The schema was validated by building CDS artifacts for 22 different interventions, targeted toward guidelines and clinical conditions called for in the 2011 Meaningful Use criteria. Custom style sheets were developed to render the XML files in human-readable form. The CDS knowledge artifacts were shared via a public web portal. Our experience also identifies gaps in existing standards and informs future development of standards for CDS knowledge representation and sharing.

A study of diverse clinical decision support rule authoring environments and requirements for integration

Background

Efficient rule authoring tools are critical to allow clinical Knowledge Engineers (KEs), Software Engineers (SEs), and Subject Matter Experts (SMEs) to convert medical knowledge into machine executable clinical decision support rules. The goal of this analysis was to identify the critical success factors and challenges of a fully functioning Rule Authoring Environment (RAE) in order to define requirements for a scalable, comprehensive tool to manage enterprise level rules.

Methods

The authors evaluated RAEs in active use across Partners Healthcare, including enterprise wide, ambulatory only, and system specific tools, with a focus on rule editors for reminder and medication rules. We conducted meetings with users of these RAEs to discuss their general experience and perceived advantages and limitations of these tools.

Results

While the overall rule authoring process is similar across the 10 separate RAEs, the system capabilities and architecture vary widely. Most current RAEs limit the ability of the clinical decision support (CDS) interventions to be standardized, sharable, interoperable, and extensible. No existing system meets all requirements defined by knowledge management users.

Conclusions

A successful, scalable, integrated rule authoring environment will need to support a number of key requirements and functions in the areas of knowledge representation, metadata, terminology, authoring collaboration, user interface, integration with electronic health record (EHR) systems, testing, and reporting.

Using a service oriented architecture approach to clinical decision support: performance results from two CDS Consortium demonstrations

The Clinical Decision Support Consortium has completed two demonstration trials involving a web service for the execution of clinical decision support (CDS) rules in one or more electronic health record (EHR) systems. The initial trial ran in a local EHR at Partners HealthCare. A second EHR site, associated with Wishard Memorial Hospital, Indianapolis, IN, was added in the second trial. Data were gathered during each 6 month period and analyzed to assess performance, reliability, and response time in the form of means and standard deviations for all technical components of the service, including assembling and preparation of input data. The mean service call time for each period was just over 2 seconds. In this paper we report on the findings and analysis to date while describing the areas for further analysis and optimization as we continue to expand our use of a Services Oriented Architecture approach for CDS across multiple institutions.

Drug-drug interactions that should be non-interruptive in order to reduce alert fatigue in electronic health records

OBJECTIVE

Alert fatigue represents a common problem associated with the use of clinical decision support systems in electronic health records (EHR). This problem is particularly profound with drug-drug interaction (DDI) alerts for which studies have reported override rates of approximately 90%. The objective of this study is to report consensus-based recommendations of an expert panel on DDI that can be safely made non-interruptive to the provider's workflow, in EHR, in an attempt to reduce alert fatigue.

METHODS

We utilized an expert panel process to rate the interactions. Panelists had expertise in medicine, pharmacy, pharmacology and clinical informatics, and represented both academic institutions and vendors of medication knowledge bases and EHR. In addition, representatives from the US Food and Drug Administration and the American Society of Health-System Pharmacy contributed to the discussions.

RESULTS

Recommendations and considerations of the panel resulted in the creation of a list of 33 class-based low-priority DDI that do not warrant being interruptive alerts in EHR. In one institution, these accounted for 36% of the interactions displayed.

DISCUSSION

Development and customization of the content of medication knowledge bases that drive DDI alerting represents a resource-intensive task. Creation of a standardized list of low-priority DDI may help reduce alert fatigue across EHR.

CONCLUSIONS

Future efforts might include the development of a consortium to maintain this list over time. Such a list could also be used in conjunction with financial incentives tied to its adoption in EHR.

Interface design principles for usable decision support: a targeted review of best practices for clinical prescribing interventions

Developing effective clinical decision support (CDS) systems for the highly complex and dynamic domain of clinical medicine is a serious challenge for designers. Poor usability is one of the core barriers to adoption and a deterrent to its routine use. We reviewed reports describing system implementation efforts and collected best available design conventions, procedures, practices and lessons learned in order to provide developers a short compendium of design goals and recommended principles. This targeted review is focused on CDS related to medication prescribing. Published reports suggest that important principles include consistency of design concepts across networked systems, use of appropriate visual representation of clinical data, use of controlled terminology, presenting advice at the time and place of decision making and matching the most appropriate CDS interventions to clinical goals. Specificity and contextual relevance can be increased by periodic review of trigger rules, analysis of performance logs and maintenance of accurate allergy, problem and medication lists in health records in order to help avoid excessive alerting. Developers need to adopt design practices that include user-centered, iterative design and common standards based on human-computer interaction (HCI) research methods rooted in ethnography and cognitive science. Suggestions outlined in this report may help clarify the goals of optimal CDS design but larger national initiatives are needed for systematic application of human factors in health information technology (HIT) development. Appropriate design strategies are essential for developing meaningful decision support systems that meet the grand challenges of high-quality healthcare.

Key principles for a national clinical decision support knowledge sharing framework: synthesis of insights from leading subject matter experts

OBJECTIVE

To identify key principles for establishing a national clinical decision support (CDS) knowledge sharing framework.

MATERIALS AND METHODS

As part of an initiative by the US Office of the National Coordinator for Health IT (ONC) to establish a framework for national CDS knowledge sharing, key stakeholders were identified. Stakeholders' viewpoints were obtained through surveys and in-depth interviews, and findings and relevant insights were summarized. Based on these insights, key principles were formulated for establishing a national CDS knowledge sharing framework.

RESULTS

Nineteen key stakeholders were recruited, including six executives from electronic health record system vendors, seven executives from knowledge content producers, three executives from healthcare provider organizations, and three additional experts in clinical informatics. Based on these stakeholders' insights, five key principles were identified for effectively sharing CDS knowledge nationally. These principles are (1) prioritize and support the creation and maintenance of a national CDS knowledge sharing framework; (2) facilitate the development of high-value content and tooling, preferably in an open-source manner; (3) accelerate the development or licensing of required, pragmatic standards; (4) acknowledge and address medicolegal liability concerns; and (5) establish a self-sustaining business model.

DISCUSSION

Based on the principles identified, a roadmap for national CDS knowledge sharing was developed through the ONC's Advancing CDS initiative.

CONCLUSION

The study findings may serve as a useful guide for ongoing activities by the ONC and others to establish a national framework for sharing CDS knowledge and improving clinical care.

Use of order sets in inpatient computerized provider order entry systems: a comparative analysis of usage patterns at seven sites

BACKGROUND

Many computerized provider order entry (CPOE) systems include the ability to create electronic order sets: collections of clinically related orders grouped by purpose. Order sets promise to make CPOE systems more efficient, improve care quality and increase adherence to evidence-based guidelines. However, the development and implementation of order sets can be expensive and time-consuming and limited literature exists about their utilization.

METHODS

Based on analysis of order set usage logs from a diverse purposive sample of seven sites with commercially and internally developed inpatient CPOE systems, we developed an original order set classification system. Order sets were categorized across seven non-mutually exclusive axes: admission/discharge/transfer (ADT), perioperative, condition-specific, task-specific, service-specific, convenience, and personal. In addition, 731 unique subtypes were identified within five axes: four in ADT (S=4), three in perioperative, 144 in condition-specific, 513 in task-specific, and 67 in service-specific.

RESULTS

Order sets (n=1914) were used a total of 676,142 times at the participating sites during a one-year period. ADT and perioperative order sets accounted for 27.6% and 24.2% of usage respectively. Peripartum/labor, chest pain/acute coronary syndrome/myocardial infarction and diabetes order sets accounted for 51.6% of condition-specific usage. Insulin, angiography/angioplasty and arthroplasty order sets accounted for 19.4% of task-specific usage. Emergency/trauma, obstetrics/gynecology/labor delivery and anesthesia accounted for 32.4% of service-specific usage. Overall, the top 20% of order sets accounted for 90.1% of all usage. Additional salient patterns are identified and described.

CONCLUSION

We observed recurrent patterns in order set usage across multiple sites as well as meaningful variations between sites. Vendors and institutional developers should identify high-value order set types through concrete data analysis in order to optimize the resources devoted to development and implementation.

Healthcare information technology and economics

At the 2011 American College of Medical Informatics (ACMI) Winter Symposium we studied the overlap between health IT and economics and what leading healthcare delivery organizations are achieving today using IT that might offer paths for the nation to follow for using health IT in healthcare reform. We recognized that health IT by itself can improve health value, but its main contribution to health value may be that it can make possible new care delivery models to achieve much larger value. Health IT is a critically important enabler to fundamental healthcare system changes that may be a way out of our current, severe problem of rising costs and national deficit. We review the current state of healthcare costs, federal health IT stimulus programs, and experiences of several leading organizations, and offer a model for how health IT fits into our health economic future.

An hour of bright white light in the early morning improves performance and advances sleep and circadian phase during the Antarctic winter

Previous work has demonstrated that exposure to an hour of bright light in the morning and the evening during the Polar winter has beneficial effects on circadian phase. This study investigated the effect of a single hour of bright white morning light on circadian phase, sleep, alertness and cognitive performance. Nine individuals (eight male, one female, median age 30 years), wintering at Halley Research Station (75°S), Antarctica from 7th May until 6th August 2007, were exposed to bright white light for a fortnight from 08:30 to 09:30 h, with two fortnight control periods on either side. This sequence was performed twice, before and following Midwinter. Light exposure, sleep and alertness were assessed daily by actigraphy, sleep diaries and subjective visual analogue scales. Circadian phase (assessed by urinary 6-sulphatoxymelatonin rhythm) and cognitive performance were evaluated at the end of each fortnight. During light exposure circadian phase was advanced from 4.97 ± 0.96 decimal hours (dh) (mean ± SD) to 4.08 ± 0.68 dh (p = 0.003). Wake-up time was shifted by a similar margin from 8.45 ± 1.83 dh to 7.59 ± 0.78 dh (p < 0.001). Sleep start time was also advanced (p = 0.047) but by a lesser amount, consequently, actual sleep time was slightly reduced. There was no change in objective or subjective measures of sleep quality or subjective measures of alertness. An improvement in cognitive performance was found with both the Single Letter Cancellation Test (p < 0.001) and the Digit Symbol Substitution Test (p = 0.026) with preserved circadian variation. These beneficial effects of a single short duration light treatment may have implications not only for the Antarctic but other remote environments where access to natural light and delayed circadian phase, is problematic. These results require validation in larger studies at varying locations.