Computer-critiqued blood ordering using the HELP system

Automatic Detection of Thyroid and Adrenal Incidentals Using Radiology Reports and Deep Learning

BACKGROUND

Computed tomography (CT) is commonly performed when evaluating trauma patients with up to 55% showing incidental findings. Current workflows to identify and inform patients are time-consuming and prone to error. Our objective was to automatically identify thyroid and adrenal lesions in radiology reports using deep learning.

MATERIALS AND METHODS

All trauma patients who presented to an accredited Level 1 Trauma Center between January 2008 and January 2019 were included. Radiology reports of CT scans that included either a thyroid or adrenal gland were obtained. Preprocessing included word tokenization, removal of stop words, removal of punctuation, and replacement of misspellings. A word2vec model was trained using 1.4 million radiology reports. Both training and testing reports were selected at random, manually reviewed, and were considered the gold standard. True positive cases were defined as any lesions in the thyroid or adrenal gland, respectively. Training data was used to create models that would identify reports that contained either thyroid or adrenal lesions. Our primary outcomes were sensitivity and specificity of the models using predetermined thresholds on a separate testing dataset.

RESULTS

A total of 51,771 reports were identified on 35,859 trauma patients. A total of 1,789 reports were annotated for training and 500 for testing. The thyroid model predictions resulted in a 90.0% sensitivity and 95.3% specificity. The adrenal model predictions resulted in a 92.3% sensitivity and a 91.1% specificity. A total of 240 reports were confirmed to have thyroid incidentals (mean age 69.1 yrs ± 18.9, 35% M) and 214 reports with adrenal incidentals (mean age 68.7 yrs ± 16.9, 50.5% M).

CONCLUSIONS

Both the thyroid and adrenal models have excellent performance with sensitivities and specificities in the 90s. Our deep learning model has the potential to reduce administrative costs and improve the process of informing patients.

A four-phase model of the evolution of clinical decision support architectures

BACKGROUND

A large body of evidence over many years suggests that clinical decision support systems can be helpful in improving both clinical outcomes and adherence to evidence-based guidelines. However, to this day, clinical decision support systems are not widely used outside of a small number of sites. One reason why decision support systems are not widely used is the relative difficulty of integrating such systems into clinical workflows and computer systems.

PURPOSE

To review and synthesize the history of clinical decision support systems, and to propose a model of various architectures for integrating clinical decision support systems with clinical systems.

METHODS

The authors conducted an extensive review of the clinical decision support literature since 1959, sequenced the systems and developed a model.

RESULTS

The model developed consists of four phases: standalone decision support systems, decision support integrated into clinical systems, standards for sharing clinical decision support content and service models for decision support. These four phases have not heretofore been identified, but they track remarkably well with the chronological history of clinical decision support, and show evolving and increasingly sophisticated attempts to ease integrating decision support systems into clinical workflows and other clinical systems.

CONCLUSIONS

Each of the four evolutionary approaches to decision support architecture has unique advantages and disadvantages. A key lesson was that there were common limitations that almost all the approaches faced, and no single approach has been able to entirely surmount: (1) fixed knowledge representation systems inherently circumscribe the type of knowledge that can be represented in them, (2) there are serious terminological issues, (3) patient data may be spread across several sources with no single source having a complete view of the patient, and (4) major difficulties exist in transferring successful interventions from one site to another.

The anatomy of decision support during inpatient care provider order entry (CPOE): empirical observations from a decade of CPOE experience at Vanderbilt

The authors describe a pragmatic approach to the introduction of clinical decision support at the point of care, based on a decade of experience in developing and evolving Vanderbilt's inpatient "WizOrder" care provider order entry (CPOE) system. The inpatient care setting provides a unique opportunity to interject CPOE-based decision support features that restructure clinical workflows, deliver focused relevant educational materials, and influence how care is delivered to patients. From their empirical observations, the authors have developed a generic model for decision support within inpatient CPOE systems. They believe that the model's utility extends beyond Vanderbilt, because it is based on characteristics of end-user workflows and on decision support considerations that are common to a variety of inpatient settings and CPOE systems. The specific approach to implementing a given clinical decision support feature within a CPOE system should involve evaluation along three axes: what type of intervention to create (for which the authors describe 4 general categories); when to introduce the intervention into the user's workflow (for which the authors present 7 categories), and how disruptive, during use of the system, the intervention might be to end-users' workflows (for which the authors describe 6 categories). Framing decision support in this manner may help both developers and clinical end-users plan future alterations to their systems when needs for new decision support features arise.

Effect of CPOE user interface design on user-initiated access to educational and patient information during clinical care

OBJECTIVE

Authors evaluated whether displaying context sensitive links to infrequently accessed educational materials and patient information via the user interface of an inpatient computerized care provider order entry (CPOE) system would affect access rates to the materials.

DESIGN

The CPOE of Vanderbilt University Hospital (VUH) included "baseline" clinical decision support advice for safety and quality. Authors augmented this with seven new primarily educational decision support features. A prospective, randomized, controlled trial compared clinicians' utilization rates for the new materials via two interfaces. Control subjects could access study-related decision support from a menu in the standard CPOE interface. Intervention subjects received active notification when study-related decision support was available through context sensitive, visibly highlighted, selectable hyperlinks.

MEASUREMENTS

Rates of opportunities to access and utilization of study-related decision support materials from April 1999 through March 2000 on seven VUH Internal Medicine wards.

RESULTS

During 4,466 intervention subject-days, there were 240,504 (53.9/subject-day) opportunities for study-related decision support, while during 3,397 control subject-days, there were 178,235 (52.5/subject-day) opportunities for such decision support, respectively (p = 0.11). Individual intervention subjects accessed the decision support features at least once on 3.8% of subject-days logged on (278 responses); controls accessed it at least once on 0.6% of subject-days (18 responses), with a response rate ratio adjusted for decision support frequency of 9.17 (95% confidence interval 4.6-18, p < 0.0005). On average, intervention subjects accessed study-related decision support materials once every 16 days individually and once every 1.26 days in aggregate.

CONCLUSION

Highlighting availability of context-sensitive educational materials and patient information through visible hyperlinks significantly increased utilization rates for study-related decision support when compared to "standard" VUH CPOE methods, although absolute response rates were low.

Methodology for evaluating physician order entry (POE) implementations

The body of physician order entry (POE) implementations literature uses statistical evaluation methods to demonstrate changes in specified variables after POE implementation. To understand and manage the holistic impact of POE on the health care institution, a methodology that utilizes feedback to guide the POE implementation towards the satisfaction of stakeholder objectives is presented. Stakeholders jointly define quantitative and qualitative metrics for their objectives, establish target value vectors for the metrics that represent acceptable implementation outcomes and specify evaluation milestones. These are used to compare pre- and post-POE implementation clinical performance, enabling a socio-technical feedback-improvement cycle. A case study is provided to illustrate how the methodology is being used at Sunnybrook and Women's College Health Science Centre in Toronto, Canada.

Does the physician order-entry system increase the revenue of a general hospital?

OBJECTIVE

The purpose of this study was to examine whether the physician order-entry system (POE) could increase the outpatient and inpatient revenue of hospitals.

METHOD

We analyzed the inpatient and outpatient revenue data of all general hospitals (212) in South Korea obtained from the Korean National Health Insurance Corporation (KNHIC) during the period from 1996 to 1999 using the mixed model for repeated measure data.

RESULTS

Analysis of the 4-years' panel data showed that both outpatient and inpatient revenues increased significantly after POE introduction. The hospital characteristics significantly influencing inpatient revenue were the number of beds, number of physicians and the tertiary status of a hospital; whereas those for outpatient revenue were the number of beds, number of physicians, the private status of a hospital, the tertiary status of a hospital and the urban status of a hospital.

CONCLUSION

The revenues from both outpatients and inpatients were found to be increased after the introduction of the POE, while controlling for population size, competition, income, hospital location, hospital size, tertiary status and public status.

A UMLS-based knowledge acquisition tool for rule-based clinical decision support system development

Decision support systems in the medical field have to be easily modified by medical experts themselves. The authors have designed a knowledge acquisition tool to facilitate the creation and maintenance of a knowledge base by the domain expert and its sharing and reuse by other institutions. The Unified Medical Language System (UMLS) contains the domain entities and constitutes the relations repository from which the expert builds, through a specific browser, the explicit domain ontology. The expert is then guided in creating the knowledge base according to the pre-established domain ontology and condition-action rule templates that are well adapted to several clinical decision-making processes. Corresponding medical logic modules are eventually generated. The application of this knowledge acquisition tool to the construction of a decision support system in blood transfusion demonstrates the value of such a pragmatic methodology for the design of rule-based clinical systems that rely on the highly progressive knowledge embedded in hospital information systems.

Guiding the decision to transfuse

Guiding the decision to transfuse can improve transfusion practices. Effective processes must first identify problem(s) in transfusion practice and then include the attending physician as an educational target. Process improvements that have been shown to be effective include the following: (1) briefly meeting one-on-one with physicians, (2) teaching at scheduled conferences, (3) making daily clinical rounds of patients who receive transfusion, (4) concurrently reviewing orders for transfusion before issue of the blood product, and (5) installing algorithms and guidelines in the operating room. Transfusion practices improved with these process improvements.

Contrasting views of physicians and nurses about an inpatient computer-based provider order-entry system

OBJECTIVE

Many hospitals are investing in computer-based provider order-entry (POE) systems, and providers' evaluations have proved important for the success of the systems. The authors assessed how physicians and nurses viewed the effects of one modified commercial POE system on time spent patients, resource utilization, errors with orders, and overall quality of care.

DESIGN

Survey.

MEASUREMENTS

Opinions of 271 POE users on medicine wards of an urban teaching hospital: 96 medical house officers, 49 attending physicians, 19 clinical fellows with heavy inpatient loads, and 107 nurses.

RESULTS

Responses were received from 85 percent of the sample. Most physicians and nurses agreed that orders were executed faster under POE. About 30 percent of house officers and attendings or fellows, compared with 56 percent of nurses, reported improvement in overall quality of care with POE. Forty-four percent of house officers and 34 percent of attendings/fellows reported that their time with patients decreased, whereas 56 percent of nurses indicated that their time with patients increased (P < 0.001). Sixty percent of house officers and 41 percent of attendings/fellows indicated that order errors increased, whereas 69 percent of nurses indicated a decrease or no change in errors. Although most nurses reported no change in the frequency of ordering tests and medications with POE, 61 percent of house officers reported an increased frequency.

CONCLUSION

Physicians and nurses had markedly different views about effects of a POE system on patient care, highlighting the need to consider both perspectives when assessing the impact of POE. With this POE system, most nurses saw beneficial effects, whereas many physicians saw negative effects.

The transfusion audit as an educational tool

Audits of transfusion used as educational tools can improve transfusion practices. Effective audits must first identify problem(s) in transfusion practice and must then include as educational target, the attending physician. Educational methods that have been shown to the effective include: (1) meeting briefly one-on-one with physicians, (2) teaching at scheduled conferences, (3) making daily clinical rounds on patients who receive transfusion, (4) concurrent review of orders for transfusion prior to issue of the blood product and (5) installing algorithms and guidelines in the operating room. Transfusion practices improved with these educational audit methods.

Reduction of inappropriate use of blood products by prospective monitoring of transfusion request forms

OBJECTIVE

To determine the effect of prospective monitoring on appropriateness of transfusions of red cells, platelets and fresh frozen plasma (FFP).

DESIGN

Prospective interventional study.

SETTING

Royal Melbourne Hospital (a tertiary teaching hospital), Melbourne, Victoria, March-May 1996.

INTERVENTION

The blood product request form was modified to incorporate indications for transfusion and clinical and laboratory data. Requests were monitored by blood bank laboratory staff for conformation with hospital transfusion guidelines; non-conforming requests were discussed with the requesting medical practitioner by the Haematology Registrar before blood products were issued. In case of disagreement, blood products were always issued.

SUBJECTS

200 consecutive transfusion episodes for each product (red cells, platelets and FFP).

OUTCOME MEASURES

Appropriateness of transfusion, assessed by a Consultant Haematologist according to hospital guidelines. Rates of inappropriate transfusion episodes after intervention were compared with rates in a previous study.

RESULTS

After intervention, rates of inappropriate transfusion episodes fell significantly (red cells, 16% to 3% [P = 0.004]; platelets, 13% to 2.5% [P = 0.02]; and FFP, 31% to 15% [P = 0.02]). Almost all inappropriate FFP transfusion episodes post-intervention were due to failure to demonstrate prolongation of prothrombin or activated partial thromboplastin times more than 1.5 times the control value.

CONCLUSION

Prospective monitoring of request forms can reduce rates of inappropriate transfusions. High rates of inappropriate FFP transfusions possibly reflect uncertainty about appropriate laboratory criteria for FFP transfusion. While results of large prospective randomised controlled clinical trials of FFP transfusions are awaited, currently laboratory criteria can be retained, but should be applied with flexibility.

Recommendations for responsible monitoring and regulation of clinical software systems. American Medical Informatics Association, Computer-based Patient Record Institute, Medical Library Association, Association of Academic Health Science Libraries, American Health Information Management Association, American Nurses Association

In mid-1996, the FDA called for discussions on regulation of clinical software programs as medical devices. In response, a consortium of organizations dedicated to improving health care through information technology has developed recommendations for the responsible regulation and monitoring of clinical software systems by users, vendors, and regulatory agencies. Organizations assisting in development of recommendations, or endorsing the consortium position include the American Medical Informatics Association, the Computer-based Patient Record Institute, the Medical Library Association, the Association of Academic Health Sciences Libraries, the American Health Information Management Association, the American Nurses Association, the Center for Healthcare Information Management, and the American College of Physicians. The consortium proposes four categories of clinical system risks and four classes of measured monitoring and regulatory actions that can be applied strategically based on the level of risk in a given setting. The consortium recommends local oversight of clinical software systems, and adoption by healthcare information system developers of a code of good business practices. Budgetary and other constraints limit the type and number of systems that the FDA can regulate effectively. FDA regulation should exempt most clinical software systems and focus on those systems posing highest clinical risk, with limited opportunities for competent human intervention.

Evaluating laboratory usage in the intensive care unit: patient and institutional characteristics that influence frequency of blood sampling

OBJECTIVES

To develop a predictive equation to estimate the frequency of blood drawing for intensive care unit (ICU) laboratory tests and to evaluate variations in ICU blood sampling practices after adjusting for patient and institutional factors.

DESIGN

Prospective, inception, cohort study.

SETTING

Forty-two ICUs in 40 hospitals, including 20 teaching and 17 nonteaching ICUs.

PATIENTS

A consecutive sample of 17,440 ICU admissions, in which 14,043 blood samples were drawn for laboratory testing on ICU days 2 to 7.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Patient demographic, physiologic, and treatment data were obtained on ICU day 1; the type and number of blood samples for laboratory testing were recorded on ICU days 1 to 7. In the 42 ICUs, a mean of 16.2 blood samples were drawn for tests on ICU days 2 to 7, but varied between 23 samples in the teaching ICUs and 9.9 samples in nonteaching ICUs. Using only ICU day 1 patient data, we predicted the subsequent number of samples drawn on ICU day 2 (R2 = .26 across individual patients) and on ICU days 2 to 7 (R2 = .26 across individual patients). The most important determinants of the number of blood samples drawn on ICU days 2 to 7 were the ICU day 1 Acute Physiology Score and admission diagnosis. After controlling for patient variables, hospital teaching status, number of beds, and location in the East and South were significantly (p < .05) associated with increased blood sampling on ICU day 2 and on ICU days 2 to 7. More frequent use of an arterial cannula and mechanical ventilation were also associated with increased blood sampling on subsequent days.

CONCLUSIONS

The ability to adjust for patient and institutional variables and to predict the number of blood samples drawn for laboratory tests can allow ICUs to compare their practices with those of other units. When integrated into a continuous quality improvement process, this information can be used to identify and focus on opportunities for improving blood conservation and reducing excessive diagnostic testing.

Integrating pharmacokinetics into point-of-care information systems

Computer-based patient care information systems (PCIS) have emerged as an integral component of healthcare organisations. Currently, 4 models of PCIS exist: the centralised model, the hub-and-spoke model, the network model, and the distributed model. The centralised model has the advantage of a central patient database; however, a major disadvantage of this model is the inability to easily interface with other software packages. The hub-and-spoke model links satellite or feeder systems into a mainframe computer; thus, each satellite has the ability to work independently. This system is limited by the ability to interface satellite systems with the mainframe computer. The network model works via a local area network (LAN) using client server technology which allows for high speed data access and transfer. The network model does not provide an integrated view of patient information and can access only 1 host system at a time. The distributed model is similar to the network model in design but provides for data and system integration via relational databases. This allows for the creation of a central data repository and support for decision-support tools. Computer-assisted decision support has the potential to significantly improve clinical decision-making. Six types of computer-assisted decision-support have been defined: alerting, interpreting, assisting, critiquing, diagnosing and managing. Software representing each type of decision-support software has been incorporated into clinical practice; however, with the exception of drug interaction programs, widespread incorporation of decision-support software into PCIS is uncommon. Clinical pharmacokinetic programs are a category of pharmacy-related decision-support software, and current clinical pharmacokinetic software systems can be categorised as interpreting, assisting or critiquing decision-support. Despite the potential for significant clinical contributions, the integration of clinical pharmacokinetic software into PCIS is uncommon. Most packages are available only as stand alone programs or as a module of a pharmacy information system. These packages usually maintain their own centralised database and require special file transfer protocols for integration. Although PCIS are becoming more commonplace, the integration of commercial clinical pharmacokinetic packages into PCIS is limited. New technology using standardised and relational databases should allow for easier integration in the future.

Audit and education in transfusion medicine

Because transfusions carry risks to the patient and because inappropriate transfusions are costly, interest in audits and effective education in transfusion medicine has increased over the last decade. Audits identify areas of practice that can be improved by follow-up education of the physicians who prescribe the transfusions. Successful educational approaches to follow-up on problems identified by audit include 30-min one-on-one meetings with surgeons, traditional scheduled teaching conferences, daily clinical rounds on transfused patients, prospective review of blood transfusions and installation of transfusion practice algorithms in the operating room. Other than identifying inappropriate transfusions, audit and education have also been used successfully to improve bedside blood administration practices, decrease unnecessary crossmatches and reduce outdating of donor blood. Multi-institutional audits play a useful benchmarking role. In summary, audit followed by targeted education can improve practices in transfusion medicine.

The role of medical informatics in telemedicine

The role of medical informatics in telemedicine is dependent on using the power of the computerized database to not only feed patient specific information to the health care providers, but to use the epidemiological and statistical information in the data base to improve decision making and ultimately care. The computer is also a powerful tool to facilitate standardizing and monitoring of care and when applied in continuous quality improvement methodology it can enhance the improvement process well beyond what can be done by hand. The coupling of medical informatics with telemedicine allows sophisticated medical informatics systems to be applied in low population density and remote areas.

An event model of medical information representation

OBJECTIVE

Develop a model for structured and encoded representation of medical information that supports human review, decision support applications, ad hoc queries, statistical analysis, and natural-language processing.

DESIGN

A medical information representation model was developed from manual and semiautomated analysis of patient data. The key assumption of the model is that medical information can be represented as a series of linked events. The event representation has two main components. The first component is a frame or template definition that specifies the attributes of the event. The second component is a structured vocabulary, the terms of which are taken as the values of the slots in the event template structure. Individual event instances are linked by specific named relationships.

RESULTS

The proposed model was used to represent a chest-radiograph report.

CONCLUSIONS

The event model of medical information representation provides a mechanism for formal definition of the logical structure of medical data and allows explicit time-oriented and associative relationships between event instances.

Evaluation of user acceptance of a clinical expert system

OBJECTIVE

To measure the attitudes of physicians and nurses who use the Health Evaluation through Logical Processing (HELP) clinical information system.

DESIGN

Questionnaire survey of 360 attending physicians and 960 staff nurses practicing at the LDS Hospital. The physicians' responses were signed, permitting follow-up for nonresponse and use of demographic data from staff files. The nurses' responses were anonymous and their demographic data were obtained from the questionnaires.

MEASUREMENTS

Fixed-choice questions with a Likert-type scale, supplemented by free-text comments. Question categories included: computer experience; general attitudes about impact of the system on practice; ranking of available functions; and desired future capabilities.

RESULTS

The response rate was 68% for the physicians and 39% for the nurses. Age, specialty, and general computer experience did not correlate with attitudes. Access to patient data and clinical alerts were rated highly. Respondents did not feel that expert computer systems would lead to external monitoring, or that these systems might compromise patient privacy. The physicians and nurses did not feel that computerized decision support decreased their decision-making power.

CONCLUSION

The responses to the questionnaire and "free-text comments" provided encouragement for future development and deployment of medical expert systems at LDS Hospital and sister hospitals. Although there has been some fear on the part of medical expert system developers that physicians would not adapt to or appreciate recommendations given by these systems, the results presented here are promising and may be of help to other system developers and evaluators.

Computer-based physician order entry: the state of the art

Direct computer-based physician order entry has been the subject of debate for over 20 years. Many sites have implemented systems successfully. Others have failed outright or flirted with disaster, incurring substantial delays, cost overruns, and threatened work actions. The rationale for physician order entry includes process improvement, support of cost-conscious decision making, clinical decision support, and optimization of physicians' time. Barriers to physician order entry result from the changes required in practice patterns, roles within the care team, teaching patterns, and institutional policies. Key ingredients for successful implementation include: the system must be fast and easy to use, the user interface must behave consistently in all situations, the institution must have broad and committed involvement and direction by clinicians prior to implementation, the top leadership of the organization must be committed to the project, and a group of problem solvers and users must meet regularly to work out procedural issues. This article reviews the peer-reviewed scientific literature to present the current state of the art of computer-based physician order entry.

Medical informatics in the intensive care unit: state of the art 1991

Intensive care medicine requires timely, accurate, and integrated patient records to provide the highest quality patient care. Computerized patient records offer the best method to achieve these needs. The expectations of society for medical progress through increased use of computers is growing. For optimal use of computers in the ICU there must be a harmonious collaboration between medical informaticists, physicians, nurses, therapists, and administrators. The future use of computers in ICU care will be evolutionary rather than revolutionary. We are on the frontier of some exciting times in the next decade as computers become commonplace in the clinical care process rather than an unusual event. This paper discusses the progress and challenges of computers in the ICU.