Current Challenges and Opportunities for Better Integration of Human Factors Research with Development of Clinical Information Systems

Objectives Clinical information system (CIS) developers and implementers have begun to look to other scientific disciplines for new methods, tools, and techniques to help them better understand clinicians and their organizational structures, clinical work environments, capabilities of clinical information and communications technology, and the way these structures and processes interact. The goal of this article is to help CIS researchers, developers, implementers, and evaluators better understand the methods, tools, techniques, and literature of the field of human factors.

Methods We developed a framework that explains how six key human factors topics relate to the design, implementation, and evaluation of CISs.

Results Using this framework we discuss the following six topics: 1) informatics and patient safety; 2) user interface design and evaluation; 3) workflow and task analysis; 4) clinical decision making and decision support; 5) distributed cognition; and 6) mental workload and situation awareness.

Conclusions Integrating the methods, tools, and lessons learned from each of these six areas of human factors research early in CIS design and incorporating them iteratively during development can improve user performance, user satisfaction, and integration into clinical workflow. Ultimately, this approach will improve clinical information systems and healthcare delivery.

State of the Art in Clinical Informatics: Evidence and Examples

Objective: The field of clinical informatics has expanded substantially in the six decades since its inception. Early research focused on simple demonstrations that health information technology (HIT) such as electronic health records (EHRs), computerized provider order entry (CPOE), and clinical decision support (CDS) systems were feasible and potentially beneficial in clinical practice.

Methods: In this review, we present recent evidence on clinical informatics in the United States covering three themes: 1) clinical informatics systems and interventions for providers, including EHRs, CPOE, CDS, and health information exchange; 2) consumer health informatics systems, including personal health records and web-based and mobile HIT; and 3) methods and governance for clinical informatics, including EHR usability; data mining, text mining, natural language processing, privacy, and security.

Results: Substantial progress has been made in demonstrating that various clinical informatics methodologies and applications improve the structure, process, and outcomes of various facets of the healthcare system.

Conclusion: Over the coming years, much more will be expected from the field. As we move past the “early adopters” in Rogers' diffusion of innovations' curve through the “early majority” and into the “late majority,” there will be a crucial need for new research methodologies and clinical applications that have been rigorously demonstrated to work (i.e., to improve health outcomes) in multiple settings with different types of patients and clinicians.

A Quantitative Ranking of the Biomedical Informatics Serials

We have developed a quantitative serial ranking system based on multiple citation analysis techniques, library use statistics, expert opinion, and selected distinguishing publication characteristics. Evaluation criteria categories include: average Science Citation Index (Impact Factor, Immediacy Index, Total citations) rankings from 1987 to 1992; citation source counts of multiple “core” biomedical informatics publications; a questionnaire sent to American College of Medical Informatics Fellows; publication delay; distinguishing characteristics (e. g., subscription cost, total circulation, year established, places indexed, affiliation with a professional society, major biomedical resource library holdings); and the total number of interlibrary loan requests to the U. S. National Library of Medicine. The top serials were Computers and Biomedical Research, MD Computing, Methods of Information in Medicine, Medical Decision Making and Computers in Biology and Medicine.

Work-Sampling: A Statistical Approach to Evaluation of the Effect of Computers on Work Patterns in Healthcare

An increasing number of health-care institutions are in the process of implementing clinical computing systems. The need for an accurate assessment of the clinical, administrative, social, and financial effects of such systems has been recognized. Techniques have been developed to evaluate these effects on the work patterns of health-care workers including: time-motion analysis, subjective evaluations, review of departmental statistics, personal activity records, and work-sampling. This study reviews these techniques, discusses both positive and negative aspects, and presents a step-by-step description of work-sampling.

The Multi-Trellis Software Architecture and the Intelligent Cardiovascular Monitor

A real-time, intelligent cardiovascular monitor is complex. It must process multiple waveforms, recognize artifacts, extract pertinent parameters, recognize a patient’s clinical state, analyze the problem and formulate a response. This paper presents the multi-trellis (a collection of process trellises), a software architecture for building such a monitor. A process trellis is a uniform hierarchical framework for heterogeneous program modules. The multi-trellis extension allows one to compile several process trellis programs with widely varying run-time requirements into a single executable program that it is efficient, predictable and usable. Our prototype consists of two process trellises. The lower trellis contains processes to analyze three different analog signals: the blood pressure from a non-invasive monitor and an arterial catheter, and the ECG. The upper trellis contains processes to help detect evolving hemodynamic trends, identify abnormalities, and present a succinct summary to the clinician. Our prototype shows that the multi-trellis is a demonstrably useful software architecture for building these real-time, intelligent monitors.

Grand Challenges in Dental Informatics

When truly significant scientific challenges are overcome, it profoundly changes the daily activities, as well as the future research activities, of everyone involved in the related field. By identifying and describing the grand challenges facing a scientific field, we can help funding agencies identify and prioritize projects for support, stimulate and encourage new investigators to work on these intellectual and technological challenges, and help define the field itself. In this article, we present an informatics-oriented, future-patient-care scenario, then describe a series of applications and the related informatics grand challenges facing the dental field today. New techniques and technologies to help us overcome these challenges would facilitate the development of truly monumental applications, such as a comprehensive electronic oral health record, an automated dental treatment planning system for all diagnoses, or a system to profile patient risk for chronic oral diseases.

New Unintended Adverse Consequences of Electronic Health Records

Although the health information technology industry has made considerable progress in the design, development, implementation, and use of electronic health records (EHRs), the lofty expectations of the early pioneers have not been met. In 2006, the Provider Order Entry Team at Oregon Health & Science University described a set of unintended adverse consequences (UACs), or unpredictable, emergent problems associated with computer-based provider order entry implementation, use, and maintenance. Many of these originally identified UACs have not been completely addressed or alleviated, some have evolved over time, and some new ones have emerged as EHRs became more widely available. The rapid increase in the adoption of EHRs, coupled with the changes in the types and attitudes of clinical users, has led to several new UACs, specifically: complete clinical information unavailable at the point of care; lack of innovations to improve system usability leading to frustrating user experiences; inadvertent disclosure of large amounts of patient-specific information; increased focus on computer-based quality measurement negatively affecting clinical workflows and patient-provider interactions; information overload from marginally useful computer-generated data; and a decline in the development and use of internally-developed EHRs. While each of these new UACs poses significant challenges to EHR developers and users alike, they also offer many opportunities. The challenge for clinical informatics researchers is to continue to refine our current systems while exploring new methods of overcoming these challenges and developing innovations to improve EHR interoperability, usability, security, functionality, clinical quality measurement, and information summarization and display.

Clinical Decision Support: a 25 Year Retrospective and a 25 Year Vision

OBJECTIVE

The objective of this review is to summarize the state of the art of clinical decision support (CDS) circa 1990, review progress in the 25 year interval from that time, and provide a vision of what CDS might look like 25 years hence, or circa 2040.

METHOD

Informal review of the medical literature with iterative review and discussion among the authors to arrive at six axes (data, knowledge, inference, architecture and technology, implementation and integration, and users) to frame the review and discussion of selected barriers and facilitators to the effective use of CDS.

RESULT

In each of the six axes, significant progress has been made. Key advances in structuring and encoding standardized data with an increased availability of data, development of knowledge bases for CDS, and improvement of capabilities to share knowledge artifacts, explosion of methods analyzing and inferring from clinical data, evolution of information technologies and architectures to facilitate the broad application of CDS, improvement of methods to implement CDS and integrate CDS into the clinical workflow, and increasing sophistication of the end-user, all have played a role in improving the effective use of CDS in healthcare delivery.

CONCLUSION

CDS has evolved dramatically over the past 25 years and will likely evolve just as dramatically or more so over the next 25 years. Increasingly, the clinical encounter between a clinician and a patient will be supported by a wide variety of cognitive aides to support diagnosis, treatment, care-coordination, surveillance and prevention, and health maintenance or wellness.

Validation of a Crowdsourcing Methodology for Developing a Knowledge Base of Related Problem-Medication Pairs

BACKGROUND

Clinical knowledge bases of problem-medication pairs are necessary for many informatics solutions that improve patient safety, such as clinical summarization. However, developing these knowledge bases can be challenging.

OBJECTIVE

We sought to validate a previously developed crowdsourcing approach for generating a knowledge base of problem-medication pairs in a large, non-university health care system with a widely used, commercially available electronic health record.

METHODS

We first retrieved medications and problems entered in the electronic health record by clinicians during routine care during a six month study period. Following the previously published approach, we calculated the link frequency and link ratio for each pair then identified a threshold cutoff for estimated problem-medication pair appropriateness through clinician review; problem-medication pairs meeting the threshold were included in the resulting knowledge base. We selected 50 medications and their gold standard indications to compare the resulting knowledge base to the pilot knowledge base developed previously and determine its recall and precision.

RESULTS

The resulting knowledge base contained 26,912 pairs, had a recall of 62.3% and a precision of 87.5%, and outperformed the pilot knowledge base containing 11,167 pairs from the previous study, which had a recall of 46.9% and a precision of 83.3%.

CONCLUSIONS

We validated the crowdsourcing approach for generating a knowledge base of problem-medication pairs in a large non-university health care system with a widely used, commercially available electronic health record, indicating that the approach may be generalizable across healthcare settings and clinical systems. Further research is necessary to better evaluate the knowledge, to compare crowdsourcing with other approaches, and to evaluate if incorporating the knowledge into electronic health records improves patient outcomes.

Death, taxes and advance directives

Suboptimal care at the end-of-life can be due to lack of access or knowledge of patient wishes. Ambiguity is often the result of non-standardized formats. Borrowing digital technology from other industries and using existing health information infrastructure can greatly improve the completion, storage, and distribution of advance directives. We believe several simple, low-cost adaptations to regional and federal programs can raise the standard of end-of-life care.

Hospital and health information systems - Current perspectives. Contribution of the IMIA Health Information Systems Working Group

OBJECTIVE

To celebrate over 30 years of health information systems' (HIS) evolution by bringing together pioneers in the field, members of the next generation of leaders, and government officials from several developing nations in Africa to discuss the past, present, and future of HISs.

METHODS

Participants gathered in Le Franschhoek, South Africa for a 2 1/2 day working conference consisting of scientific presentations followed by several concurrent breakout sessions. A small writing group prepared draft statements representing their positions on various topics of discussion which were circulated and revised by the entire group.

RESULTS

Many new tools, techniques and technologies were described and discussed in great detail. Interestingly, all of the key themes identified in the first HIS meeting held over 30 years ago are still of vital importance today: Patient Centered design, Clinical User Support, Real-time Education, Human-computer Factors and Measuring Clinical User Performance, Meaningful use.

CONCLUSIONS

As we continue to work to develop next-generation HISs, we must remember the lessons of the past as we strive to develop the solutions for tomorrow.

Rapid assessment of clinical information systems in the healthcare setting: an efficient method for time-pressed evaluation

OBJECTIVE

Recent legislation in the United States provides strong incentives for implementation of electronic health records (EHRs). The ensuing transformation in U.S. health care will increase demand for new methods to evaluate clinical informatics interventions. Timeline constraints and a rapidly changing environment will make traditional evaluation techniques burdensome. This paper describes an anthropological approach that provides a fast and flexible way to evaluate clinical information systems.

METHODS

Adapting mixed-method evaluation approaches from anthropology, we describe a rapid assessment process (RAP) for assessing clinical informatics interventions in health care that we developed and used during seven site visits to diverse community hospitals and primary care settings in the U.S.

SETTING

Our multidisciplinary team used RAP to evaluate factors that either encouraged people to use clinical decision support (CDS) systems or interfered with use of these systems in settings ranging from large urban hospitals to single-practitioner, private family practices in small towns.

RESULTS

Critical elements of the method include: 1) developing a fieldwork guide; 2) carefully selecting observation sites and participants; 3) thoroughly preparing for site visits; 4) partnering with local collaborators; 5) collecting robust data by using multiple researchers and methods; and 6) analyzing and reporting data in a structured manner helpful to the organizations being evaluated.

CONCLUSIONS

RAP, iteratively developed over the course of visits to seven clinical sites across the U.S., has succeeded in allowing a multidisciplinary team of informatics researchers to plan, gather and analyze data, and report results in a maximally efficient manner.

A regression model for predicting conditional survival for head and neck cancer patients: A SEER analysis

6527

Background: Survival probability changes as patients survive longer periods of time after diagnosis, and estimates of prognosis at diagnosis are no longer valid. Conditional survival (CS) accounts for the changing hazard rate over time and is a more accurate estimate of survival probability for these cancer survivors. The specific aim of this project was to build a statistical model and web-based tool to predict conditional survival for an individual head & neck cancer patient based on tumor and patient characteristics. Methods: Using 27,825 patients diagnosed with head & neck cancer between 1988–97 from the Surveillance, Epidemiology, and End Results 17 (SEER) database, we built a multivariate Cox proportional hazards regression prediction model. Patient and tumor characteristics included as covariates were age, sex, race, tumor site, stage, and grade. The primary endpoint was conditional overall survival. The model was validated for discrimination using the concordance index and a calibration plot was constructed. Bootstrapping was used to correct for optimistic bias. We also built a web-browser software tool to allow a user to enter patient information into the model and calculate conditional survival probability. Results: The regression model showed very good calibration and discrimination with a bootstrap-corrected C-index of 0.71. For a 65-yr old white male with a moderately-differentiated tonsil cancer with regional lymph nodes, the model predicted that the 5-yr conditional overall survival would increase from 50% at the time of diagnosis to 63% at 3 years after diagnosis. For a 75-yr old black male with a well-differentiated localized lip cancer, 5-yr conditional survival would improve from 58% at diagnosis to 70% by 3 years from diagnosis. Conclusions: Our regression model can accurately predict conditional survival for head & neck cancer patients based upon specific patient and tumor characteristics. This tool allows the calculation of more specific prognosis predictions for individual cancer patients who have already survived a period of time after diagnosis and treatment.

No significant financial relationships to disclose.

The emerging science of very early detection of disease outbreaks

A surge of development of new public health surveillance systems designed to provide more timely detection of outbreaks suggests that public health has a new requirement: extreme timeliness of detection. The authors review previous work relevant to measuring timeliness and to defining timeliness requirements. Using signal detection theory and decision theory, the authors identify strategies to improve timeliness of detection and position ongoing system development within that framework.

A survey of patient-provider e-mail communication: what do patients think?

Communication between patients and providers forms the backbone of the patient-provider relationship. Often such communication is strained due to time and space limitations on the part of both patients and providers. Many healthcare organizations are developing secure e-mail communication facilities to allow patients to exchange e-mail messages with their providers. Providers are worried that opening such lines of communication will inundate them with vast quantities of e-mail from their patients. Patients are worried that their messages will be intercepted and read by unauthorized people. In an attempt to determine how a group of internet-active, e-mail-ready patients currently use, or potentially view, the ability to exchange e-mail messages with their health care providers, we distributed a survey via e-mail to over 9500 patients. After determining each patient's e-mail activity level (based on the number of messages sent each day), we asked questions such as: "Have you ever sent e-mail to your provider?" "What issues or concerns have prevented you from sending e-mail messages to your provider?" "If your provider were to tell you that someone in his/her office may screen, read or perhaps reply to your message before he/she sees it, to what extent would you be concerned about this?" and "How would you rate your overall satisfaction with the use of e-mail to communicate with your provider?" Results from the survey indicate that nearly 85% of the patients surveyed send at least one e-mail message per day, but that very few (i.e. 6%) of the patients have actually sent an e-mail message to their provider. Interestingly, over half of the patients indicated that they would like to send their providers e-mail, but that they do not know their provider's e-mail address.

Subjective assessment of usefulness and appropriate presentation mode of alerts and reminders in the outpatient setting

There is very little known about the limits of alerting in the setting of the outpatient Electronic Medical Record (EMR). We are interested in how users value and prefer such alerts. One hundred Kaiser Permanente primary care clinicians were sent a four-page questionnaire. It contained questions related to the usability and usefulness of different approaches to presenting reminder and alert information. The survey also contained questions about the desirability of six categories of alerts. Forty-three of 100 questionnaires were returned. Users generally preferred an active, more intrusive interaction model for "alerts" and a passive, less intrusive model for order messages and other types of reminders and notifications. Drug related alerts were more highly rated than health maintenance or disease state reminders. Users indicated that more alerts would make the system "more useful" but "less easy to use".

Communication breakdown in the outpatient referral process

OBJECTIVE

To evaluate primary care and specialist physicians' satisfaction with interphysician communication and to identify the major problems in the current referral process.

DESIGN

Surveys were mailed to providers to determine satisfaction with the referral process; then patient-specific surveys were e-mailed to this group to obtain real-time referral information.

SETTING

Academic tertiary care medical center.

PARTICIPANTS

Attending-level primary care physicians (PCPs) and specialists.

MEASUREMENTS AND MAIN RESULTS

The response rate for mail surveys for PCPs was 57% and for specialists was 51%. In the mail survey, 63% of PCPs and 35% of specialists were dissatisfied with the current referral process. Respondents felt that major problems with the current referral system were lack of timeliness of information and inadequate referral letter content. Information considered important by recipient groups was often not included in letters that were sent. The response rate for the referral specific e-mail surveys was 56% for PCPs and 53% for specialists. In this e-mail survey, 68% of specialists reported that they received no information from the PCP prior to specific referral visits, and 38% of these said that this information would have been helpful. In addition, four weeks after specific referral visits, 25% of PCPs had still not received any information from specialists.

CONCLUSIONS

Substantial problems were present in the referral process. The major issues were physician dissatisfaction, lack of timeliness, and inadequate content of interphysician communication. Information obtained from the general survey and referral-specific survey was congruent. Efforts to improve the referral system could improve both physician satisfaction and quality of patient care.

Techniques for identifying the applicability of new information management technologies in the clinical setting: an example focusing on handheld computers

This article describes techniques and strategies used to judge the potential applicability of new information management technologies in the clinical setting and to develop specific design recommendations for new features and services. We focus on a project carried out to identify the potential uses of handheld computers (i.e., the Palm Pilot or a small WinCE-based device) in the ambulatory practice setting. We found that the potential for a robust handheld computing device to positively affect the outpatient ambulatory clinical setting is enormous, and that the information derived from the exploratory research project is useful in creating specific design recommendations for further development.

A computer-based outpatient clinical referral system

The process of generating a clinical referral for a patient, and the resulting transfer of information from the primary care physician to the specialist and back again, are key components in the struggle to deliver less costly and more effective clinical care. We have created a computer-based, outpatient clinical referral application that facilitates: (1) identifying an appropriate specialist; (2) collecting the clinical, demographic, and financial data required to generate a referral; and (3) transferring the information between the specialist and the primary care physician (PCP). This article describes the development of the application itself and several of the knowledge bases that were created to facilitate this process. Preliminary results indicate that the new computer-based referral process is faster to use than conventional methods.

Components of the optimal ambulatory care computing environment

We present here a framework of core components of an ambulatory care computing environment, based on clinical and functional needs and workflow scenarios. We have established this framework through the use of two study devices: a vision of the clinical office of the future, and a survey of possible computer applications, both designed to help clinicians and practice directors communicate their information needs to systems designers. Clinicians prioritize applications based on strategic and practice goals: support for clinical users' workflow, improved quality of care, reduced cost of care, and the ability to measure performance and status. By reorganizing the needed functionality from a clinical viewpoint into a technical viewpoint, we are able to identify core information components for systems design. Based on this analysis, information needs in the ambulatory environment can be divided into five primary functions: patient data retrieval, documentation, communication, knowledge resources, and aggregate reporting. Three other fundamental processes--knowledge-based interventions, information integration, and confidentiality--run through all of these front-line functions. Component applications and data structures built with this framework in mind will afford a maximum combination of functionality and flexibility to handle future changes in the clinical environment.

Design and development of a computer-based clinical referral system for use within a physician hospital organization

The process of creating a clinical referral for a patient and the transfer of information from the primary care physician to the specialist and back again is a key component in the struggle to deliver less costly and more effective clinical care. We have created a computer-based clinical referral application which facilitates 1) identifying an appropriate specialist; 2) collecting the clinical, demographic, and financial data required to generate a referral; and 3) transferring the information between the specialist and the primary care physician. Preliminary results indicate that the new computer-based process is faster.

Use of fuzzy set theory to extend Dhawan's journal selection model: ranking the biomedical informatics serials

OBJECTIVE

Experts disagree on the parameters to use to identify the "best" serials within a scientific field. The author set out to develop an extension to Dhawan's journal selection model for ranking serials in any scientific field.

METHODS

Comparison of three different instantiations of Dhawan's model were used to rank thirty-four biomedical informatics serials.

RESULTS

The first instantiation of Dhawan's model identified seven serials and divided them into two groups. The second instantiation of Dhawan's model identified twelve serials and separated them into two groups. Using fuzzy set theory the new extended model produced a rank ordered list of the top twelve biomedical informatics serials.

CONCLUSIONS

Use of fuzzy set theory to assign set membership and combine data in Dhawan's journal selection model allows one to: (1) eliminate the need to determine arbitrary cutoff points for inclusion of serials within each of Dhawan's evaluation criteria categories, (2) combine data from disparate sources, and (3) obtain a rank-ordered list of the biomedical informatics serials rather than simply identifying a set of the "top" serials. Such a ranked list provides librarians and researchers alike with the information necessary to help them make their biomedical informatics serial selection decisions based on objective, quantifiable data.

Evaluating physician satisfaction regarding user interactions with an electronic medical record system

A limiting factor in realizing the full potential of electronic medical records (EMR) is physician reluctance to use these applications. There have been very few formal usability studies of experienced physician users of EMRs in routine clinical use. We distributed the Questionnaire for User Interaction Satisfaction (QUIS) to 75 primary care physicians who routinely use the Brigham and Women's Integrated Computing System (BICS). BICS scored highest in the area of screen design and lowest in the area of system capability. Overall user satisfaction was most highly correlated with screen design and layout, and surprisingly not with system response time. Human-computer interaction studies can help focus our design efforts as we strive to increase clinician usage of information technology.

Modifiable templates facilitate customization of physician order entry

Physician order entry is a key factor in improving the quality of healthcare, while simultaneously reducing its cost. This paper describes an editor, a database, and a run-time system for creating and executing highly customized, user modifiable, order entry templates. The system allows non-programmers to create new order entry templates rapidly. Over the past 18 months, the templates have been used on over 2500 patients to enter over 40,000 separate orders.

Using web technology and Java mobile software agents to manage outside referrals

A prototype, web-based referral application was created with the objective of providing outside primary care providers (PCP's) the means to refer patients to the Massachusetts General Hospital and the Brigham and Women's Hospital. The application was designed to achieve the two primary objectives of providing the consultant with enough data to make decisions even at the initial visit, and providing the PCP with a prompt response from the consultant. The system uses a web browser/server to initiate the referral and Java mobile software agents to support the workflow of the referral. This combination provides a light client implementation that can run on a wide variety of hardware and software platforms found in the office of the PCP. The implementation can guarantee a high degree of security for the computer of the PCP. Agents can be adapted to support the wide variety of data types that may be used in referral transactions, including reports with complex presentation needs and scanned (faxed) images Agents can be delivered to the PCP as running applications that can perform ongoing queries and alerts at the office of the PCP. Finally, the agent architecture is designed to scale in a natural and seamless manner for unforeseen future needs.

A software architecture to support a large-scale, multi-tier clinical information system

A robust software architecture is necessary to support a large-scale multi-tier clinical information system. This paper describes our mechanism for enterprise distribution of applications and support files, the consolidation of data-access functions and system utilities stored on the data access tier, and an application framework which implements a coherent clinical computing environment. The software architecture and systems described in this paper have been robust through pilot testing of our applications at Massachusetts General Hospital.

A graphical user interaction model for integrating complex clinical applications: a pilot study

We have developed and implemented a multi-faceted, graphical user interaction model for an advanced clinical information system. This paper describes a classification scheme for applications used by clinicians in their daily work, discusses the way clinicians interact with these applications, and the issues that arise during these user interactions. Through its emphasis on support for application interoperation, the graphical user interface that implements the model presents a single, consistent, context to the user, and thereby helps maintain patient safety and ensure ease of use.

Preserving context in a multi-tasking clinical environment: a pilot implementation

The Partners Clinical Application Suite (CAS) is a multi-tasking software architecture that facilitates the development, deployment, and use of advanced clinical information management applications. This paper describes 1) a software shell in which clinical applications run; 2) an application programming interface (API); and 3) development of a set of "Look & Feel" guidelines. Through its emphasis on support for multi-tasking and application interoperability, CAS facilitates preservation of the user's context.

Re-engineering healthcare: computer tools support the definition and analysis of work at the Vanderbilt University Hospital and Clinic

In November 1993 The Vanderbilt University Hospital and Clinic (VUH/TVC) convened a 10-member Collaborative Organization Design (COD) team comprised of a multi-disciplinary team representing a diagonal slice through the organization. This team, lead by Gelinas * James, Inc. a consulting firm specializing in restructuring, was charged to develop, recommend, and implement a new organizational design that would promote stronger patient focus, increased efficiency, and lower cost. The COD process is structured to inspire and enable employees to rebuild their organization to respond to the challenges and opportunities that exist within their environment, to customer needs, and their own aspirations. This manuscript presents an overview of (1) the computer tools developed and/or employed to support the re-engineering process, and (2) the findings obtained as the work of patient care was defined and analyzed at VUH/TVC.

Identifying a core set of medical informatics serials: an analysis using the MEDLINE database

A study was undertaken to test the hypothesis that a core set of medical informatics serials could be identified by using standard bibliometric techniques. All journal articles indexed by the National Library of Medicine between 1990 and 1994 were included. Articles were identified by using the "MEDICAL INFORMATICS" Medical Subject Heading (MeSH) term. Each serial title containing articles was then ranked according to (1) the total number of medical informatics journal articles indexed and (2) the percentage of medical informatics journal articles indexed. Twenty-eight serials had more than 100 articles indexed under the "MEDICAL INFORMATICS" MeSH term. Thirty serials had more than 40% of their articles indexed under the "MEDICAL INFORMATICS" MESH term. A "core" set of fourteen serials had 100 or more medical informatics articles indexed, including more than 70% of all articles they published. The methodology described provides librarians with another tool to use in the difficult task of journal selection. The set of "core" serials identified provides librarians with a ranked list of serials, based on which a medical informatics collection can be developed.

WWW-based interfaces to clinical information systems: the state of the art

Nine WWW-based interfaces to clinical information systems were reviewed. Five have progressed past the proof of concept phase and into alpha testing in the clinical environment. All key features desirable in an advanced clinical information were present in at least one interface, however many implementations were rudimentary. Much human computer interface research and WWW tool development needs to occur before implementation of a WWW-based interface to a clinical information system should be considered for a mission-critical, production environment.

A quantitative ranking of the Biomedical Informatics serials

We have developed a quantitative serial ranking system based on multiple citation analysis techniques, library use statistics, expert opinion, and selected distinguishing publication characteristics. Evaluation criteria categories include: average Science Citation Index (Impact Factor, Immediacy Index, Total citations) rankings from 1987 to 1992; citation source counts of multiple "core" biomedical informatics publications; a questionnaire sent to American College of Medical Informatics Fellows; publication delay; distinguishing characteristics (e.g., subscription cost, total circulation, year established, places indexed, affiliation with a professional society, major biomedical resource library holdings); and the total number of interlibrary loan requests to the U. S. National Library of Medicine. The top serials were Computers and Biomedical Research, MD Computing, Methods of Information in Medicine, Medical Decision Making and Computers in Biology and Medicine.

Developing interactive computer-based simulations: an object-oriented development methodology enhances computer-assisted instruction

The purpose of this research was to investigate the application of object-oriented technology and AI techniques to enhance development of computer-based training simulations. Towards that end, a comprehensive computer-assisted instructional unit was developed to teach the skills and concepts of window-based applications, the OS/2 desktop, and the use of a patient care information system. By taking advantage of sophisticated computer graphics for the visual representation of objects and the behavioral modeling capabilities of the object-oriented language, domain knowledge modeling and human-computer interactions were implemented without complex natural language processing techniques. The results of this research indicate that nurses and physicians are able to learn the basic skills and concepts of computer systems and how to query for patient information. The new methodology described for building these computer-assisted instructional simulations significantly eased the training and teaching of large numbers of nurses and physicians and simplified their transition to a complex, computer-based hospital information system environment.

An innovative dicrotic notch detection algorithm which combines rule-based logic with digital signal processing techniques

Automated, real-time localization of the dicrotic notch, a component of the arterial pressure waveform, represents a deceptively complex problem in computerized biomedical signal processing. The high-frequency nature of the notch can make it difficult to distinguish from artifactual noise or from other high-frequency physiological components of the waveform. In addition, the contour of the notch varies with vascular status and with propagation through arterial beds, requiring any detection algorithm to recognize various possible notch conformations. Finally, location of the notch along the waveform may vary widely depending on other hemodynamic variables, further complicating detection algorithms. We have reviewed various published algorithms and have implemented a number of them to determine the strengths and shortcomings of each. We then developed a reliable and accurate hybrid algorithm which utilizes the strengths of the various algorithmic approaches reviewed; after analyzing the waveform, the algorithm selects the most appropriate method for accurate notch localization based on a series of waveform features. The application of rule-based logic represents a relatively unique approach to digital signal processing.

Building a data foundation for tomorrow's healthcare information management systems

Accurate, timely data are a necessary foundation for an effective healthcare information management system. Today's systems do not achieve their potential because they lack this essential ingredient. Technology can reduce the problem by making the data entry task easier, but a complete solution will require new strategies for system implementation and use. Functions must be implemented in a sequence that allows the initial functions to establish a data foundation for subsequent functions. Each atomic data item must be collected separately, with a consistent definition. Data capture strategies must permit data to be collected once, at the source. Finally, the data must be used, reused, and corrected by all members of the healthcare team.

Evaluating a computer-based experiential learning simulation: a case study using criterion-referenced testing

A case study of a computer-based experiential learning program involving the use of criterion-referenced testing as a quantitative evaluation tool is described. The simulation was designed to introduce nurses to basic computer operations. One hundred and thirty-eight nurses participated in the evaluation using a specially prepared version of the program that recorded the time required to perform actions, the number of unsuccessful attempts to complete instructions, and the number of correct responses to specific questions. The ability of the nurses to "drag and drop" increased significantly from less than 20% to more than 90% (p < .05, Student's t-test). Their knowledge of the definitions of four computer-related terms (i.e., trackball, desktop, window, and icon) increased significantly (from 65% to 80%, p < .1). In subjective evaluations, 82% of respondents reported that they enjoyed the lesson and 89% wanted more computer-based instructional units to be developed. Use of criterion-referenced testing helped to shed new light on the utility of such computer aided instructional systems.

The role of the information architect at King Faisal Specialist Hospital and Research Centre

Successful implementation of integrated clinical information system requires modification of the institution's long range strategic plans and its personnel's behavior. The changes warrant a concerted effort on the part of many different individuals; this paper describes the role of the Information Architect whose primary functions are to steer the process to fulfill stated objectives and build consensus where divergent forces are at work. The workings of the Architect is presented in context of a unique Middle-Eastern institution currently undergoing automation of clinical information.

Medical informatics on the Internet: creating the sci.med. informatics newsgroup

A Usenet newsgroup, sci.med.informatics, has been created to serve as an international electronic forum for discussion of issues related to medical informatics. The creation process follows a set of administrative rules set out by the Usenet administration on the Internet and consists of five steps: 1) informal discussion, 2) request for formal discussion, 3) formal discussion, 4) voting, and 5) posting of results. The newsgroup can be accessed using any news reader via the Internet.

Randomized clinical trial of pressure-controlled inverse ratio ventilation and extracorporeal CO2 removal for adult respiratory distress syndrome

The impact of a new therapy that includes pressure-controlled inverse ratio ventilation followed by extracorporeal CO2 removal on the survival of patients with severe ARDS was evaluated in a randomized controlled clinical trial. Computerized protocols generated around-the-clock instructions for management of arterial oxygenation to assure equivalent intensity of care for patients randomized to the new therapy limb and those randomized to the control, mechanical ventilation limb. We randomized 40 patients with severe ARDS who met the ECMO entry criteria. The main outcome measure was survival at 30 days after randomization. Survival was not significantly different in the 19 mechanical ventilation (42%) and 21 new therapy (extracorporeal) (33%) patients (p = 0.8). All deaths occurred within 30 days of randomization. Overall patient survival was 38% (15 of 40) and was about four times that expected from historical data (p = 0.0002). Extracorporeal treatment group survival was not significantly different from other published survival rates after extracorporeal CO2 removal. Mechanical ventilation patient group survival was significantly higher than the 12% derived from published data (p = 0.0001). Protocols controlled care 86% of the time. Average PaO2 was 59 mm Hg in both treatment groups. Intensity of care required to maintain arterial oxygenation was similar in both groups (2.6 and 2.6 PEEP changes/day; 4.3 and 5.0 FIO2 changes/day). We conclude that there was no significant difference in survival between the mechanical ventilation and the extracorporeal CO2 removal groups. We do not recommend extracorporeal support as a therapy for ARDS. Extracorporeal support for ARDS should be restricted to controlled clinical trials.

A quantitative method for measuring library user journal needs: a pilot study using CD plus MEDLINE usage statistics

OBJECTIVE

To develop a quantitative method for measuring library user journal needs based on an analysis of bibliographic search results.

DESIGN

Retrospective bibliometric comparison of citation selections generated by users in the library.

MEASURES

Number of times each journal was identified by library users during multiple bibliographic search sessions.

RESULTS

Library users identified 4907 journal titles. The top 200 journal titles accounted for 55% of the library user journal needs. Of the 1380 unique titles identified, 652 were selected once.

CONCLUSION

Our pilot study demonstrated that analysis of bibliographic search results can be used to identify library user journal needs. Such a method could also be used to estimate user requirements for online, full-text scientific journals.

Computer tools to support collaborative organization design: definition and analysis of the work at the Vanderbilt University Hospital and Clinic

In November, 1993 The Vanderbilt University Hospital and Clinic (VUH/TVC) convened a 10-member Collaborative Organization Design (COD) team that represented a diagonal slice through the organization. This team, lead by Gelinas & James, an outside consulting firm, was charged to develop, recommend, and implement a new organizational design which would promote a stronger patient focus, increased efficiency, and lower costs. The COD process is structured to inspire and enable employees to rebuild their organization so that it can respond to the challenges and opportunities that exist within their environment, to customer needs, and their own aspirations. This manuscript describes several of the computer tools which were utilized in the definition and analysis of the work of patient care at VUH/TVC. Specific examples of the findings from this phase of the work are utilized to illustrate their use and value.

Computer-based physician order entry: the state of the art

Direct computer-based physician order entry has been the subject of debate for over 20 years. Many sites have implemented systems successfully. Others have failed outright or flirted with disaster, incurring substantial delays, cost overruns, and threatened work actions. The rationale for physician order entry includes process improvement, support of cost-conscious decision making, clinical decision support, and optimization of physicians' time. Barriers to physician order entry result from the changes required in practice patterns, roles within the care team, teaching patterns, and institutional policies. Key ingredients for successful implementation include: the system must be fast and easy to use, the user interface must behave consistently in all situations, the institution must have broad and committed involvement and direction by clinicians prior to implementation, the top leadership of the organization must be committed to the project, and a group of problem solvers and users must meet regularly to work out procedural issues. This article reviews the peer-reviewed scientific literature to present the current state of the art of computer-based physician order entry.