A Nationwide Network of Health AI Assurance Laboratories

Importance

Given the importance of rigorous development and evaluation standards needed of artificial intelligence (AI) models used in health care, nationwide accepted procedures to provide assurance that the use of AI is fair, appropriate, valid, effective, and safe are urgently needed.

Observations

While there are several efforts to develop standards and best practices to evaluate AI, there is a gap between having such guidance and the application of such guidance to both existing and new AI models being developed. As of now, there is no publicly available, nationwide mechanism that enables objective evaluation and ongoing assessment of the consequences of using health AI models in clinical care settings.

Conclusion and Relevance

The need to create a public-private partnership to support a nationwide health AI assurance labs network is outlined here. In this network, community best practices could be applied for testing health AI models to produce reports on their performance that can be widely shared for managing the lifecycle of AI models over time and across populations and sites where these models are deployed.

Data-Driven and Technology-Enabled Trial Innovations Toward Decentralization of Clinical Trials: Opportunities and Considerations

Traditional trial designs have well-recognized inefficiencies and logistical barriers to participation. Decentralized trials and digital health solutions have been suggested as potential solutions and have certainly risen to the challenge during the pandemic. Clinical trial designs are now increasingly data driven. The use of distributed clinical data networks and digitization has helped to fundamentally upgrade existing research systems. A trial design may vary anywhere from fully decentralized to hybrid to traditional on-site. Various decentralization components are available for stakeholders to increase the reach and pace of their trials, such as electronic informed consent, remote interviews, administration, outcome assessment, monitoring, and laboratory and imaging modalities. Furthermore, digital health technologies can be included to enrich study conduct. However, careful consideration is warranted, including assessing verification and validity through usability studies and having various contingencies in place through dedicated risk assessment. Selecting the right combination depends not just on the ability to handle patient care and the medical know-how but also on the availability of appropriate technologic infrastructure, skills, and human resources. Throughout this process, quality of evidence generation and physician-patient relation must not be undermined. Here we also address some knowledge gaps, cost considerations, and potential impact of decentralization and digitization on inclusivity, recruitment, engagement, and retention. Last, we mention some future directions that may help drive the necessary change in the right direction.

Prospective evaluation of smartwatch-enabled detection of left ventricular dysfunction

Although artificial intelligence (AI) algorithms have been shown to be capable of identifying cardiac dysfunction, defined as ejection fraction (EF) ≤ 40%, from 12-lead electrocardiograms (ECGs), identification of cardiac dysfunction using the single-lead ECG of a smartwatch has yet to be tested. In the present study, a prospective study in which patients of Mayo Clinic were invited by email to download a Mayo Clinic iPhone application that sends watch ECGs to a secure data platform, we examined patient engagement with the study app and the diagnostic utility of the ECGs. We digitally enrolled 2,454 unique patients (mean age 53 ± 15 years, 56% female) from 46 US states and 11 countries, who sent 125,610 ECGs to the data platform between August 2021 and February 2022; 421 participants had at least one watch-classified sinus rhythm ECG within 30 d of an echocardiogram, of whom 16 (3.8%) had an EF ≤ 40%. The AI algorithm detected patients with low EF with an area under the curve of 0.885 (95% confidence interval 0.823-0.946) and 0.881 (0.815-0.947), using the mean prediction within a 30-d window or the closest ECG relative to the echocardiogram that determined the EF, respectively. These findings indicate that consumer watch ECGs, acquired in nonclinical environments, can be used to identify patients with cardiac dysfunction, a potentially life-threatening and often asymptomatic condition.

Assessing socioeconomic bias in machine learning algorithms in health care: a case study of the HOUSES index

OBJECTIVE

Artificial intelligence (AI) models may propagate harmful biases in performance and hence negatively affect the underserved. We aimed to assess the degree to which data quality of electronic health records (EHRs) affected by inequities related to low socioeconomic status (SES), results in differential performance of AI models across SES.

MATERIALS AND METHODS

This study utilized existing machine learning models for predicting asthma exacerbation in children with asthma. We compared balanced error rate (BER) against different SES levels measured by HOUsing-based SocioEconomic Status measure (HOUSES) index. As a possible mechanism for differential performance, we also compared incompleteness of EHR information relevant to asthma care by SES.

RESULTS

Asthmatic children with lower SES had larger BER than those with higher SES (eg, ratio = 1.35 for HOUSES Q1 vs Q2-Q4) and had a higher proportion of missing information relevant to asthma care (eg, 41% vs 24% for missing asthma severity and 12% vs 9.8% for undiagnosed asthma despite meeting asthma criteria).

DISCUSSION

Our study suggests that lower SES is associated with worse predictive model performance. It also highlights the potential role of incomplete EHR data in this differential performance and suggests a way to mitigate this bias.

CONCLUSION

The HOUSES index allows AI researchers to assess bias in predictive model performance by SES. Although our case study was based on a small sample size and a single-site study, the study results highlight a potential strategy for identifying bias by using an innovative SES measure.

Surveillance of Safety of 3 Doses of COVID-19 mRNA Vaccination Using Electronic Health Records

Importance

Recent reports on waning of COVID-19 vaccine-induced immunity have led to the approval and rollout of additional doses and booster vaccinations. Individuals at increased risk of SARS-CoV-2 infection are receiving additional vaccine doses in addition to the regimen that was tested in clinical trials. Risks and adverse event profiles associated with additional vaccine doses are currently not well understood.

Objective

To evaluate the safety of third-dose vaccination with US Food and Drug Administration (FDA)-approved COVID-19 mRNA vaccines.

Design, Setting, and Participants

This cohort study was conducted using electronic health record (EHR) data from December 2020 to October 2021 from the multistate Mayo Clinic Enterprise. Participants included all 47 999 individuals receiving 3-dose COVID-19 mRNA vaccines within the study setting who met study inclusion criteria. Participants were divided into 2 cohorts by vaccine brand administered and served as their own control groups, with no comparison made between cohorts. Data were analyzed from September through November 2021.

Exposures

Three doses of an FDA-authorized COVID-19 mRNA vaccine, BNT162b2 or mRNA-1273.

Main Outcomes and Measures

Vaccine-associated adverse events were assessed via EHR report. Adverse event risk was quantified using the percentage of study participants who reported the adverse event within 14 days after each vaccine dose and during a 14-day control period, immediately preceding the first vaccine dose.

Results

Among 47 999 individuals who received 3-dose COVID-19 mRNA vaccines, 38 094 individuals (21 835 [57.3%] women; median [IQR] age, 67.4 [52.5-76.5] years) received BNT162b2 (79.4%) and 9905 individuals (5099 [51.5%] women; median [IQR] age, 67.7 [59.5-73.9] years) received mRNA-1273 (20.6%). Reporting of severe adverse events remained low after the third vaccine dose, with rates of pericarditis (0.01%; 95% CI, 0%-0.02%), anaphylaxis (0%; 95% CI, 0%-0.01%), myocarditis (0%; 95% CI, 0%-0.01%), and cerebral venous sinus thrombosis (no individuals) consistent with results from earlier studies. Significantly more individuals reported low-severity adverse events after the third dose compared with after the second dose, including fatigue (2360 individuals [4.92%] vs 1665 individuals [3.47%]; P < .001), lymphadenopathy (1387 individuals [2.89%] vs 995 individuals [2.07%]; P < .001), nausea (1259 individuals [2.62%] vs 979 individuals [2.04%]; P < .001), headache (1185 individuals [2.47%] vs 992 individuals [2.07%]; P < .001), arthralgia (1019 individuals [2.12%] vs 816 individuals [1.70%]; P < .001), myalgia (956 individuals [1.99%] vs 784 individuals [1.63%]; P < .001), diarrhea (817 individuals [1.70%] vs 595 individuals [1.24%]; P < .001), fever (533 individuals [1.11%] vs 391 individuals [0.81%]; P < .001), vomiting (528 individuals [1.10%] vs 385 individuals [0.80%]; P < .001), and chills (224 individuals [0.47%] vs 175 individuals [0.36%]; P = .01).

Conclusions and Relevance

This study found that although third-dose vaccination against SARS-CoV-2 infection was associated with increased reporting of low-severity adverse events, risk of severe adverse events remained comparable with risk associated with the standard 2-dose regime. These findings suggest the safety of third vaccination doses in individuals who were eligible for booster vaccination at the time of this study.

The digital transformation of hepatology: The patient is logged in

The rise in innovative digital health technologies has led a paradigm shift in health care toward personalized, patient-centric medicine that is reaching beyond traditional brick-and-mortar facilities into patients' homes and everyday lives. Digital solutions can monitor and detect early changes in physiological data, predict disease progression and health-related outcomes based on individual risk factors, and manage disease intervention with a range of accessible telemedicine and mobile health options. In this review, we discuss the unique transformation underway in the care of patients with liver disease, specifically examining the digital transformation of diagnostics, prediction and clinical decision-making, and management. Additionally, we discuss the general considerations needed to confirm validity and oversight of new technologies, usability and acceptability of digital solutions, and equity and inclusivity of vulnerable populations.

Blockchain Integration With Digital Technology and the Future of Health Care Ecosystems: Systematic Review

BACKGROUND

In the era of big data, artificial intelligence (AI), and the Internet of Things (IoT), digital data have become essential for our everyday functioning and in health care services. The sensitive nature of health care data presents several crucial issues such as privacy, security, interoperability, and reliability that must be addressed in any health care data management system. However, most of the current health care systems are still facing major obstacles and are lacking in some of these areas. This is where decentralized, secure, and scalable databases, most notably blockchains, play critical roles in addressing these requirements without compromising security, thereby attracting considerable interest within the health care community. A blockchain can be maintained and widely distributed using a large network of nodes, mostly computers, each of which stores a full replica of the data. A blockchain protocol is a set of predefined rules or procedures that govern how the nodes interact with the network, view, verify, and add data to the ledger.

OBJECTIVE

In this article, we aim to explore blockchain technology, its framework, current applications, and integration with other innovations, as well as opportunities in diverse areas of health care and clinical research, in addition to clarifying its future impact on the health care ecosystem. We also elucidate 2 case studies to instantiate the potential role of blockchains in health care.

METHODS

To identify related existing work, terms based on Medical Subject Headings were used. We included studies focusing mainly on health care and clinical research and developed a functional framework for implementation and testing with data. The literature sources for this systematic review were PubMed, Medline, and the Cochrane library, in addition to a preliminary search of IEEE Xplore.

RESULTS

The included studies demonstrated multiple framework designs and various implementations in health care including chronic disease diagnosis, management, monitoring, and evaluation. We found that blockchains exhibit many promising applications in clinical trial management such as smart-contract application, participant-controlled data access, trustless protocols, and data validity. Electronic health records (EHRs), patient-centered interoperability, remote patient monitoring, and clinical trial data management were found to be major areas for blockchain usage, which can become a key catalyst for health care innovations.

CONCLUSIONS

The potential benefits of blockchains are limitless; however, concrete data on long-term clinical outcomes based on blockchains powered and supplemented by AI and IoT are yet to be obtained. Nonetheless, implementing blockchains as a novel way to integrate EHRs nationwide and manage common clinical problems in an algorithmic fashion has the potential for improving patient outcomes, health care experiences, as well as the overall health and well-being of individuals.

Building a best-in-class automated de-identification tool for electronic health records through ensemble learning

The presence of personally identifiable information (PII) in natural language portions of electronic health records (EHRs) constrains their broad reuse. Despite continuous improvements in automated detection of PII, residual identifiers require manual validation and correction. Here, we describe an automated de-identification system that employs an ensemble architecture, incorporating attention-based deep-learning models and rule-based methods, supported by heuristics for detecting PII in EHR data. Detected identifiers are then transformed into plausible, though fictional, surrogates to further obfuscate any leaked identifier. Our approach outperforms existing tools, with a recall of 0.992 and precision of 0.979 on the i2b2 2014 dataset and a recall of 0.994 and precision of 0.967 on a dataset of 10,000 notes from the Mayo Clinic. The de-identification system presented here enables the generation of de-identified patient data at the scale required for modern machine-learning applications to help accelerate medical discoveries.

Long-term SARS-CoV-2 RNA shedding and its temporal association to IgG seropositivity

Longitudinal characterization of SARS-CoV-2 PCR testing from COVID-19 patient's nasopharynx and its juxtaposition with blood-based IgG-seroconversion diagnostic assays is critical to understanding SARS-CoV-2 infection durations. Here, we retrospectively analyze 851 SARS-CoV-2-positive patients with at least two positive PCR tests and find that 99 of these patients remain SARS-CoV-2-positive after 4 weeks from their initial diagnosis date. For the 851-patient cohort, the mean lower bound of viral RNA shedding was 17.3 days (SD: 7.8), and the mean upper bound of viral RNA shedding from 668 patients transitioning to confirmed PCR-negative status was 22.7 days (SD: 11.8). Among 104 patients with an IgG test result, 90 patients were seropositive to date, with mean upper bound of time to seropositivity from initial diagnosis being 37.8 days (95% CI: 34.3-41.3). Our findings from juxtaposing IgG and PCR tests thus reveal that some SARS-CoV-2-positive patients are non-hospitalized and seropositive, yet actively shed viral RNA (14 of 90 patients). This study emphasizes the need for monitoring viral loads and neutralizing antibody titers in long-term non-hospitalized shedders as a means of characterizing the SARS-CoV-2 infection lifecycle.

Long-term SARS-CoV-2 RNA Shedding and its Temporal Association to IgG Seropositivity

Analysis of 851 COVID-19 patients with a SARS-CoV-2-positive PCR at follow-up shows 99 patients remained SARS-CoV-2-positive after four weeks from initial diagnosis. Surprisingly, a majority of these long-term viral RNA shedders were not hospitalized (61 of 99), with variable PCR Crossing point values over the month post diagnosis. For the 851-patient cohort, the mean lower bound of viral RNA shedding was 17.3 days (SD: 7.8), and the mean upper bound of viral RNA shedding from 668 patients transitioning to confirmed PCR-negative status was 22.7 days (SD: 11.8). Among 104 patients with an IgG test result, 90 patients were seropositive to date, with mean upper bound of time to seropositivity from initial diagnosis being 37.8 days (95%CI: 34.3-41.3). Juxtaposing IgG/PCR tests revealed that 14 of 90 patients are non-hospitalized and seropositive yet shed viral RNA. This study emphasizes the need for monitoring viral loads and neutralizing antibody titers in long-term shedders.

Artificial Intelligence and the Future of Primary Care: Exploratory Qualitative Study of UK General Practitioners' Views

Background

The potential for machine learning to disrupt the medical profession is the subject of ongoing debate within biomedical informatics and related fields.

Objective

This study aimed to explore general practitioners’ (GPs’) opinions about the potential impact of future technology on key tasks in primary care.

Methods

In June 2018, we conducted a Web-based survey of 720 UK GPs’ opinions about the likelihood of future technology to fully replace GPs in performing 6 key primary care tasks, and, if respondents considered replacement for a particular task likely, to estimate how soon the technological capacity might emerge. This study involved qualitative descriptive analysis of written responses (“comments”) to an open-ended question in the survey.

Results

Comments were classified into 3 major categories in relation to primary care: (1) limitations of future technology, (2) potential benefits of future technology, and (3) social and ethical concerns. Perceived limitations included the beliefs that communication and empathy are exclusively human competencies; many GPs also considered clinical reasoning and the ability to provide value-based care as necessitating physicians’ judgments. Perceived benefits of technology included expectations about improved efficiencies, in particular with respect to the reduction of administrative burdens on physicians. Social and ethical concerns encompassed multiple, divergent themes including the need to train more doctors to overcome workforce shortfalls and misgivings about the acceptability of future technology to patients. However, some GPs believed that the failure to adopt technological innovations could incur harms to both patients and physicians.

Conclusions

This study presents timely information on physicians’ views about the scope of artificial intelligence (AI) in primary care. Overwhelmingly, GPs considered the potential of AI to be limited. These views differ from the predictions of biomedical informaticians. More extensive, stand-alone qualitative work would provide a more in-depth understanding of GPs’ views.

Clinical 3D printing: A protected health information (PHI) and compliance perspective

Advanced manufacturing techniques such as 3-dimensional (3D) printing, while mature in other industries, are starting to become more commonplace in clinical care. Clinicians are producing physical objects based on patient clinical data for use in planning care and educating patients, all of which should be managed like any other healthcare system data, except it exists in the "real" world. There are currently no provisions in the Health Insurance Portability and Accountability Act (HIPAA) either in its original 1996 form or in more recent updates that address the nature of physical representations of clinical data. We submit that if we define the source data as protected health information (PHI), then the objects 3D printed from that data need to be treated as both (PHI), and if used clinically, part of the clinical record, and propose some basic guidelines for quality and privacy like all documentation until regulatory frameworks can catch up to this technology. Many of the mechanisms designed in the paper and film chart era will work well with 3D printed patient data.

Best Practices in Clinical Decision Support: the Case of Preventive Care Reminders

BACKGROUND: Evidence demonstrates that clinical decision support (CDS) is a powerful tool for improving healthcare quality and ensuring patient safety. However, implementing and maintaining effective decision support interventions presents multiple technical and organizational challenges. PURPOSE: To identify best practices for CDS, using the domain of preventive care reminders as an example. METHODS: We assembled a panel of experts in CDS and held a series of facilitated online and in-person discussions. We analyzed the results of these discussions using a grounded theory method to elicit themes and best practices. RESULTS: Eight best practice themes were identified as important: deliver CDS in the most appropriate ways, develop effective governance structures, consider use of incentives, be aware of workflow, keep content current, monitor and evaluate impact, maintain high quality data, and consider sharing content. Keys themes within each of these areas were also described. CONCLUSION: Successful implementation of CDS requires consideration of both technical and socio-technical factors. The themes identified in this study provide guidance on crucial factors that need consideration when CDS is implemented across healthcare settings. These best practice themes may be useful for developers, implementers, and users of decision support.

Early experiences with big data at an academic medical center

Beth Israel Deaconess Medical Center (BIDMC), an academic health care institution affiliated with Harvard University, has been an early adopter of electronic applications since the 1970s. Various departments of the medical center and the physician practice groups affiliated with it have implemented electronic health records, filmless imaging, and networked medical devices to such an extent that data storage at BIDMC now amounts to three petabytes and continues to grow at a rate of 25 percent a year. Initially, the greatest technical challenge was the cost and complexity of data storage. However, today the major focus is on transforming raw data into information, knowledge, and wisdom. This article discusses the data growth, increasing importance of analytics, and changing user requirements that have shaped the management of big data at BIDMC.

An EHR with Teeth

Citation: Kalenderian E, Halamka JD, Spallek H. An EHR with teeth.

Making the most of federal health information technology regulations

"Meaningful use" of interoperable electronic health records throughout the U.S. health care delivery system--the goal set forth in the American Reinvestment and Recovery Act (ARRA) of 2009--is a critical national goal. Proposed federal regulations on data exchange standards and the definition of meaningful use are well conceived and provide a foundation for the nation to begin the journey.

There are important reasons for delaying implementation of the new ICD-10 coding system

Federal authorities have recently signaled that they would consider delaying some aspects of implementation of the newest version of the International Classification of Diseases, known as ICD-10-CM, a coding system used to define health care charges and diagnoses. Some industry groups have reacted with dismay, and many providers with relief. We are concerned that adopting this new classification system for reimbursement will be disruptive and costly and will offer no material improvement over the current system. Because the health care community is also working to integrate health information technology and federal meaningful-use specifications that require the adoption of other complex coding standardization systems (such as the system called SNOMED CT), we recommend that the Centers for Medicare and Medicaid Services consider delaying the adoption of ICD-10-CM. Policy makers should also begin planning now for ways to make the coming transition to ICD-11 as tolerable as possible for the health care and payment community.

Internet portal use in an academic multiple sclerosis center

OBJECTIVE

To evaluate the use of a secure internet portal in an academic Multiple Sclerosis (MS) Center.

MATERIALS AND METHODS

Retrospective case-control chart review of 240 patients during the years 2008 and 2009. Patient demographic and clinical information was extracted from our online medical records, and portal use metrics were provided by Information Systems. Descriptive statistics were utilized to explore characteristics of portal users, how the portal is used, and what associations exist between medical resource utilization and active portal use. Logistic regression identified independent patient predictors and barriers to portal use.

RESULTS

Portal users tended to be young professionals with minimal physical disability. The most frequently used portal feature was secure patient-physician messaging. Message content largely consisted of requests for medications or refills in addition to self-reported side effects. Independent predictors and barriers of portal use include the number of medications prescribed by our staff (OR 1.69, p<0.0001), Caucasian ethnicity (OR 5.04, p=0.007), arm and hand disability (OR 0.23, p=0.01), and impaired vision (OR 0.31, p=0.01). Discussion MS patients use the internet in a greater proportion than the general US population, yet physical disability limits their access. Technological adaptations such as voice-activated commands and easy font-size adjustment may help patients overcome these barriers.

CONCLUSION

Future research should explore the influence of portal technology on healthcare resource utilization and cost. Additional emedicine applications could be linked to the patient portal for disease monitoring and prospective investigation.

Delphi consensus on the feasibility of translating the ACEP clinical policies into computerized clinical decision support

Clinical practice guidelines are developed to reduce variations in clinical practice, with the goal of improving health care quality and cost. However, evidence-based practice guidelines face barriers to dissemination, implementation, usability, integration into practice, and use. The American College of Emergency Physicians (ACEP) clinical policies have been shown to be safe and effective and are even cited by other specialties. In spite of the benefits of the ACEP clinical policies, implementation of these clinical practice guidelines into physician practice continues to be a challenge. Translation of the ACEP clinical policies into real-time computerized clinical decision support systems could help address these barriers and improve clinician decision making at the point of care. The investigators convened an emergency medicine informatics expert panel and used a Delphi consensus process to assess the feasibility of translating the current ACEP clinical policies into clinical decision support content. This resulting consensus document will serve to identify limitations to implementation of the existing ACEP Clinical Policies so that future clinical practice guideline development will consider implementation into clinical decision support at all stages of guideline development.

Early experiences with personal health records

Over the past year, several payers, employers, and commercial vendors have announced personal health record projects. Few of these are widely deployed and few are fully integrated into ambulatory or hospital-based electronic record systems. The earliest adopters of personal health records have many lessons learned that can inform these new initiatives. We present three case studies--MyChart at Palo Alto Medical Foundation, PatientSite at Beth Israel Deaconess Medical Center, and Indivo at Children's Hospital Boston. We describe our implementation challenges from 1999 to 2007 and postulate the evolving challenges we will face over the next five years.

Integration of the personally controlled electronic medical record into regional inter-regional data exchanges: a national demonstration

We present the approach taken in a Massachusetts-based national demonstration project to integrate the PING personally controlled health record (PCHR) with the MA-SHARE network, the state-wide inter-organizational data exchange. We describe how we have created a patient-controlled gateway to the network, and how PCHRs have become a first class data source in the network.

Emergency department triage of patients infected with HIV

OBJECTIVE

The emergency department (ED) and HIV specialty clinics are primary sources of care for persons infected with HIV. HIV disease may be complicated by vague and complex symptomatology, and determining the degree of illness at triage is often difficult. The goals of this project were to characterize the ED presentation of HIV-related conditions, to develop a clinical decision rule to triage HIV-infected patients, and to validate the rule in clinical practice.

METHODS

The study population consisted of ambulatory patients with self-reported HIV infection who presented for care to the ED of a 553-bed public hospital that serves a medically indigent, minority population. An Illness Severity Instrument was developed by an expert panel to serve as the criterion standard for defining medical urgency for HIV-infected patients presenting to the ED for care. Two phases of the study were conducted. Data from the first phase, a noninterventional cohort study, were used to develop a clinical decision rule for the ED triage of HIV-infected patients. The second phase was a prospective validation of the clinical decision rule.

RESULTS

During phase I, data from 542 patient visits were collected. Data from 441 (81%) patient visits were used in a classification and regression tree (CART) analysis to produce a decision rule, the Clinical Triage Instrument. During phase II, the prospective validation of the Clinical Triage Instrument, 156 patient visits occurred. Of these, 88 (56%) patient visits were triaged using the Clinical Triage Instrument and could be scored using the Illness Severity Instrument. The Clinical Triage Instrument accurately triaged 45 [51%; 95% confidence interval (95% CI) = 40% to 62%] patient visits, undertriaged 11 (13%; 95% CI = 6% to 21%) patient visits, and overtriaged 32 (36%; 95% CI = 26% to 47%) patient visits. Sensitivities and specificities for determining emergent, urgent, and nonurgent medical conditions by the Clinical Triage Instrument were 56% (95% CI = 31% to 75%) and 84% (95% CI = 74% to 92%), 71% (95% CI = 55% to 84%) and 39% (95% CI = 25% to 55%), and 18% (95% CI = 6% to 37%) and 93% (95% CI = 84% to 98%), respectively. The positive and negative predictive values for determining an emergent medical condition using the Clinical Triage Instrument were 48% (95% CI = 26% to 70%) and 88% (95% CI = 78% to 95%), respectively. The positive and negative predictive values for determining a nonurgent medical condition using the Clinical Triage Instrument were 56% (95% CI = 21% to 86%) and 71% (95% CI = 60% to 81%), respectively.

CONCLUSIONS

The Clinical Triage Instrument was not sufficiently accurate for clinical use. Until accurate and reliable triage methods are developed, all patients infected with HIV who present to the ED for care should receive timely evaluation and care.

CareWeb, a web-based medical record for an integrated healthcare delivery system

With the advent of Integrated Healthcare Delivery Systems, medical records are increasingly distributed across multiple institutions. Timely access to these medical records is a critical need for healthcare providers. The CareWeb project provides an architecture for World Wide Web-based retrieval of electronic medical records from heterogeneous data sources. Using Health Level 7 (HL7), web technologies and readily available software components, we consolidated the electronic records of Boston's Beth Israel and Deaconess Hospitals. We report on the creation of CareWeb (freya.bidmc.harvard.edu/careweb.htm) and propose it as a means to electronically link Integrated Health Care Delivery Systems and geographically distant information resources.

CareWeb, a web-based medical record for an integrated health care delivery system

With the advent of Integrated Health care Delivery Systems, medical records are increasingly distributed across multiple institutions. Timely access to these medical records is a critical need for health care providers. The CareWeb project provides an architecture for World Wide Web-based retrieval of electronic medical records from heterogeneous data sources. Using Health Level 7 (HL7), web technologies and readily available software components, we consolidated the electronic records of Boston's Beth Israel and Deaconess Hospitals. We report on the creation of CareWeb (freya.bidmc.harvard.edu/careweb.htm) and propose it as a means to electronically link Integrated Health care Delivery Systems and geographically distant information resources.

Web technology for emergency medicine and secure transmission of electronic patient records

The American Heritage dictionary defines the word "web" as "something intricately contrived, especially something that ensnares or entangles." The wealth of medical resources on the World Wide Web is now so extensive, yet disorganized and unmonitored, that such a definition seems fitting. In emergency medicine, for example, a field in which accurate and complete information, including patients' records, is urgently needed, more than 5000 Web pages are available today, whereas fewer than 50 were available in December 1994. Most sites are static Web pages using the Internet to publish textbook material, but new technology is extending the scope of the Internet to include online medical education and secure exchange of clinical information. This article lists some of the best Web sites for use in emergency medicine and then describes a project in which the Web is used for transmission and protection of electronic medical records.

Implementation of an on-line Emergency Unit nursing system

We designed and implemented an Emergency Unit nursing system which has been in practice since March of 1995. The system has been very successful and is used on 97% of all Emergency Unit visits. The system is designed for rapid entry and retrieval of information. We have found that the time spent by nurses documenting patient care in this system is minimal. On average, over a six day study period only five minutes per patient visit, and about 30 minutes per nurse per shift was spent actually entering information into the system. We plan to continue to evaluate the use of the system and expand it to include use by other disciplines in the near future.

Managing care in an integrated delivery system via an Intranet

The CareGroup Provider Service Network is a managed care contracting organization which provides central administrative services for over 1800 physicians and 200,000 managed care lives. Services include utilization management, disease management and credentialing for the entire network. The management model of the Provider Service Network empowers local physician groups with information and education. To meet the managed care information needs of the network, we implemented an intranet-based executive information system, PSNWeb, which retrieves data from a managed care data warehouse. The project required the integration of diverse technologies and development of a complex security/confidentiality infrastructure to deliver information to 8 major clinician groups, each with different information needs.

A WWW implementation of national recommendations for protecting electronic health information

In March of 1997, the National Research Council (NRC) of the National Academy of Sciences issued the report, "For the Record: Protecting Electronic Health Information." Concluding that the current practices at the majority of health care facilities in the United States are insufficient, the Council delineated both technical and organizational approaches to protecting electronic health information. The Beth Israel Deaconess Medical Center recently implemented a proof-of-concept, Web-based, cross-institutional medical record, CareWeb, which incorporates the NRC security and confidentiality recommendations. We report on our WWW implementation of the NRC recommendations and an initial evaluation of the balance between ease of use and confidentiality.

Virtual consolidation of Boston's Beth Israel and New England Deaconess Hospitals via the World Wide Web

With the advent of Integrated Healthcare Delivery Systems, medical records are increasingly distributed across multiple institutions. Timely access to these medical records is a critical need for healthcare providers. The CareWeb project provides an architecture for World Wide Web-based retrieval of electronic medical records from heterogeneous data sources. Using Health Level 7 (HL7), web technologies and readily available software components, we consolidated the electronic records of Boston's Beth Israel and Deaconess Hospitals. We report on the creation of CareWeb (freya.bidmc.harvard.edu/careweb.htm) and propose it as a means to electronically link Integrated Health Care Delivery Systems and geographically distant information resources.