Evaluating Internet end-to-end performance: overview of test methodology and results

Are we making a difference? Outreach evaluation in practice

Donald A.B. Lindberg, M.D., brought with him when he joined NLM an inquisitive mind, tech savvy, and new ideas. He was an early advocate of both outreach and evaluation innovation at NLM. Dr. Lindberg initiated and supported multiple pilot test and implementation projects to strengthen NLM's health information outreach to healthcare providers, research scientists, health science and hospital librarians, and the general public, including minority and underserved populations. He helped steer NLM's transition to the Internet, and NLM's development of a robust framework for evaluating Internet and Web-based health information dissemination and outreach to its many audiences. Dr. Lindberg's leadership led to numerous landmark accomplishments, including the capacity-building "Measuring the Difference" outreach evaluation Guide, and a multi-dimensional approach to Internet and website evaluation that placed NLM at the forefront of federal agencies using these new and emerging technologies to support their missions.

The internet

The growing use of email and the world wide web (WWW), by the public, academics, and clinicians-as well as the increasing availability of high quality information on the WWW-make a working knowledge of the internet important. Although this article aims to enhance readers' existing use of the internet and medical resources on the WWW, it is also intelligible to someone unfamiliar with the internet. A web browser is one of the central pieces of software in modern computing: it is a window on the WWW, file transfer protocol sites, networked newsgroups, and your own computer's files. Effective use of the internet for professional purposes requires an understanding of the best strategies to search the WWW and the mechanisms for ensuring secure data transfer, as well as a compendium of online resources including journals, textbooks, medical portals, and sites providing high quality patient information. This article summarises these resources, available to incorporate into your web browser as downloadable "Favorites" or "Bookmarks" from www.jnnp.com, where there are also freely accessible hypertext links to the recommended sites.

Public library consumer health information pilot project: results of a National Library of Medicine evaluation

In October 1998, the National Library of Medicine (NLM) launched a pilot project to learn about the role of public libraries in providing health information to the public and to generate information that would assist NLM and the National Network of Libraries of Medicine (NN/LM) in learning how best to work with public libraries in the future. Three regional medical libraries (RMLs), eight resource libraries, and forty-one public libraries or library systems from nine states and the District of Columbia were selected for participation. The pilot project included an evaluation component that was carried out in parallel with project implementation. The evaluation ran through September 1999. The results of the evaluation indicated that participating public librarians were enthusiastic about the training and information materials provided as part of the project and that many public libraries used the materials and conducted their own outreach to local communities and groups. Most libraries applied the modest funds to purchase additional Internet-accessible computers and/or upgrade their health-reference materials. However, few of the participating public libraries had health information centers (although health information was perceived as a top-ten or top-five topic of interest to patrons). Also, the project generated only minimal usage of NLM's consumer health database, known as MEDLINEplus, from the premises of the monitored libraries (patron usage from home or office locations was not tracked). The evaluation results suggested a balanced follow-up by NLM and the NN/LM, with a few carefully selected national activities, complemented by a package of targeted activities that, as of January 2000, are being planned, developed, or implemented. The results also highlighted the importance of building an evaluation component into projects like this one from the outset, to assure that objectives were met and that evaluative information was available on a timely basis, as was the case here.

Electronic health record meets digital library: a new environment for achieving an old goal

Linking the electronic health record to the digital library is a Web-era reformulation of the long-standing informatics goal of seamless integration of automated clinical data and relevant knowledge-based information to support informed decisions. The spread of the Internet, the development of the World Wide Web, and converging format standards for electronic health data and digital publications make effective linking increasingly feasible. Some existing systems link electronic health data and knowledge-based information in limited settings or limited ways. Yet many challenging informatics research problems remain to be solved before flexible and seamless linking becomes a reality and before systems become capable of delivering the specific piece of information needed at the time and place a decision must be made. Connecting the electronic health record to the digital library also requires positive resolution of important policy issues, including health data privacy, government encouragement of high-speed communications, electronic intellectual property rights, and standards for health data and for digital libraries. Both the research problems and the policy issues should be important priorities for the field of medical informatics.

Network latency and operator performance in teleradiology applications

Teleradiology applications often use an interactive conferencing mode with remote control mouse pointers. When a telephone is used for voice communication, latencies of the data network can create a temporal discrepancy between the position of the mouse pointer and the verbal communication. To assess the effects of this dissociation, we examined the performance of 5 test persons carrying out simple teleradiology tasks under varying simulated network conditions. When the network latency exceeded 400 milliseconds, the performance of the test persons dropped, and an increasing number of errors were made. This effect was the same for constant latencies, which can occur on the network path, and for variable delays caused by the Nagle algorithm, an internal buffering scheme used by the TCP/IP protocol. Because the Nagle algorithm used in typical TCP/IP implementations causes a latency of about 300 milliseconds even before a data packet is sent, any additional latency in the network of 100 milliseconds or more will result in a decreased operator performance in teleradiology applications. These conditions frequently occur on the public Internet or on overseas connections. For optimal performance, the authors recommend bypassing the Nagle algorithm in teleradiology applications.