UpToDate versus DynaMed: a cross-sectional study comparing the speed and accuracy of two point-of-care information tools

Smartphone apps for point-of-care information summaries: systematic assessment of the quality and content

BACKGROUND

Clinicians need easy access to evidence-based information to inform their clinical practice. Point-of-care information summaries are increasingly available in the form of smartphone apps. However, the quality of information from the apps is questionable as there is currently no regulation on the content of the medical apps.

OBJECTIVES

This study aimed to systematically assess the quality and content of the medical apps providing point-of-care information summaries that were available in two major app stores. We evaluated apps designed specifically for healthcare professionals and assessed their content development, editorial policy, coverage of medical conditions and trustworthiness.

METHODS

We conducted a systematic assessment of medical apps providing point-of-care information summaries available in Google Play and Apple app stores. Apps launched or updated since January 2020 were identified through a systematic search using 42matters. Apps meeting the inclusion criteria were downloaded and assessed. The data extraction and app assessment were done in parallel and independently by at least two reviewers. Apps were evaluated against the adapted criteria: (1) general characteristics, (2) content presentation of the summaries, (3) editorial quality, (4) evidence-based methodology, (5) coverage (volume) of the medical conditions, (6) usability of apps and (7) trustworthiness of the app based on HONcode principles. HONcode principles are guidelines used to inform users about the credibility and reliability of health information online. The results were reported as a narrative review.

RESULTS

Eight medical apps met the inclusion criteria and were systematically appraised. Based on our evaluation criteria, UpToDate supported 16 languages, and all other apps were English. Bullet points and brief paragraphs were used in all apps, and only DynaMed and Micromedex and Pathway-medical knowledge provided a formal grading system for the strength of recommendations for all the medical conditions in their apps. All the other apps either lacked a formal grading system altogether or offered one for some of the medical conditions. About 30% of the editorial quality assessment and 47.5% of the evidence-based methodology assessment were unclear or missing. UpToDate contained the most point-of-care evidence-based documents with >10 500 documents. All apps except 5-Minute Clinical Consult and DynaMed and Micromedex were available for offline access. Only Medscape complied with the HONcode principles.

CONCLUSIONS

Future apps should report a more detailed evidence-based methodology, be accessible for offline use and support search in more than one language. There should be clearer information provided in future apps regarding the declaration of authorship and conflict of interest.

Performance of a Web-Based Reference Database With Natural Language Searching Capabilities: Usability Evaluation of DynaMed and Micromedex With Watson

BACKGROUND

Evidence-based point-of-care information (POCI) tools can facilitate patient safety and care by helping clinicians to answer disease state and drug information questions in less time and with less effort. However, these tools may also be visually challenging to navigate or lack the comprehensiveness needed to sufficiently address a medical issue.

OBJECTIVE

This study aimed to collect clinicians' feedback and directly observe their use of the combined POCI tool DynaMed and Micromedex with Watson, now known as DynaMedex. EBSCO partnered with IBM Watson Health, now known as Merative, to develop the combined tool as a resource for clinicians. We aimed to identify areas for refinement based on participant feedback and examine participant perceptions to inform further development.

METHODS

Participants (N=43) within varying clinical roles and specialties were recruited from Brigham and Women's Hospital and Massachusetts General Hospital in Boston, Massachusetts, United States, between August 10, 2021, and December 16, 2021, to take part in usability sessions aimed at evaluating the efficiency and effectiveness of, as well as satisfaction with, the DynaMed and Micromedex with Watson tool. Usability testing methods, including think aloud and observations of user behavior, were used to identify challenges regarding the combined tool. Data collection included measurements of time on task; task ease; satisfaction with the answer; posttest feedback on likes, dislikes, and perceived reliability of the tool; and interest in recommending the tool to a colleague.

RESULTS

On a 7-point Likert scale, pharmacists rated ease (mean 5.98, SD 1.38) and satisfaction (mean 6.31, SD 1.34) with the combined POCI tool higher than the physicians, nurse practitioner, and physician's assistants (ease: mean 5.57, SD 1.64, and satisfaction: mean 5.82, SD 1.60). Pharmacists spent longer (mean 2 minutes, 26 seconds, SD 1 minute, 41 seconds) on average finding an answer to their question than the physicians, nurse practitioner, and physician's assistants (mean 1 minute, 40 seconds, SD 1 minute, 23 seconds).

CONCLUSIONS

Overall, the tool performed well, but this usability evaluation identified multiple opportunities for improvement that would help inexperienced users.

Race-based medicine in the point-of-care clinical resource UpToDate: A systematic content analysis

BACKGROUND

Race-based practices in medical education and clinical care may exacerbate health inequities. Misguided use of race in popular point-of-care clinical decision-making tools like UpToDate® may promote harmful practices of race-based medicine. This article investigates the nature of mentions of Black/African American race in UpToDate®.

METHODS

We conducted a systematic content analysis of UpToDate® articles mentioning Black or African American race to assess for biological interpretations of racial categories. Following a simple text search for the terms "Black" and "African American" in UpToDate® on January 24 and March 19, 2020, respectively, removal of duplicates yielded an analytical sample of 208 documents. We adopted a deductive coding approach and systematically applied 16 a priori codes to all documents, refining the codebook to achieve a final inter-rater reliability of 0.91. We then developed these codes into two themes: (1) biologization of race and (2) racialized research and practice.

FINDINGS

Biologization of race occurred nearly universally across all documents (93.3%), with discussions of inherent physiological differences between racial groups and presentation of epidemiologic disparities without context emerging most frequently. Sixty-eight documents (32.7%) included codes related to racialized biomedical research and clinical practice, including references to racialized patterns of behavior and cultural practices, insufficient data on Black populations, research limiting study to a specific racial group, and race-based clinical practices guidelines.

INTERPRETATION

Our findings suggest that UpToDate® articles often inappropriately link Black race to genetics or clinical phenotype-without considering socio-structural variables or the health effects of structural racism-thus perpetuating a false narrative that race is inherently biological. UpToDate® articles may also promote unequal treatment by recommending race-based clinical practices. Such racial essentialism risks exacerbating racialized health inequities.

FUNDING

The study is supported by the Health Policy Research Scholars Program, Robert Wood Johnson Foundation, Medical Scientist Training Program, National Institutes of Health, the National Science Foundation, the JPB Foundation, the Minnesota Population, the Eunice Kennedy Shriver National Institute for Child Health and Human Development (NICHD), and the Center for Antiracism Research for Health Equity at the University of Minnesota.

Decision-Making in the Pediatric Emergency Department-A Survey of Guidance Strategies among Residents

(1) Introduction: Working in an emergency department requires fast and straightforward decisions. Therefore, decision guidance represents an essential tool for successful patient-centered care. Beyond the residents' own knowledge and experience, printed books have been the primary source of information in the past. The aim of this study was to discover which strategies current residents use the most and to identify alternative quick reference strategies in the digital age. (2) Materials and Methods: This study analyzed the responses of a short questionnaire directed at 41 residents in a single pediatric emergency department (32 pediatric and 9 pediatric surgery residents) over a period of one month. (3) Results: Thirty-three (80.5%) residents answered the entire questionnaire. Strikingly, responses indicated that printed books are still pivotal in guiding decision-making. In addition, the acquisition of information via computers or smartphones plays an increasing role. However, the opinion and council of the attending physician is still of great value to the residents and is not to be underestimated. Overall, most of the residents would prefer to have access to a specially designed smartphone application. (4) Conclusions: Certainty and validity are essential in decision-making in a pediatric emergency department. Although printed books and attending physicians are still considered as reliable sources of information, internet-based information plays an increasing role. In order to provide the best up-to-date and most recent information, a validated and consistently updated smartphone application could be a useful option.

Examination of Potential Industry Conflicts of Interest and Disclosures by Contributors to Online Medical Resource Databases

IMPORTANCE

Physicians contribute content to online databases, and other health care professionals use these websites to support their decision-making. Financial conflicts of interest (COI) have the potential to adversely impact evidence-based patient care.

OBJECTIVES

To quantify the potential COI among content contributors to 2 popular point-of-care medical resources, UpToDate and DynaMed, overall and by gender, and to compare self-reported and industry-mandated disclosures.

DESIGN, SETTING, AND PARTICIPANTS

This cross-sectional study compiled an initial list of contributors for each website using the Centers for Disease Control and Prevention's Leading Causes of Death. The top 50 causes were used to identify relevant articles from each database from November 30 to December 7, 2020. The authors and editors of those articles were investigated. Data were analyzed from January 2021 to March 2022.

MAIN OUTCOMES AND MEASURES

Self-reported contributor disclosure status was compared with financial remuneration as reported in the Centers for Medicare & Medicaid Services Open Payments (OP) database from 2013 to 2018.

RESULTS

A total of 179 point-of-care database authors and editors were assessed. Combined, they received $77.7 million, with a mean of $583 218 (95% CI, $0-$4 679 651) and median (range) of $29 073 ($10-$17 517 315) each. Most of the compensation ($68.1 million [87.6%]) went to UpToDate contributors. Of 128 UpToDate contributors, 76 (59.4%) reported nothing to disclose, and among these, 44 contributors (57.9%) had a record of receiving a financial payment on OP. Women UpToDate contributors received 2.5% of the total compensation paid from industry. The top 10 UpToDate contributors who received the most financial remuneration earned $56.1 million combined, were all men, and only 1 had a nothing-to-disclose status. Of 51 DynaMed contributors, 42 (82.4%) reported nothing to disclose, and among these, 35 contributors (83.3%) had an OP entry (mean, $79 820; 95% CI, $0-$400 774; median [range], $1403 [$26-$630 424]). Among the top 10 DynaMed contributors, 8 (80.0%) were men. Six of the top 10 DynaMed contributors reported nothing to disclose yet had an OP entry.

CONCLUSIONS AND RELEVANCE

This cross-sectional study found that contributors to point-of-care databases were the recipients of nearly $78 million from pharmaceutical companies and medical device manufacturers, and these payments were often not disclosed in association with contributed content. Although these findings do not necessarily suggest ethical lapses among the physicians studied, point-of-care resource websites, like UpToDate and DynaMed, should consider implementing more stringent COI policies and employ an unbiased team to verify self-reported disclosure statuses among content contributors against OP reports.

Looking at Library Resource Usage Trends in Medical Residents & Fellows: Survey Results from 2006-2021

Since 2006, the University of Tennessee's Preston Medical Library has collected survey feedback from exiting residents through targeted emails or at the required exit process in the library. Questions ask residents which types of articles or sources they use to find information and address the resident's use of the library's physical space. Survey results from 2006 to 2021 were examined for trends and changes in resident utilization of resources to better inform future library decisions on instruction and marketing. Resident resource usage varied over time and demonstrated an encouraging increase in attention to evidence-based tools. UpToDate's consistent popularity shows the durability of the product. A reported increase in use of reviews, randomized controlled trials, case reports, and practice guidelines reflects greater employment of more in-depth resources than merely expert opinion. At the same time, residents clearly valued the library's physical space. Survey results will inform future outreach focus.

Comparing the Use of DynaMed and UpToDate by Physician Trainees in Clinical Decision-Making: A Randomized Crossover Trial

BACKGROUND

Costs vary substantially among electronic medical knowledge resources used for clinical decision support, warranting periodic assessment of institution-wide adoption.

OBJECTIVES

To compare two medical knowledge resources, UpToDate and DynaMed Plus, regarding accuracy and time required to answer standardized clinical questions and user experience.

METHODS

A crossover trial design was used, wherein physicians were randomized to first use one of the two medical knowledge resources to answer six standardized questions. Following use of each resource, they were surveyed regarding their user experience. The percentage of accurate answers and time required to answer each question were recorded. The surveys assessed ease of use, enjoyment using the resource, quality of information, and ability to assess level of evidence. Tests of carry-over effects were performed. Themes were identified within open-ended survey comments regarding overall user experience.

RESULTS

Among 26 participating physicians, accuracy of answers differed by 4 percentage points or less. For all but one question, there were no significant differences in the time required for completion. Most participants felt both resources were easy to use, contained high quality of information, and enabled assessment of the level of evidence. A greater proportion of participants endorsed enjoyment of use with UpToDate (23/26, 88%) compared with DynaMed Plus (16/26, 62%). Themes from open-ended comments included interface/information presentation, coverage of clinical topics, search functions, and utility for clinical decision-making. The majority (59%) of open-ended comments expressed an overall preference for UpToDate, compared with 19% preferring DynaMed Plus.

CONCLUSION

DynaMed Plus is noninferior to UpToDate with respect to ability to achieve accurate answers, time required for answering clinical questions, ease of use, quality of information, and ability to assess level of evidence. However, user experience was more positive with UpToDate. Future studies of electronic medical knowledge resources should continue to emphasize evaluation of usability and user experience.