A MeSH based intelligent search intermediary for Consumer Health Information Systems

Interventions to support consumer evaluation of online health information credibility: A scoping review

PURPOSE

Various interventions have been designed to help consumers better evaluate the credibility of online health information (OHI). However, assessing information credibility remained the most widely reported challenge by online health consumers. This review aims to provide an overview of major intervention approaches for improving consumer ability to evaluate OHI credibility in order to identify opportunities for future interventions.

METHODS

A scoping review was performed. Seven relevant scientific databases were searched to identify articles that report the design and/or evaluation of interventions to support, facilitate, or assist consumers in assessing the credibility of OHI. Thirty-one articles met the inclusion criteria. Relevant content was extracted from the articles and all codes were validated by second coders.

RESULTS

Three major intervention approaches for enhancing consumers' ability to evaluate OHI credibility were identified: educational program, algorithm, and interactive interface. The design of most interventions (particularly the credibility evaluation component) lacked the guidance of theories, and very few studies systematically evaluated their effectiveness in real online search contexts. Few interventions can provide spontaneous support to consumers while they search online.

CONCLUSION

Our understanding of what theoretical constructs contribute to effective OHI credibility evaluation interventions and how intervention outcomes should be measured remained limited. Future efforts need to focus on the design, development, test, and evaluation of theory-guided OHI credibility evaluation interventions that are scalable, sustainable, and can provide real-time support to consumers.

A study of the influence of task familiarity on user behaviors and performance with a MeSH term suggestion interface for PubMed bibliographic search

PURPOSE

Previous research has shown that information seekers in biomedical domain need more support in formulating their queries. A user study was conducted to evaluate the effectiveness of a metadata based query suggestion interface for PubMed bibliographic search. The study also investigated the impact of search task familiarity on search behaviors and the effectiveness of the interface.

METHODS

A real user, user search request and real system approach was used for the study. Unlike tradition IR evaluation, where assigned tasks were used, the participants were asked to search requests of their own. Forty-four researchers in Health Sciences participated in the evaluation - each conducted two research requests of their own, alternately with the proposed interface and the PubMed baseline. Several performance criteria were measured to assess the potential benefits of the experimental interface, including users' assessment of their original and eventual queries, the perceived usefulness of the interfaces, satisfaction with the search results, and the average relevance score of the saved records.

RESULTS

The results show that, when searching for an unfamiliar topic, users were more likely to change their queries, indicating the effect of familiarity on search behaviors. The results also show that the interface scored higher on several of the performance criteria, such as the "goodness" of the queries, perceived usefulness, and user satisfaction. Furthermore, in line with our hypothesis, the proposed interface was relatively more effective when less familiar search requests were attempted.

CONCLUSIONS

Results indicate that there is a selective compatibility between search familiarity and search interface. One implication of the research for system evaluation is the importance of taking into consideration task familiarity when assessing the effectiveness of interactive IR systems.

Improving image retrieval effectiveness via query expansion using MeSH hierarchical structure

OBJECTIVE

We explored two strategies for query expansion utilizing medical subject headings (MeSH) ontology to improve the effectiveness of medical image retrieval systems. In order to achieve greater effectiveness in the expansion, the search text was analyzed to identify which terms were most amenable to being expanded.

DESIGN

To perform the expansions we utilized the hierarchical structure by which the MeSH descriptors are organized. Two strategies for selecting the terms to be expanded in each query were studied. The first consisted of identifying the medical concepts using the unified medical language system metathesaurus. In the second strategy the text of the query was divided into n-grams, resulting in sequences corresponding to MeSH descriptors.

MEASUREMENTS

For the evaluation of the system, we used the collection made available by the ImageCLEF organization in its 2011 medical image retrieval task. The main measure of efficiency employed for evaluating the techniques developed was the mean average precision (MAP).

RESULTS

Both strategies exceeded the average MAP score in the ImageCLEF 2011 competition (0.1644). The n-gram expansion strategy achieved a MAP of 0.2004, which represents an improvement of 21.89% over the average MAP score in the competition. On the other hand, the medical concepts expansion strategy scored 0.2172 in the MAP, representing a 32.11% improvement. This run won the text-based medical image retrieval task in 2011.

CONCLUSIONS

Query expansion exploiting the hierarchical structure of the MeSH descriptors achieved a significant improvement in image retrieval systems.

Assisting consumer health information retrieval with query recommendations

OBJECTIVE

Health information retrieval (HIR) on the Internet has become an important practice for millions of people, many of whom have problems forming effective queries. We have developed and evaluated a tool to assist people in health-related query formation.

DESIGN

We developed the Health Information Query Assistant (HIQuA) system. The system suggests alternative/additional query terms related to the user's initial query that can be used as building blocks to construct a better, more specific query. The recommended terms are selected according to their semantic distance from the original query, which is calculated on the basis of concept co-occurrences in medical literature and log data as well as semantic relations in medical vocabularies.

MEASUREMENTS

An evaluation of the HIQuA system was conducted and a total of 213 subjects participated in the study. The subjects were randomized into 2 groups. One group was given query recommendations and the other was not. Each subject performed HIR for both a predefined and a self-defined task.

RESULTS

The study showed that providing HIQuA recommendations resulted in statistically significantly higher rates of successful queries (odds ratio = 1.66, 95% confidence interval = 1.16-2.38), although no statistically significant impact on user satisfaction or the users' ability to accomplish the predefined retrieval task was found.

CONCLUSION

Providing semantic-distance-based query recommendations can help consumers with query formation during HIR.