Wiki-Surgery? Internal validity of Wikipedia as a medical and surgical reference

Does the packaging of health information affect the assessment of its reliability? A randomized controlled trial protocol

Background Wikipedia is frequently used as a source of health information. However, the quality of its content varies widely across articles. The DISCERN tool is a brief questionnaire developed in 1996 by the Division of Public Health and Primary Health Care of the Institute of Health Sciences of the University of Oxford. They claim it provides users with a valid and reliable way of assessing the quality of written information. However, the DISCERN instrument’s reliability in measuring the quality of online health information, particularly whether or not its scores are affected by reader biases about specific publication sources, has not yet been explored. Methods This study is a double-blind randomized assessment of a Wikipedia article versus a BMJ literature review using a modified version of the DISCERN tool. Participants will include physicians and medical residents from four university campuses in Ontario and British Columbia and will be randomized into one of four study arms. Inferential statistics tests (paired t-test, multi-level ordinal regression, and one-way ANOVA) will be conducted with the data collected from the study. Outcomes The primary outcome of this study will be to determine whether a statistically significant difference in DISCERN scores exists, which could suggest whether or not how health information is packaged influences how it is assessed for quality. Plain Language Summary The internet, and in particular Wikipedia, is an important way for professionals, students and the public to obtain health information. For this reason, the DISCERN tool was developed in 1996 to help users assess the quality of the health information they find. The ability of DISCERN to measure the quality of online health information has been supported with research, but the role of bias has not necessarily been accounted for. Does how the information is packaged influence how the information itself is evaluated? This study will compare the scores assigned to articles in their original format to the same articles in a modified format in order to determine whether the DISCERN tool is able to overcome bias. A significant difference in ratings between original and inverted articles will suggest that the DISCERN tool lacks the ability to overcome bias related to how health information is packaged.

Situating Wikipedia as a health information resource in various contexts: A scoping review

BACKGROUND

Wikipedia's health content is the most frequently visited resource for health information on the internet. While the literature provides strong evidence for its high usage, a comprehensive literature review of Wikipedia's role within the health context has not yet been reported.

OBJECTIVE

To conduct a comprehensive review of peer-reviewed, published literature to learn what the existing body of literature says about Wikipedia as a health information resource and what publication trends exist, if any.

METHODS

A comprehensive literature search in OVID Medline, OVID Embase, CINAHL, LISTA, Wilson's Web, AMED, and Web of Science was performed. Through a two-stage screening process, records were excluded if: Wikipedia was not a major or exclusive focus of the article; Wikipedia was not discussed within the context of a health or medical topic; the article was not available in English, the article was irretrievable, or; the article was a letter, commentary, editorial, or popular media article.

RESULTS

89 articles and conference proceedings were selected for inclusion in the review. Four categories of literature emerged: 1) studies that situate Wikipedia as a health information resource; 2) investigations into the quality of Wikipedia, 3) explorations of the utility of Wikipedia in education, and 4) studies that demonstrate the utility of Wikipedia in research.

CONCLUSION

The literature positions Wikipedia as a prominent health information resource in various contexts for the public, patients, students, and practitioners seeking health information online. Wikipedia's health content is accessed frequently, and its pages regularly rank highly in Google search results. While Wikipedia itself is well into its second decade, the academic discourse around Wikipedia within the context of health is still young and the academic literature is limited when attempts are made to understand Wikipedia as a health information resource. Possibilities for future research will be discussed.

The Evolution of the Concept of Semantic Web in the Context of Wikipedia: An Exploratory Approach to Study the Collective Conceptualization in a Digital Collaborative Environment

Wikipedia, as a “social machine”, is a privileged place to observe the collective construction of concepts without central control. Based on Dahlberg’s theory of concept, and anchored in the pragmatism of Hjørland—in which the concepts are socially negotiated meanings—the evolution of the concept of semantic web (SW) was analyzed in the English version of Wikipedia. An exploratory, descriptive, and qualitative study was designed and we identified 26 different definitions (between 12 July 2001 and 31 December 2017), of which eight are of particular relevance for their duration, with the latter being the two recorded at the end of the analyzed period. According to them, SW: “is an extension of the web” and “is a Web of Data”; the latter, used as a complementary definition, links to Berners-Lee’s publications. In Wikipedia, the evolution of the SW concept appears to be based on the search for the use of non-technical vocabulary and the control of authority carried out by the debate. As a space for collective bargaining of meanings, the Wikipedia study may bring relevant contributions to a community’s understanding of a particular concept and how it evolves over time.

Collaborative writing applications in healthcare: effects on professional practice and healthcare outcomes

BACKGROUND

Collaborative writing applications (CWAs), such as wikis and Google Documents, hold the potential to improve the use of evidence in both public health and healthcare. Although a growing body of literature indicates that CWAs could have positive effects on healthcare, such as improved collaboration, behavioural change, learning, knowledge management, and adaptation of knowledge to local context, this has never been assessed systematically. Moreover, several questions regarding safety, reliability, and legal aspects exist.

OBJECTIVES

The objectives of this review were to (1) assess the effects of the use of CWAs on process (including the behaviour of healthcare professionals) and patient outcomes, (2) critically appraise and summarise current evidence on the use of resources, costs, and cost-effectiveness associated with CWAs to improve professional practices and patient outcomes, and (3) explore the effects of different CWA features (e.g. open versus closed) and different implementation factors (e.g. the presence of a moderator) on process and patient outcomes.

SEARCH METHODS

We searched CENTRAL, MEDLINE, Embase, and 11 other electronic databases. We searched the grey literature, two trial registries, CWA websites, individual journals, and conference proceedings. We also contacted authors and experts in the field. We did not apply date or language limits. We searched for published literature to August 2016, and grey literature to September 2015.

SELECTION CRITERIA

We included randomised controlled trials (RCTs), non-randomised controlled trials (NRCTs), controlled before-and-after (CBA) studies, interrupted time series (ITS) studies, and repeated measures studies (RMS), in which CWAs were used as an intervention to improve the process of care, patient outcomes, or healthcare costs.

DATA COLLECTION AND ANALYSIS

Teams of two review authors independently assessed the eligibility of studies. Disagreements were resolved by discussion, and when consensus was not reached, a third review author was consulted.

MAIN RESULTS

We screened 11,993 studies identified from the electronic database searches and 346 studies from grey literature sources. We analysed the full text of 99 studies. None of the studies met the eligibility criteria; two potentially relevant studies are ongoing.

AUTHORS' CONCLUSIONS

While there is a high number of published studies about CWAs, indicating that this is an active field of research, additional studies using rigorous experimental designs are needed to assess their impact and cost-effectiveness on process and patient outcomes.

Accuracy and completeness of drug information in Wikipedia: a comparison with standard textbooks of pharmacology

The online resource Wikipedia is increasingly used by students for knowledge acquisition and learning. However, the lack of a formal editorial review and the heterogeneous expertise of contributors often results in skepticism by educators whether Wikipedia should be recommended to students as an information source. In this study we systematically analyzed the accuracy and completeness of drug information in the German and English language versions of Wikipedia in comparison to standard textbooks of pharmacology. In addition, references, revision history and readability were evaluated. Analysis of readability was performed using the Amstad readability index and the Erste Wiener Sachtextformel. The data on indication, mechanism of action, pharmacokinetics, adverse effects and contraindications for 100 curricular drugs were retrieved from standard German textbooks of general pharmacology and compared with the corresponding articles in the German language version of Wikipedia. Quantitative analysis revealed that accuracy of drug information in Wikipedia was 99.7% ± 0.2% when compared to the textbook data. The overall completeness of drug information in Wikipedia was 83.8 ± 1.5% (p < 0.001). Completeness varied in-between categories, and was lowest in the category "pharmacokinetics" (68.0% ± 4.2%; p < 0.001) and highest in the category "indication" (91.3% ± 2.0%) when compared to the textbook data overlap. Similar results were obtained for the English language version of Wikipedia. Of the drug information missing in Wikipedia, 62.5% was rated as didactically non-relevant in a qualitative re-evaluation study. Drug articles in Wikipedia had an average of 14.6 ± 1.6 references and 262.8 ± 37.4 edits performed by 142.7 ± 17.6 editors. Both Wikipedia and textbooks samples had comparable, low readability. Our study suggests that Wikipedia is an accurate and comprehensive source of drug-related information for undergraduate medical education.

Wikis and collaborative writing applications in health care: a scoping review

Background

Collaborative writing applications (eg, wikis and Google Documents) hold the potential to improve the use of evidence in both public health and health care. The rapid rise in their use has created the need for a systematic synthesis of the evidence of their impact as knowledge translation (KT) tools in the health care sector and for an inventory of the factors that affect their use.

Objective

Through the Levac six-stage methodology, a scoping review was undertaken to explore the depth and breadth of evidence about the effective, safe, and ethical use of wikis and collaborative writing applications (CWAs) in health care.

Methods

Multiple strategies were used to locate studies. Seven scientific databases and 6 grey literature sources were queried for articles on wikis and CWAs published between 2001 and September 16, 2011. In total, 4436 citations and 1921 grey literature items were screened. Two reviewers independently reviewed citations, selected eligible studies, and extracted data using a standardized form. We included any paper presenting qualitative or quantitative empirical evidence concerning health care and CWAs. We defined a CWA as any technology that enables the joint and simultaneous editing of a webpage or an online document by many end users. We performed qualitative content analysis to identify the factors that affect the use of CWAs using the Gagnon framework and their effects on health care using the Donabedian framework.

Results

Of the 111 studies included, 4 were experimental, 5 quasi-experimental, 5 observational, 52 case studies, 23 surveys about wiki use, and 22 descriptive studies about the quality of information in wikis. We classified them by theme: patterns of use of CWAs (n=26), quality of information in existing CWAs (n=25), and CWAs as KT tools (n=73). A high prevalence of CWA use (ie, more than 50%) is reported in 58% (7/12) of surveys conducted with health care professionals and students. However, we found only one longitudinal study showing that CWA use is increasing in health care. Moreover, contribution rates remain low and the quality of information contained in different CWAs needs improvement. We identified 48 barriers and 91 facilitators in 4 major themes (factors related to the CWA, users’ knowledge and attitude towards CWAs, human environment, and organizational environment). We also found 57 positive and 23 negative effects that we classified into processes and outcomes.

Conclusions

Although we found some experimental and quasi-experimental studies of the effectiveness and safety of CWAs as educational and KT interventions, the vast majority of included studies were observational case studies about CWAs being used by health professionals and patients. More primary research is needed to find ways to address the different barriers to their use and to make these applications more useful for different stakeholders.

An evaluation of Wikipedia as a resource for patient education in nephrology

Wikipedia, a multilingual online encyclopedia, is a common starting point for patient medical searches. As its articles can be authored and edited by anyone worldwide, the credibility of the medical content of Wikipedia has been openly questioned. Wikipedia medical articles have also been criticized as too advanced for the general public. This study assesses the comprehensiveness, reliability, and readability of nephrology articles on Wikipedia. The International Statistical Classification of Diseases and Related problems, 10th Edition (ICD-10) diagnostic codes for nephrology (N00-N29.8) were used as a topic list to investigate the English Wikipedia database. Comprehensiveness was assessed by the proportion of ICD-10 codes that had corresponding articles. Reliability was measured by both the number of references per article and proportion of references from substantiated sources. Finally, readability was assessed using three validated indices (Flesch-Kincaid grade level, Automated readability index, and Flesch reading ease). Nephrology articles on Wikipedia were relatively comprehensive, with 70.5% of ICD-10 codes being represented. The articles were fairly reliable, with 7.1 ± 9.8 (mean ± SD) references per article, of which 59.7 ± 35.0% were substantiated references. Finally, all three readability indices determined that nephrology articles are written at a college level. Wikipedia is a comprehensive and fairly reliable medical resource for nephrology patients that is written at a college reading level. Accessibility of this information for the general public may be improved by hosting it at alternative Wikipedias targeted at a lower reading level, such as the Simple English Wikipedia.

Capturing intra-operative safety information using surgical wikis

BACKGROUND

Expert surgeons use a mass of intra-operative information, as well as pre- and post-operative information to complete operations safely. Trainees acquired this intra-operative knowledge at the operating table, now largely diminished by the working time directive. Wikis offer unexplored approaches to capturing and disseminating expert knowledge to further promote safer surgery for the trainee.

METHODS

Grafting an abdominal aortic aneurysm represents a potentially high-risk operation demanding extreme safety measures. Operative details, presented on a surgical wiki in the form of a script and content analysed to classify types of safety information.

RESULTS

The intra-operative part of the script contained 2,743 items of essential surgical information, comprising 21 sections, 405 steps and 2,317 items of back-up information; 155 (5.7%) of them were also specific intra-operative safety checks. Best case scenarios consisted of 1,077 items of intra-operative information, 69 of which were safety checks. Worse case and rare scenarios required a further 1,666 items of information, including 86 safety checks.

CONCLUSIONS

Wikis are relevant to surgical practice specifically as a platform for knowledge sharing and optimising the available operating time of trainees, as a very large amount of minutely detailed information essential for a safe major operation can be captured.

[Presence and adequacy of pharmaceutical preparations in the Spanish edition of Wikipedia]

Objetivo

Determinar la presencia y analizar la adecuación de los principios activos farmacológicos en la edición española de la Wikipedia.

Diseño

Estudio descriptivo transversal.

Emplazamiento

Los términos a estudiar se obtuvieron del Vademécum, UBM Medica Spain S.A. (http://vademécum.es).

Participantes

Se calculó, a partir de los principios activos existentes en el Vademécum, una muestra mediante muestreo aleatorio simple sin reposición, efectuando la estimación de parámetros poblacionales (valor esperado aproximado a 0,5, precisión del intervalo 0,05 y nivel de confianza = 0,95) en una población infinita.

Método

La existencia de la terminología y su adecuación se constató accediendo a la edición española de la Wikipedia. Se consideró el término adecuado cuando la información contenía correctamente su uso, la posología y los efectos secundarios.

Mediciones principales

Las variables cualitativas dicotómicas sí/no (presencia del principio activo, uso, posología, efectos secundarios, adecuación) se describieron por su frecuencia y porcentaje, las cuantitativas (consultas, actualidad) mediante su media y desviación estándar. La existencia de asociación, entre variables cualitativas, se analizó mediante la prueba de ji al cuadrado y la significación de la diferencia de medias, para muestras independientes, mediante la prueba de la t de Student.

Resultados

De la muestra a estudio (n = 386) se determinó la existencia de 171 términos, siendo 15 adecuados. Se observaron diferencias significativas entre la adecuación y la posología (p < 0,001) y con los efectos secundarios (p < 0,001), no observándose con su uso (p = 0,193).

Conclusiones

Las entradas sobre principios activos farmacológicos en la edición española de la Wikipedia son aún escasas y la adecuación de su información sigue siendo inconsistente.

Personal name vocabularies as linked open data: A case study of jazz artist names

This article describes the process of constructing a vocabulary of personal names of jazz artists in the form of Linked Open Data (LOD). Created as a name directory to support the development of the Linked Jazz project, it provides a case study that demonstrates the value and the challenges of developing a domain-specific vocabulary tool that draws upon the semantics of DBpedia, a major LOD dataset. The article also addresses possible strategies for enhancing the directory to make it a more robust personal name vocabulary.

Quality of information sources about mental disorders: a comparison of Wikipedia with centrally controlled web and printed sources

BACKGROUND

Although mental health information on the internet is often of poor quality, relatively little is known about the quality of websites, such as Wikipedia, that involve participatory information sharing. The aim of this paper was to explore the quality of user-contributed mental health-related information on Wikipedia and compare this with centrally controlled information sources.

METHOD

Content on 10 mental health-related topics was extracted from 14 frequently accessed websites (including Wikipedia) providing information about depression and schizophrenia, Encyclopaedia Britannica, and a psychiatry textbook. The content was rated by experts according to the following criteria: accuracy, up-to-dateness, breadth of coverage, referencing and readability.

RESULTS

Ratings varied significantly between resources according to topic. Across all topics, Wikipedia was the most highly rated in all domains except readability.

CONCLUSIONS

The quality of information on depression and schizophrenia on Wikipedia is generally as good as, or better than, that provided by centrally controlled websites, Encyclopaedia Britannica and a psychiatry textbook.

Wikis and collaborative writing applications in health care: a scoping review protocol

The rapid rise in the use of collaborative writing applications (eg, wikis, Google Documents, and Google Knol) has created the need for a systematic synthesis of the evidence of their impact as knowledge translation (KT) tools in the health care sector and for an inventory of the factors that affect their use. While researchers have conducted systematic reviews on a range of software-based information and communication technologies as well as other social media (eg, virtual communities of practice, virtual peer-to-peer communities, and electronic support groups), none have reviewed collaborative writing applications in the medical sector. The overarching goal of this project is to explore the depth and breadth of evidence for the use of collaborative writing applications in health care. Thus, the purposes of this scoping review will be to (1) map the literature on collaborative writing applications; (2) compare the applications’ features; (3) describe the evidence of each application’s positive and negative effects as a KT intervention in health care; (4) inventory and describe the barriers and facilitators that affect the applications’ use; and (5) produce an action plan and a research agenda. A six-stage framework for scoping reviews will be used: (1) identifying the research question; (2) identifying relevant studies within the selected databases (using the EPPI-Reviewer software to classify the studies); (3) selecting studies (an iterative process in which two reviewers search the literature, refine the search strategy, and review articles for inclusion); (4) charting the data (using EPPI-Reviewer’s data-charting form); (5) collating, summarizing, and reporting the results (performing a descriptive, numerical, and interpretive synthesis); and (6) consulting knowledge users during three planned meetings. Since this scoping review concerns the use of collaborative writing applications as KT interventions in health care, we will use the Knowledge to Action (KTA) framework to describe and compare the various studies and collaborative writing projects we find. In addition to guiding the use of collaborative writing applications in health care, this scoping review will advance the science of KT by testing tools that could be used to evaluate other social media. We also expect to identify areas that require further systematic reviews and primary research and to produce a highly relevant research agenda that explores and leverages the potential of collaborative writing software. To date, this is the first study to use the KTA framework to study the role collaborative writing applications in KT, and the first to involve three national and international institutional knowledge users as part of the research process.

Patient-oriented cancer information on the internet: a comparison of wikipedia and a professionally maintained database

PURPOSE

A wiki is a collaborative Web site, such as Wikipedia, that can be freely edited. Because of a wiki's lack of formal editorial control, we hypothesized that the content would be less complete and accurate than that of a professional peer-reviewed Web site. In this study, the coverage, accuracy, and readability of cancer information on Wikipedia were compared with those of the patient-orientated National Cancer Institute's Physician Data Query (PDQ) comprehensive cancer database.

METHODS

For each of 10 cancer types, medically trained personnel scored PDQ and Wikipedia articles for accuracy and presentation of controversies by using an appraisal form. Reliability was assessed by using interobserver variability and test-retest reproducibility. Readability was calculated from word and sentence length.

RESULTS

Evaluators were able to rapidly assess articles (18 minutes/article), with a test-retest reliability of 0.71 and interobserver variability of 0.53. For both Web sites, inaccuracies were rare, less than 2% of information examined. PDQ was significantly more readable than Wikipedia: Flesch-Kincaid grade level 9.6 versus 14.1. There was no difference in depth of coverage between PDQ and Wikipedia (29.9, 34.2, respectively; maximum possible score 72). Controversial aspects of cancer care were relatively poorly discussed in both resources (2.9 and 6.1 for PDQ and Wikipedia, respectively, NS; maximum possible score 18). A planned subanalysis comparing common and uncommon cancers demonstrated no difference.

CONCLUSION

Although the wiki resource had similar accuracy and depth as the professionally edited database, it was significantly less readable. Further research is required to assess how this influences patients' understanding and retention.

The sources and popularity of online drug information: an analysis of top search engine results and web page views

BACKGROUND

The Internet has become a popular source of health information. However, there is little information on what drug information and which Web sites are being searched.

OBJECTIVE

To investigate the sources of online information about prescription drugs by assessing the most common Web sites returned in online drug searches and to assess the comparative popularity of Web pages for particular drugs.

METHODS

This was a cross-sectional study of search results for the most commonly dispensed drugs in the US (n=278 active ingredients) on 4 popular search engines: Bing, Google (both US and Canada), and Yahoo. We determined the number of times a Web site appeared as the first result. A linked retrospective analysis counted Wikipedia page hits for each of these drugs in 2008 and 2009.

RESULTS

About three quarters of the first result on Google USA for both brand and generic names linked to the National Library of Medicine. In contrast, Wikipedia was the first result for approximately 80% of generic name searches on the other 3 sites. On these other sites, over two thirds of brand name searches led to industry-sponsored sites. The Wikipedia pages with the highest number of hits were mainly for opiates, benzodiazepines, antibiotics, and antidepressants.

CONCLUSIONS

Wikipedia and the National Library of Medicine rank highly in online drug searches. Further, our results suggest that patients most often seek information on drugs with the potential for dependence, for stigmatized conditions, that have received media attention, and for episodic treatments. Quality improvement efforts should focus on these drugs.

Wikipedia: a key tool for global public health promotion

The Internet has become an important health information resource for patients and the general public. Wikipedia, a collaboratively written Web-based encyclopedia, has become the dominant online reference work. It is usually among the top results of search engine queries, including when medical information is sought. Since April 2004, editors have formed a group called WikiProject Medicine to coordinate and discuss the English-language Wikipedia's medical content. This paper, written by members of the WikiProject Medicine, discusses the intricacies, strengths, and weaknesses of Wikipedia as a source of health information and compares it with other medical wikis. Medical professionals, their societies, patient groups, and institutions can help improve Wikipedia's health-related entries. Several examples of partnerships already show that there is enthusiasm to strengthen Wikipedia's biomedical content. Given its unique global reach, we believe its possibilities for use as a tool for worldwide health promotion are underestimated. We invite the medical community to join in editing Wikipedia, with the goal of providing people with free access to reliable, understandable, and up-to-date health information.

Prosocial behavior on the Net

Volunteers and charitable organizations contribute significantly to community welfare through their prosocial behavior: that is, discretionary behavior such as assisting, comforting, sharing, and cooperating intended to help worthy beneficiaries. This essay focuses on prosocial behavior on the Internet. It describes how offline charitable organizations are using the Net to become more efficient and effective. It also considers entirely new models of Net-based volunteer behavior directed at creating socially beneficial information goods and services. After exploring the scope and diversity of online prosocial behavior, the essay focuses on ways to encourage this kind of behavior through appropriate task and social structures, motivational signals, and trust indicators. It concludes by asking how local offline communities ultimately could be diminished or strengthened as prosocial behavior increases online.

Scope, completeness, and accuracy of drug information in Wikipedia

BACKGROUND

With the advent of Web 2.0 technologies, user-edited online resources such as Wikipedia are increasingly tapped for information. However, there is little research on the quality of health information found in Wikipedia.

OBJECTIVE

To compare the scope, completeness, and accuracy of drug information in Wikipedia with that of a free, online, traditionally edited database (Medscape Drug Reference [MDR]).

METHODS

Wikipedia and MDR were assessed on 8 categories of drug information. Questions were constructed and answers were verified with authoritative resources. Wikipedia and MDR were evaluated according to scope (breadth of coverage) and completeness. Accuracy was tracked by factual errors and errors of omission. Descriptive statistics were used to summarize the components. Fisher's exact test was used to compare scope and paired Student's t-test was used to compare current results in Wikipedia with entries 90 days prior to the current access.

RESULTS

Wikipedia was able to answer significantly fewer drug information questions (40.0%) compared with MDR (82.5%; p < 0.001). Wikipedia performed poorly regarding information on dosing, with a score of 0% versus the MDR score of 90.0%. Answers found in Wikipedia were 76.0% complete, while MDR provided answers that were 95.5% complete; overall, Wikipedia answers were less complete than those in Medscape (p < 0.001). No factual errors were found in Wikipedia, whereas 4 answers in Medscape conflicted with the answer key; errors of omission were higher in Wikipedia (n = 48) than in MDR (n = 14). There was a marked improvement in Wikipedia over time, as current entries were superior to those 90 days prior (p = 0.024).

CONCLUSIONS

Wikipedia has a more narrow scope, is less complete, and has more errors of omission than the comparator database. Wikipedia may be a useful point of engagement for consumers, but is not authoritative and should only be a supplemental source of drug information.