Lost in publication: Half of all renal practice evidence is published in non-renal journals

Evidence-Based Decision-Making 2: Systematic Reviews and Meta-Analysis

The number of studies published in the biomedical literature has dramatically increased over the last few decades. This massive proliferation of literature makes clinical medicine increasingly complex, and information from multiple studies is often needed to inform a particular clinical decision. However, available studies often vary in their design, methodological quality, and population studied, and may define the research question of interest quite differently. This can make it challenging to synthesize the conclusions of multiple studies. In addition, since even highly cited trials may be challenged over time, clinical decision-making requires ongoing reconciliation of studies which provide different answers to the same question. Because it is often impractical for readers to track down and review all the primary studies, systematic reviews and meta-analyses are an important source of evidence on the diagnosis, prognosis and treatment of any given disease. This chapter summarizes methods for conducting and reading systematic reviews and meta-analyses, as well as describes potential advantages and disadvantages of these publications.

An evidence rating service provided valid correlates of the clinical importance of medical articles and journals

OBJECTIVES

The objective of this study was to determine reliability and validity of McMaster PLUS measures of scientific merit and clinical importance of articles in medical journals.

STUDY DESIGN AND SETTING

Analytic survey of peer-reviewed medical journals was carried out. Articles were qualified for inclusion by meeting (1) scientific criteria and (2) a clinical importance rating threshold. Included articles were sent as e-mail alerts to physicians according to their clinical interests. Internal measures included the number of high-quality, clinically important studies published in source journals and response to alerts. For external validation, we correlated internal measures with the Journal Impact Factor (JIF) and citation in DynaMed Plus (DMP).

RESULTS

We evaluated 34,232 articles from 57 journals. Inclusion criteria were met by 2,638 articles (7.71%). The number of qualifying articles per journal was correlated with the number of articles with high clinical importance ratings (r 0.96, P < 0.001), article alert clicks (r 0.86, P < 0.001), and DMP citations (r 0.99, P < 0.001). Correlation was lower with the JIF (r 0.68, P < 0.01).

CONCLUSIONS

Measures of scientific merit and clinical importance of medical journal articles were strongly correlated with each other, less so with JIFs. Journals varied widely by these measures but, generally, few articles were both scientifically sound and clinically important.

The Top 20 Most Prolific Authors in the American Journal of Neuroradiology: What Is Their Impact?

BACKGROUND AND PURPOSE

Many articles that are relevant to patient care but published in radiology journals may escape notice by clinicians. We sought to determine how often the 20 most prolific American Journal of Neuroradiology (AJNR) authors from 2013 to 2017 published in clinical journals and the extent to which their articles were disseminated into the clinical literature.

MATERIALS AND METHODS

We counted all authors' first- or senior-authored articles in the AJNR from 2013 to 2017 to identify the 20 most prolific authors in AJNR. We searched for these 20 authors' total articles from 2013 to 2017 to determine which were published in radiology or clinical journals and the number of citations received from radiology and clinical journals. Authors were sorted into quartiles according to these metrics, and other descriptive statistics were performed.

RESULTS

The top 20 AJNR authors contributed to 1463 articles during 5 years, including 711 (48.6%) in radiology and 752 (51.4%) in clinical journals. These articles were cited 15,857 times, including 4659 (29.3%) by articles in radiology journals. The more prolific authors published in clinical journals more often (Spearman ρ = 0.65, P = .002) and were cited more (ρ = 0.42, P = .07). Articles published in clinical journals were cited more often (mean, 12.3 clinical, 9.3 radiology general versus 8.7 in AJNR), and whether published in radiology or clinical journals, they were cited more frequently by clinical journals.

CONCLUSIONS

Regardless of where it is published, radiology research is disseminating into the clinical realm. Radiology articles published in clinical journals are cited more often than those published in radiology journals.

Endovascular Treatment for Extracranial Carotid Stenosis

Introduction:

Bibliometric analysis is a quantitative method that can evaluate publications related to a specific topic.

Methods:

A PubMed database search was conducted from 2003 to 2012 using the search term “carotid” AND “stenos*” as a part of the title or abstract.

Results:

A total of 1590 articles were published in 329 different journals. A total of 751 (47.2%) publications were original articles, 1501 (94.4%) were written in English, 153 (9.6%) received funding, 584 (36.7%) were published by the United States, and 673 (42.3%) resulted from multidisciplinary collaboration. Of the original articles, 538 (71.6%) had retrospective design and 275 (36.6%) had sample size of <50. Vascular surgery departments produced the most articles (n = 339, 21.3%), followed by radiology (n = 270, 17.0%), cardiology (n = 260, 16.4%), neurosurgery (n = 198, 12.5%), and neurology (n = 196, 12.3%). Five major departments published only a small portion (5.1%-26.5%) of articles in their own specialty journals.

Conclusion:

The publication in journals across disciplines and multidisciplinary collaboration are 2 peculiar characteristics of research in this topic.

Evidence-based decision-making 2: Systematic reviews and meta-analysis

The number of studies published in the biomedical literature has dramatically increased over the last few decades. This massive proliferation of literature makes clinical medicine increasingly complex, and information from multiple studies is often needed to inform a particular clinical decision. However, available studies often vary in their design, methodological quality, populations studied and may define the research question of interest quite differently, which can make it challenging to synthesize their conclusions. In addition, since even highly cited trials may be challenged over time, clinical decision-making requires ongoing reconciliation of studies which provide different answers to the same question. Because it is often impractical for readers to track down and review all the primary studies, systematic reviews and meta-analyses are an important source of evidence on the diagnosis, prognosis, and treatment of any given disease. This chapter summarizes methods for conducting and reading systematic reviews and meta-analyses, as well as describing potential advantages and disadvantages of these publications.

Developing evidence-based dentistry skills: how to interpret randomized clinical trials and systematic reviews

Decision-making based on reliable evidence is more likely to lead to effective and efficient treatments. Evidence-based dentistry was developed, similarly to evidence-based medicine, to help clinicians apply current and valid research findings into their own clinical practice. Interpreting and appraising the literature is fundamental and involves the development of evidence-based dentistry (EBD) skills. Systematic reviews (SRs) of randomized controlled trials (RCTs) are considered to be evidence of the highest level in evaluating the effectiveness of interventions. Furthermore, the assessment of the report of a RCT, as well as a SR, can lead to an estimation of how the study was designed and conducted.

Neurointerventional research between 2003 and 2012: slow growth, high interdisciplinary collaboration, and a low level of funding

BACKGROUND AND PURPOSE

Neurointerventional therapy of cerebrovascular disease is a greatly expanding field across many specialty disciplines. The goal of this study was to analyze the characteristics and trends of scientific publications that focused on neurointervention during the past decade.

MATERIALS AND METHODS

A bibliometric evaluation of neurointerventional research published between 2003 and 2012 was conducted by using the PubMed data base. Analyzed parameters included the year of publication, type of document, language of the article, topic, declared funding, country of origin, type of collaboration between disciplines, the first author's specialty, and subject category and the Impact Factor of the publishing journal.

RESULTS

Between 2003 and 2012, a total of 2123 articles were published, of which 1107 (52.1%) were original articles, 1948 (91.8%) were written in English, 192 (9.0%) received funding, 661 (31.1%) were published by the United States, and 1060 (49.9%) resulted from interdisciplinary collaboration. Neurosurgery departments produced the most articles (n = 910, 42.9%), followed by radiology (n = 747, 35.2%) and neurology (n = 270, 12.7%). The time-trend analysis in the number of publications demonstrated slow growth from 2003 to 2012, with an average annual growth rate of +6.0%.

CONCLUSIONS

The fields of neurosurgery, radiology, and neurology have contributed substantially to neurointervention research. Slow growth, high interdisciplinary collaboration, and a low level of funding are peculiar characteristics of research in this field.

High-performance information search filters for CKD content in PubMed, Ovid MEDLINE, and EMBASE

BACKGROUND

Finding relevant articles in large bibliographic databases such as PubMed, Ovid MEDLINE, and EMBASE to inform care and future research is challenging. Articles relevant to chronic kidney disease (CKD) are particularly difficult to find because they are often published under different terminology and are found across a wide range of journal types.

STUDY DESIGN

We used computer automation within a diagnostic test assessment framework to develop and validate information search filters to identify CKD articles in large bibliographic databases.

SETTING & PARTICIPANTS

22,992 full-text articles in PubMed, Ovid MEDLINE, or EMBASE.

INDEX TEST

1,374,148 unique search filters.

REFERENCE TEST

We established the reference standard of article relevance to CKD by manual review of all full-text articles using prespecified criteria to determine whether each article contained CKD content or not. We then assessed filter performance by calculating sensitivity, specificity, and positive predictive value for the retrieval of CKD articles. Filters with high sensitivity and specificity for the identification of CKD articles in the development phase (two-thirds of the sample) were then retested in the validation phase (remaining one-third of the sample).

RESULTS

We developed and validated high-performance CKD search filters for each bibliographic database. Filters optimized for sensitivity reached at least 99% sensitivity, and filters optimized for specificity reached at least 97% specificity. The filters were complex; for example, one PubMed filter included more than 89 terms used in combination, including "chronic kidney disease," "renal insufficiency," and "renal fibrosis." In proof-of-concept searches, physicians found more articles relevant to the topic of CKD with the use of these filters.

LIMITATIONS

As knowledge of the pathogenesis of CKD grows and definitions change, these filters will need to be updated to incorporate new terminology used to index relevant articles.

CONCLUSIONS

PubMed, Ovid MEDLINE, and EMBASE can be filtered reliably for articles relevant to CKD. These high-performance information filters are now available online and can be used to better identify CKD content in large bibliographic databases.

Bronchial asthma and chronic obstructive pulmonary disease: research activity in Arab countries

BACKGROUND

Chronic respiratory diseases, like bronchial asthma and chronic obstructive pulmonary disease (COPD), are a worldwide health problem. Quantitative and qualitative assessment of asthma and COPD-related research from Arab countries has not been explored and there are few internationally published reports on such field. The main objectives of this study were to analyze research output originating from Arab countries in the field of bronchial asthma and COPD.

METHODS

Original scientific articles or reviews published from the 22 Arab countries were screened using specific words pertaining to asthma and COPD using Scopus database and search engine. Research productivity was evaluated based on: (a) total and trends of contribution of each Arab country to asthma and COPD research and (b) journals in which researchers from Arab countries published their research.

RESULTS

The total number of original research and review articles published globally about bronchial asthma and COPD was 163,964. The leading country in bronchial asthma and COPD research was United States of America (38,632; 23.56%). Worldwide, Turkey ranked 19th while Israel and Iran ranked 25th and 29th respectively. Among Arab countries, Egypt and Kingdom of Saudi Arabia came on positions 39th and 43rd, respectively. A total of 1,304 documents about bronchial asthma and COPD were published from Arab countries which represents 0.8% of the global research output. Research in bronchial asthma was almost double that in COPD. Research from Arab countries was low and showed a significant increase after 2000. Approximately 12% of research activity in asthma and COPD from Arab countries was published in Saudi Medical Journal, Annals of Saudi Medicine, Eastern Mediterranean Health Journal and Tunisie Medicale. Kingdom of Saudi Arabia, with a total publication of 353 (27.07%) ranked first among the Arab countries while University of Kuwait was the most productive institution with a total of 123 (9.43%) documents.

CONCLUSIONS

The present data showed relatively low research productivity about bronchial asthma and COPD in Arab countries. Research output can be improved by investing more in international and national collaborative research projects in the field of asthma and COPD.

Assessing urology and nephrology research activity in Arab countries using ISI web of science bibliometric database

BACKGROUND

Bibliometric analysis is increasingly being used for research assessment. The main objective of this study was to assess research output in Urology and Nephrology subject from the Arab countries. Original scientific articles or reviews published from the 21 Arab countries in "Urology and Nephrology" subject were screened using the ISI Web of Science database. Research productivity was evaluated based on a methodology developed and used in other bibliometric studies by analyzing the annual productivity, names of journals, citations; top 10 active institution and authors as well as country contribution to Urology and Nephrology research.

RESULTS

Three thousand and seventy six documents in "urology and nephrology" subject category were retrieved from 104 journals. This represents 1.4% of the global research output in "urology and nephrology". Four hundred and two documents (12.66%) were published in Annales D Urologie Journal. The h-index of the retrieved documents was 57. The total number of citations, at the time of data analysis, was 30401 with an average citation of 9.57 per document. Egypt, with a total publication of 1284 (40.43%) ranked first among the Arab countries in "urology and nephrology" subject category. Mansoura University in Egypt was the most productive institution with a total of 561 (15.33%) documents. Arab researchers collaborated most with researchers from the United States of America (226; 7.12%) in urology and nephrology research.

CONCLUSION

The present data reveals a good contribution of some Arab countries to the field of "urology and nephrology". More efforts are needed by some other Arab countries to bridge the gap in urology and nephrology research. Overall, the quality of urology/nephrology research is considered relatively high as measured by h-index. Cooperation in urology/nephrology research should be encouraged in the Arab world to bridge the gap with that from developed countries.

Characteristics and trends of research on positron emission tomography: a bibliometric analysis, 2002-2012

OBJECTIVE

We performed a bibliometric analysis of scientific publications focused on positron emission tomography (PET) over a 10-year period.

METHODS

The MEDLINE and ISI Web of Knowledge databases were searched for English language original articles focused on PET in SCI/SCIE-indexed journals in 2002, 2007, and 2012. We selected the documents with titles that included "PET" or "positron emission." The following information was obtained from each article: journal (year of publication, title, subject category, and impact factor), subspecialty, imaging modality used, tracer, species, sample size, number of authors, affiliation of the first author, declared funding, and country of origin.

RESULTS

The yearly publication on PET increased from 547 (2002) to 986 (2007), and 1838 (2012). A total of 1753 (52.0%) articles were published in journals in the "Radiology, Nuclear Medicine & Medical Imaging" category, 1512 (44.9%) were in the subspecialty of oncology, 3245 (96.3%) used PET or PET/CT, 1698 (50.4%) used 2-fluoro-2-deoxy-D-glucose (FDG) as the radiotracer, 2378 (70.5%) were human studies, 1294 (38.4%) had a sample size of <20, 1674 (49.7%) had >7 authors, 779 (23.1%) were written by a first author from a department of nuclear medicine, and 1337 (39.7%) were supported by government funding. The United States published 948 studies (28.1%) followed by Japan (345, 10.2%) and Germany (335, 9.9%). In the time trend analysis oncology subspecialty, PET/MR as the imaging modality, FDG as the tracer, sample size>50, number of authors>7, radiology department affiliation of the first author, and government funding exhibited significantly positive trends.

CONCLUSIONS

The number of publication concerning PET has increased rapidly over the last decade. This bibliometric analysis revealed characteristics and trends of current PET research that provides useful information to researchers.

High-performance information search filters for acute kidney injury content in PubMed, Ovid Medline and Embase

BACKGROUND

We frequently fail to identify articles relevant to the subject of acute kidney injury (AKI) when searching the large bibliographic databases such as PubMed, Ovid Medline or Embase. To address this issue, we used computer automation to create information search filters to better identify articles relevant to AKI in these databases.

METHODS

We first manually reviewed a sample of 22 992 full-text articles and used prespecified criteria to determine whether each article contained AKI content or not. In the development phase (two-thirds of the sample), we developed and tested the performance of >1.3-million unique filters. Filters with high sensitivity and high specificity for the identification of AKI articles were then retested in the validation phase (remaining third of the sample).

RESULTS

We succeeded in developing and validating high-performance AKI search filters for each bibliographic database with sensitivities and specificities in excess of 90%. Filters optimized for sensitivity reached at least 97.2% sensitivity, and filters optimized for specificity reached at least 99.5% specificity. The filters were complex; for example one PubMed filter included >140 terms used in combination, including 'acute kidney injury', 'tubular necrosis', 'azotemia' and 'ischemic injury'. In proof-of-concept searches, physicians found more articles relevant to topics in AKI with the use of the filters.

CONCLUSIONS

PubMed, Ovid Medline and Embase can be filtered for articles relevant to AKI in a reliable manner. These high-performance information filters are now available online and can be used to better identify AKI content in large bibliographic databases.

Accessing biomedical literature in the current information landscape

Biomedical and life sciences literature is unique because of its exponentially increasing volume and interdisciplinary nature. Biomedical literature access is essential for several types of users including biomedical researchers, clinicians, database curators, and bibliometricians. In the past few decades, several online search tools and literature archives, generic as well as biomedicine specific, have been developed. We present this chapter in the light of three consecutive steps of literature access: searching for citations, retrieving full text, and viewing the article. The first section presents the current state of practice of biomedical literature access, including an analysis of the search tools most frequently used by the users, including PubMed, Google Scholar, Web of Science, Scopus, and Embase, and a study on biomedical literature archives such as PubMed Central. The next section describes current research and the state-of-the-art systems motivated by the challenges a user faces during query formulation and interpretation of search results. The research solutions are classified into five key areas related to text and data mining, text similarity search, semantic search, query support, relevance ranking, and clustering results. Finally, the last section describes some predicted future trends for improving biomedical literature access, such as searching and reading articles on portable devices, and adoption of the open access policy.

Reporting quality of systematic review abstracts in leading oral implantology journals

OBJECTIVES

Abstracts of systematic reviews are of critical importance, as consumers of research often do not access the full text. This study aimed to assess the reporting quality of systematic review (SR) abstracts in leading oral implantology journals.

METHODS

Six specialty journals were screened for SRs between 2008 and 2012. A 16-item checklist, based on the PRISMA statement, was used to examine the completeness of abstract reporting.

RESULTS

Ninety-three SR abstracts were included in this study. The majority were published in Clinical Oral Implants Research (43%). The mean overall reporting quality score was 72.5% (95% CI: 70.8-74.2). Most abstracts were structured (97.9%), adequately reporting objectives (97.9%) and conclusions (93.6%). Conversely, inadequate reporting of methods of the study, background (79.6%), appraisal (65.6%), and data synthesis (65.6%) were observed. Registration of reviews was not reported in any of the included abstracts. Multivariate analysis revealed no difference in reporting quality with respect to continent, number of authors, or meta-analysis conduct.

CONCLUSIONS

The results of this study suggest that the reporting quality of systematic review abstracts in implantology journals requires further improvement.

CLINICAL SIGNIFICANCE

Better reporting of SR abstracts is particularly important in ensuring the reliability of research findings, ultimately promoting the practice of evidence-based dentistry. Optimal reporting of SR abstracts should be encouraged, preferably by endorsing the PRISMA for abstracts guidelines.

Retrieving clinical evidence: a comparison of PubMed and Google Scholar for quick clinical searches

BACKGROUND

Physicians frequently search PubMed for information to guide patient care. More recently, Google Scholar has gained popularity as another freely accessible bibliographic database.

OBJECTIVE

To compare the performance of searches in PubMed and Google Scholar.

METHODS

We surveyed nephrologists (kidney specialists) and provided each with a unique clinical question derived from 100 renal therapy systematic reviews. Each physician provided the search terms they would type into a bibliographic database to locate evidence to answer the clinical question. We executed each of these searches in PubMed and Google Scholar and compared results for the first 40 records retrieved (equivalent to 2 default search pages in PubMed). We evaluated the recall (proportion of relevant articles found) and precision (ratio of relevant to nonrelevant articles) of the searches performed in PubMed and Google Scholar. Primary studies included in the systematic reviews served as the reference standard for relevant articles. We further documented whether relevant articles were available as free full-texts.

RESULTS

Compared with PubMed, the average search in Google Scholar retrieved twice as many relevant articles (PubMed: 11%; Google Scholar: 22%; P<.001). Precision was similar in both databases (PubMed: 6%; Google Scholar: 8%; P=.07). Google Scholar provided significantly greater access to free full-text publications (PubMed: 5%; Google Scholar: 14%; P<.001).

CONCLUSIONS

For quick clinical searches, Google Scholar returns twice as many relevant articles as PubMed and provides greater access to free full-text articles.

An analysis of radiological research publications in high impact general medical journals between 1996 and 2010

OBJECTIVE

To evaluate scientific papers published by radiologists in high impact general medical journals between 1996 and 2010.

METHODS

A MEDLINE search was performed in five high impact general medical journals (AIM, BMJ, JAMA, Lancet, and NEJM) for all articles of which a radiologist was the first author between 1996 and 2010. The following information was abstracted from the original articles: radiological subspecialty, imaging technique used, type of research, sample size, study design, statistical analysis, study outcome, declared funding, number of authors, collaboration, and country of the first author.

RESULTS

Of 216 (0.19%) articles were published by radiologists in five general medical journals between 1996 and 2010, 83 were original articles. Fifteen (18.1%) original articles were concerned with the field of vascular/interventional radiology, 24 (28.9%) used combined imaging techniques, 76 (91.6%) were clinical research, 63 (75.9%) had a sample size of >50, 65 (78.3%) were prospective, 78 (94.0%) performed statistical analysis, 83 (100%) showed positive study outcomes, 57 (68.7%) were funded, 49 (59.0%) had from four to seven authors, and 79 (95.2%) were collaborative studies.

CONCLUSIONS

A very small number (0.19%) in five high impact general medical journals was published by radiologists between 1996 and 2010.

Availability of renal literature in six bibliographic databases

BACKGROUND: When searching for renal literature, nephrologists must choose between several different bibliographic databases. We compared the availability of renal clinical studies in six major bibliographic databases. METHODS: We gathered 151 renal systematic reviews, which collectively contained 2195 unique citations referencing primary studies in the form of journal articles, meeting articles or meeting abstracts published between 1963 and 2008. We searched for each citation in three subscription-free bibliographic databases (PubMed, Google Scholar and Scirus) and three subscription-based databases (EMBASE, Ovid-MEDLINE and ISI Web of Knowledge). For the subscription-free databases, we determined which full-text journal articles were available free of charge via links to the article source. RESULTS: The proportion of journal articles contained within each of the six databases ranged from 96 to 97%; results were similar for meeting articles. Availability of meeting abstracts was poor, ranging from 0 to 37% (P < 0.01) with ISI Web of Knowledge containing the largest proportion [37%, 95% confidence interval (95% CI) 32-43%]. Among the subscription-free databases, free access to full-text articles was highest in Google Scholar (38% free, 95% CI 36-41%), and was only marginally higher (39%) when all subscription-free databases were searched. After 2000, free access to full-text articles increased to 49%. CONCLUSIONS: Over 99% of renal clinical journal articles are available in at least one major bibliographic database. Subscription-free databases provide free full-text access to almost half of the articles published after the year 2000, which may be of particular interest to clinicians in settings with limited access to subscription-based resources.

The evolution of academic performance in nine subspecialties of internal medicine: an analysis of journal citation reports from 1998 to 2010

BACKGROUND

Internal medicine includes several subspecialties. This study aimed to describe change trend of impact factors in different subspecialties of internal medicine during the past 12 years, as well as the developmental differences among each subspecialty, and the possible influencing factors behind these changes and differences.

METHODS

Nine subspecialties of internal medicine were chosen for comparison. All data were collected from the Science Citation Index Expanded and Journal Citation Reports database.

RESULTS

(1) Journal numbers in nine subspecialties increased significantly from 1998 to 2010, with an average increment of 80.23%, in which cardiac and cardiovascular system diseases increased 131.2% rank the first; hematology increased 45% rank the least. (2) Impact Factor in subspecialties of infectious disease, cardiac and cardiovascular system diseases, gastroenterology and hepatology, hematology, endocrinology and metabolism increased significantly (p<0.05), in which gastroenterology and hepatology had the largest increase of 65.4%. (3) Journal impact factor of 0-2 had the largest proportion in all subspecialties. Among the journals with high impact factor (IF>6), hematology had the maximum proportion of 10%, nephrology and respiratory system disease had the minimum of 4%. Among the journal with low impact factor (IF<2), journal in nephrology and allergy had the most (60%), while endocrinology and metabolism had the least (40%). There were differences in median number of IF among the different subspecialties (p<0.05), in which endocrinology and metabolism had the highest, nephrology had the lowest. (4) The highest IF had a correlation with journal numbers and total paper numbers in each field.

CONCLUSION

The IF of internal medicine journals showed an increasingly positive trend, in which gastroenterology and hepatology increase the most. Hematology had more high IF journals. Endocrinology and metabolism had higher average IF. Nephrology remained the lowest position. Numbers of journals and total papers were associated with the highest IF.

Dialysis search filters for PubMed, Ovid MEDLINE, and Embase databases

BACKGROUND AND OBJECTIVES

Physicians frequently search bibliographic databases, such as MEDLINE via PubMed, for best evidence for patient care. The objective of this study was to develop and test search filters to help physicians efficiently retrieve literature related to dialysis (hemodialysis or peritoneal dialysis) from all other articles indexed in PubMed, Ovid MEDLINE, and Embase.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

A diagnostic test assessment framework was used to develop and test robust dialysis filters. The reference standard was a manual review of the full texts of 22,992 articles from 39 journals to determine whether each article contained dialysis information. Next, 1,623,728 unique search filters were developed, and their ability to retrieve relevant articles was evaluated.

RESULTS

The high-performance dialysis filters consisted of up to 65 search terms in combination. These terms included the words "dialy" (truncated), "uremic," "catheters," and "renal transplant wait list." These filters reached peak sensitivities of 98.6% and specificities of 98.5%. The filters' performance remained robust in an independent validation subset of articles.

CONCLUSIONS

These empirically derived and validated high-performance search filters should enable physicians to effectively retrieve dialysis information from PubMed, Ovid MEDLINE, and Embase.

Glomerular disease search filters for Pubmed, Ovid Medline, and Embase: a development and validation study

BACKGROUND

Tools to enhance physician searches of Medline and other bibliographic databases have potential to improve the application of new knowledge in patient care. This is particularly true for articles about glomerular disease, which are published across multiple disciplines and are often difficult to track down. Our objective was to develop and test search filters for PubMed, Ovid Medline, and Embase that allow physicians to search within a subset of the database to retrieve articles relevant to glomerular disease.

METHODS

We used a diagnostic test assessment framework with development and validation phases. We read a total of 22,992 full text articles for relevance and assigned them to the development or validation set to define the reference standard. We then used combinations of search terms to develop 997,298 unique glomerular disease filters. Outcome measures for each filter included sensitivity, specificity, precision, and accuracy. We selected optimal sensitive and specific search filters for each database and applied them to the validation set to test performance.

RESULTS

High performance filters achieved at least 93.8% sensitivity and specificity in the development set. Filters optimized for sensitivity reached at least 96.7% sensitivity and filters optimized for specificity reached at least 98.4% specificity. Performance of these filters was consistent in the validation set and similar among all three databases.

CONCLUSIONS

PubMed, Ovid Medline, and Embase can be filtered for articles relevant to glomerular disease in a reliable manner. These filters can now be used to facilitate physician searching.

The scatter of research: cross sectional comparison of randomised trials and systematic reviews across specialties

OBJECTIVE

To estimate the degree of scatter of reports of randomised trials and systematic reviews, and how the scatter differs among medical specialties and subspecialties.

DESIGN

Cross sectional analysis.

DATA SOURCE

PubMed for all disease relevant randomised trials and systematic reviews published in 2009.

STUDY SELECTION

Randomised trials and systematic reviews of the nine diseases or disorders with the highest burden of disease, and the broader category of disease to which each belonged.

RESULTS

The scatter across journals varied considerably among specialties and subspecialties: otolaryngology had the least scatter (363 trials across 167 journals) and neurology the most (2770 trials across 896 journals). In only three subspecialties (lung cancer, chronic obstructive pulmonary disease, hearing loss) were 10 or fewer journals needed to locate 50% of trials. The scatter was less for systematic reviews: hearing loss had the least scatter (10 reviews across nine journals) and cancer the most (670 reviews across 279 journals). For some specialties and subspecialties the papers were concentrated in specialty journals; whereas for others, few of the top 10 journals were a specialty journal for that area. Generally, little overlap occurred between the top 10 journals publishing trials and those publishing systematic reviews. The number of journals required to find all trials or reviews was highly correlated (r = 0.97) with the number of papers for each specialty/subspecialty.

CONCLUSIONS

Publication rates of speciality relevant trials vary widely, from one to seven trials per day, and are scattered across hundreds of general and specialty journals. Although systematic reviews reduce the extent of scatter, they are still widely scattered and mostly in different journals to those of randomised trials. Personal subscriptions to journals, which are insufficient for keeping up to date with knowledge, need to be supplemented by other methods such as journal scanning services or systems that cover sufficient journals and filter articles for quality and relevance. Few current systems seem adequate.

Impact of PubMed search filters on the retrieval of evidence by physicians

BACKGROUND

Physicians face challenges when searching PubMed for research evidence, and they may miss relevant articles while retrieving too many nonrelevant articles. We investigated whether the use of search filters in PubMed improves searching by physicians.

METHODS

We asked a random sample of Canadian nephrologists to answer unique clinical questions derived from 100 systematic reviews of renal therapy. Physicians provided the search terms that they would type into PubMed to locate articles to answer these questions. We entered the physician-provided search terms into PubMed and applied two types of search filters alone or in combination: a methods-based filter designed to identify high-quality studies about treatment (clinical queries "therapy") and a topic-based filter designed to identify studies with renal content. We evaluated the comprehensiveness (proportion of relevant articles found) and efficiency (ratio of relevant to nonrelevant articles) of the filtered and nonfiltered searches. Primary studies included in the systematic reviews served as the reference standard for relevant articles.

RESULTS

The average physician-provided search terms retrieved 46% of the relevant articles, while 6% of the retrieved articles were relevant (corrected) (the ratio of relevant to nonrelevant articles was 1:16). The use of both filters together produced a marked improvement in efficiency, resulting in a ratio of relevant to nonrelevant articles of 1:5 (16 percentage point improvement; 99% confidence interval 9% to 22%; p < 0.003) with no substantive change in comprehensiveness (44% of relevant articles found; p = 0.55).

INTERPRETATION

The use of PubMed search filters improves the efficiency of physician searches. Improved search performance may enhance the transfer of research into practice and improve patient care.

Finding and evaluating renal evidence: bridging the knowledge gap

Nephrologists often search for clinical information to guide patient care, especially in the chronic kidney disease patient population where information needs are complex. Applying evidence-based principles while searching for current high-quality information can be challenging. This article details the "6S" framework to approaching evidence-based information resources popularized by Haynes and colleagues. This hierarchal structure of resources includes "systems," "summaries," "synopses of syntheses," "syntheses," "synopses of studies," and finally, "studies" with the intent of providing nephrologists with an efficient method of retrieving accurate, high-quality information. Accurate clinical decision support systems are ideal but are yet to reach their full potential. Summaries of best evidence for individual clinical problems are useful (UpToDate, DynaMed, Best Practice, and PIER). Syntheses of articles are an efficient way to understand the available evidence for a clinical question. The Cochrane Renal Review Group is a helpful resource which provides many systematic reviews in renal research. When using bibliographic databases, such PubMed, nephrologists can improve the precision of their search results with information filters. The application of accurate and current clinical knowledge is increasingly important in renal patient care. Better ways of achieving this will have large benefits.

The profile of general practitioners (GPs) who publish in selected family practice journals

Background

Providing support for research is one of the key issues in the ongoing attempts to improve Primary Care. However, when patient care takes up a significant part of a GP's time, conducting research is difficult. In this study we examine the working conditions and profile of GPs who publish in three leading medical journals and propose possible remedial policy actions.

Findings

The authors of all articles published in 2006 and 2007 in three international Family Medicine journals - Annals of Family Medicine, Family Practice, and Journal of Family Practice - were contacted by E-mail. They were asked to complete a questionnaire investigating the following variables: availability of specific time for research, time devoted to research, number of patients attended, and university affiliation. Only GPs were included in the study. Three hundred and ten relevant articles published between 2006 and 2007 were identified and the authors contacted using a survey tool. 124 researchers responded to our questionnaire; 45% of respondents who were not GPs were excluded. On average GPs spent 2.52 days per week and 6.9 hours per day on patient care, seeing 45 patients per week. Seventy-five per cent of GPs had specific time assigned to research, on average 13 hours per week; 79% were affiliated to a university and 69% held teaching positions.

Conclusions

Most GPs who publish original articles in leading journals have time specifically assigned to research as part of their normal working schedule. They see a relatively small number of patients. Improving the working conditions of family physicians who intend to investigate is likely to lead to better research results.

Tackling the growth of the obesity literature: obesity evidence spreads across many journals

OBJECTIVE

This study identified the journals with the highest yield of clinical obesity research articles and surveyed the scatter of such studies across journals. The study exemplifies an approach to establish a journal collection that is likely to contain most new knowledge about a field.

DESIGN AND METHODS

All original studies that were cited in 40 systematic reviews about obesity topics ('included studies') were compiled and journal titles in which they were published were extracted. The journals were ranked by the number of included studies. The highest-yielding journals for clinical obesity and the scatter across journal titles were determined. A subset of these journals was created in MEDLINE (PubMed) to test search recall and precision for high-quality studies of obesity treatment (that is, articles that pass predetermined methodology criteria, including random allocation of participants to comparison groups, assessment of clinical outcomes, and at least 80% follow-up).

RESULTS

Articles in 252 journals were cited in the systematic reviews. The three highest-yielding journals specialized in obesity, but they published only 19.2% of the research, leaving 80.8% scattered across 249 non-obesity journals. The MEDLINE journal subset comprised 241 journals (11 journals were not indexed in MEDLINE) and included 82% of the clinical obesity research articles retrieved by a search for high-quality treatment studies ('recall' of 82%). Of the articles retrieved, 11% were about clinical obesity care ('precision' of 11%), compared with precision of 6% for obesity treatment studies in the full MEDLINE database.

CONCLUSION

Obesity journals captured only a small proportion of the literature on clinical obesity care. Those wishing to keep up in this field will need to develop more inclusive strategies than reading these specialty journals. A journal subset based on these findings may be useful when searching large electronic databases to increase search precision.

Evaluating the impact of MEDLINE filters on evidence retrieval: study protocol

Background

Rather than searching the entire MEDLINE database, clinicians can perform searches on a filtered set of articles where relevant information is more likely to be found. Members of our team previously developed two types of MEDLINE filters. The 'methods' filters help identify clinical research of high methodological merit. The 'content' filters help identify articles in the discipline of renal medicine. We will now test the utility of these filters for physician MEDLINE searching.

Hypothesis

When a physician searches MEDLINE, we hypothesize the use of filters will increase the number of relevant articles retrieved (increase 'recall,' also called sensitivity) and decrease the number of non-relevant articles retrieved (increase 'precision,' also called positive predictive value), compared to the performance of a physician's search unaided by filters.

Methods

We will survey a random sample of 100 nephrologists in Canada to obtain the MEDLINE search that they would first perform themselves for a focused clinical question. Each question we provide to a nephrologist will be based on the topic of a recently published, well-conducted systematic review. We will examine the performance of a physician's unaided MEDLINE search. We will then apply a total of eight filter combinations to the search (filters used in isolation or in combination). We will calculate the recall and precision of each search. The filter combinations that most improve on unaided physician searches will be identified and characterized.

Discussion

If these filters improve search performance, physicians will be able to search MEDLINE for renal evidence more effectively, in less time, and with less frustration. Additionally, our methodology can be used as a proof of concept for the evaluation of search filters in other disciplines.

Bias and dispersal in the animal reintroduction literature

We examined the literature on animal reintroductions to assess the challenges facing individual conservation practitioners who wish to access, synthesize and interpret available evidence to inform their decision making. We undertook an extensive search in eight electronic literature databases, using seven different keyword combinations, and added the content of four bibliographies on reintroductions. We found 3,826 potentially relevant publications totalling at least 29,290 pages of text. Taxonomic bias is apparent in the distribution of general and conservation scientific literature and in reintroduction programmes. We examined whether the literature on reintroductions is biased in a similar way. Comparing the distribution of reintroduction publications to numbers of species, reintroduction programmes and the general conservation literature, there is a marked taxonomic bias favouring vertebrates, especially birds and mammals. The bias in relation to reintroduction programmes is surprising and indicates that managers working with invertebrates and amphibians are less willing and/or less able to publish their results than those working with mammals and birds. The reasons for this are unclear. The growth of the cumulative body of literature can be depicted by a sigmoid curve. Almost 40% of the items were scientific journal articles distributed across 335 journals. The large, ever-growing and dispersed evidence base results in an increased need for reviews, which must be systematic to minimize bias.

Optimal search filters for renal information in EMBASE

BACKGROUND

EMBASE is a popular database used to retrieve biomedical information. Our objective was to develop and test search filters to help clinicians and researchers efficiently retrieve articles with renal information in EMBASE.

STUDY DESIGN

We used a diagnostic test assessment framework because filters operate similarly to screening tests.

SETTINGS & PARTICIPANTS

We divided a sample of 5,302 articles from 39 journals into development and validation sets of articles.

INDEX TEST

Information retrieval properties were assessed by treating each search filter as a "diagnostic test" or screening procedure for the detection of relevant articles. We tested the performance of 1,936,799 search filters made of unique renal terms and their combinations. REFERENCE STANDARD & OUTCOME: The reference standard was manual review of each article. We calculated the sensitivity and specificity of each filter to identify articles with renal information.

RESULTS

The best renal filters consisted of multiple search terms, such as "renal replacement therapy," "renal," "kidney disease," and "proteinuria," and the truncated terms "kidney," "dialy," "neph," "glomerul," and "hemodial." These filters achieved peak sensitivities of 98.7% (95% CI, 97.9-99.6) and specificities of 98.5% (95% CI, 98.0-99.0). The retrieval performance of these filters remained excellent in the validation set of independent articles.

LIMITATIONS

The retrieval performance of any search will vary depending on the quality of all search concepts used, not just renal terms.

CONCLUSIONS

We empirically developed and validated high-performance renal search filters for EMBASE. These filters can be programmed into the search engine or used on their own to improve the efficiency of searching.

Filtering Medline for a clinical discipline: diagnostic test assessment framework

OBJECTIVE

To develop and test a Medline filter that allows clinicians to search for articles within a clinical discipline, rather than searching the entire Medline database.

DESIGN

Diagnostic test assessment framework with development and validation phases.

SETTING

Sample of 4657 articles published in 2006 from 40 journals. Reviews Each article was manually reviewed, and 19.8% contained information relevant to the discipline of nephrology. The performance of 1 155 087 unique renal filters was compared with the manual review.

MAIN OUTCOME MEASURES

Sensitivity, specificity, precision, and accuracy of each filter.

RESULTS

The best renal filters combined two to 14 terms or phrases and included the terms "kidney" with multiple endings (that is, truncation), "renal replacement therapy", "renal dialysis", "kidney function tests", "renal", "nephr" truncated, "glomerul" truncated, and "proteinuria". These filters achieved peak sensitivities of 97.8% and specificities of 98.5%. Performance of filters remained excellent in the validation phase.

CONCLUSIONS

Medline can be filtered for the discipline of nephrology in a reliable manner. Storing these high performance renal filters in PubMed could help clinicians with their everyday searching. Filters can also be developed for other clinical disciplines by using similar methods.

Bibliometric analysis of the Spanish MR radiological production (2001-2007)

OBJECTIVE

To evaluate the number and characteristics of papers on MR imaging written by radiologists in Spain and published in 2001-2007 Medline-indexed journals, including a comparison with the MR research output from German radiological departments.

MATERIALS AND METHODS

Specific search profiles were devised to retrieve items from the Medline database. Relationship with the topic and major thematic areas of the articles, publication year, journal, language of publication, and mean impact factors were analyzed. Spanish and the German institutions with the highest MR productivity were identified. Also, the number of articles from Spain and Germany published in their respective official journals (Radiología and ROFO) was recorded.

RESULTS

There were 332 Spanish articles published in 101 different journals. The higher number of papers was published in Spanish radiology and non-radiology journals (n=105, 32%, mean IF: 0.191). The journal with the higher number of articles was Radiología (n=51, 15%). "Neuroradiology" was the most frequent topic (n=139, 42%). The Spanish productivity on MR imaging was yearly stable (p=0.67), with the higher percentage of papers (17%) published in 2006. The topic with the higher IF was "neuroradiology" (IF: 2.317). There were no yearly variations for the three major thematic areas ("Neuro imaging": p=0.64; "Body imaging": p=0.91; and "Non-clinical miscellanea": p=0.46). The highest number of MR publications was found in two Spanish institutions (Dr. Peset University Hospital and Vall d'Hebron Hospital) (both, n=28, 8%). In comparison, there were 1681 articles on MR imaging from Germany, having the two most productive institutions more than 150 papers in this period (University Hospital Essen and Eberhard-Karls University of Tübingen). The adjusted number of articles per million populations was 20.4 for Germany and 8.1 for Spain. A total of 395 (23%) articles originating from Germany were published in the official German radiological journal ROFO. The percentage of articles originating from Spain and Germany published in their respective official journals was statistically different.

DISCUSSION

Spanish radiologists published approximately two-third of their MR articles in several non-Spanish journals, while the official Spanish radiological journal Radiología leads the ranking of Spanish journals. Spanish radiologists are mainly active in the "neuroradiology" topic. The Spanish healthcare sector is the most active. German institutions published more in their official journal ROFO.

Primer on systematic review and meta-analysis

A systematic review uses an explicitly defined process to comprehensively identify all studies pertaining to a specific focused question, appraise the methods of the studies, summarize the results, identify reasons for different findings across studies, and cite limitations of current knowledge. Meta-analyses usually combine aggregate-level data reported in each primary study, which may provide a more precise estimate of the "true effect" than any individual study. However, the conclusions may be limited by between-trial heterogeneity, publication bias, or deficits in the conduct or reporting of individual primary studies.