Methodology and analytic techniques used in clinical research: associations with journal impact factor

Characteristics and Research Techniques Associated with the Journal Impact Factor and Other Key Metrics in Pharmacology Journals

In the present age, there is intense pressure on researchers to publish their research in ‘high-impact factor’ journals. It would be interesting to understand the trend of research publications in the field of pharmacology by exploring the characteristics of research articles, including research techniques, in relation to the journal’s key bibliometrics, particularly journal impact factor (JIF), the seemingly most mentioned metric. This study aimed to determine the characteristics and research techniques in relation to research articles in pharmacology journals with higher or lower JIF values. A cross-sectional study was conducted on primary research journals under the ‘Pharmacology and Pharmacy’ category. Analysis of 768 original research articles across 32 journals (with an average JIF of 2.565 ± 0.887) demonstrated that research studies involving molecular techniques, in vivo experiments on animals, and bioinformatics and computational modeling were significantly associated with a higher JIF value of the journal in which such contributions were published. Our analysis suggests that research studies involving such techniques/approaches are more likely to be published in higher-ranked pharmacology journals.

Number and quality of randomized controlled trials in obstetrics published in the top general medical and obstetrics and gynecology journals

BACKGROUND

There has been an increasing number of randomized controlled trials published in obstetrics and maternal-fetal medicine to reduce biases of treatment effect and to provide insights on the cause-effect of the relationship between treatment and outcomes.

OBJECTIVE

This study aimed to identify obstetrical randomized controlled trials published in top weekly general medical journals and monthly obstetrics and gynecology journals, to assess their quality in reporting and identify factors associated with publication in different journals.

STUDY DESIGN

The 4 weekly medical journals with the highest 2019 impact factor (New England Journal of Medicine, The Lancet, The Journal of the American Medical Association, and British Medical Journal), the top 4 monthly obstetrics and gynecology journals with obstetrics-related research (American Journal of Obstetrics & Gynecology, Ultrasound in Obstetrics & Gynecology, Obstetrics & Gynecology, and the British Journal of Obstetrics and Gynaecology), and the American Journal of Obstetrics & Gynecology Maternal-Fetal Medicine were searched for obstetrical randomized controlled trials in the years 2018 to 2020. The primary outcome was the number of obstetrical randomized controlled trials published in the obstetrics and gynecology journals vs the weekly medical journals and the percentage of trials published, overall and per journal. The secondary outcomes included the proportion of positive vs negative trials overall and per journal and the assessment of the study characteristics of published trials, including quality assessment criteria.

RESULTS

Of the 4024 original research articles published in the 9 journals during the 3-year study period, 1221 (30.3%) were randomized controlled trials, with 137 (11.2%) randomized controlled trials being in obstetrics (46 in 2018, 47 in 2019, and 44 studies in 2020). Furthermore, 33 (24.1%) were published in weekly medical journals, and 104 (75.9%) were published in obstetrics and gynecology journals. The percentage of obstetrical randomized controlled trials published ranged from 1.5% to 9.6% per journal. Overall, 34.3% of obstetrical trials were statistically significant or "positive" for the primary outcome. Notably, 24.8% of the trials were retrospectively registered after the enrollment of the first study patient. Trials published in the 4 weekly medical journals enrolled significantly more patients (1801 vs 180; P<.001), received more often funding from the federal government (78.8% vs 35.6%; P<.001), and were more likely to be multicenter (90.9% vs 42.3%; P<.001), non-United States based (69.7% vs 49.0%; P=.03), and double blinded (45.5% vs 18.3%; P=.003) than trials published in the obstetrics and gynecology journals. There was no difference in study type (noninferiority vs superiority) and trial quality characteristics, including pretrial registration, ethics approval statement, informed consent statement, and adherence to the Consolidated Standards of Reporting Trials guidelines statement between studies published in weekly medical journals and studies published in obstetrics and gynecology journals.

CONCLUSION

Approximately 45 trials in obstetrics are being published every year in the highest impact journals, with one-fourth being in the weekly medical journals and the remainder in the obstetrics and gynecology journals. Only about a third of published obstetrical trials are positive. Trials published in weekly medical journals are larger, more likely to be funded by the government, multicenter, international, and double blinded. Quality metrics are similar between weekly medical journals and obstetrics and gynecology journals.

Reporting practices for genomic epidemiology of tuberculosis: a systematic review of the literature using STROME-ID guidelines as a benchmark

BACKGROUND

Pathogen genomics have become increasingly important in infectious disease epidemiology and public health. The Strengthening the Reporting of Molecular Epidemiology for Infectious Diseases (STROME-ID) guidelines were developed to outline a minimum set of criteria that should be reported in genomic epidemiology studies to facilitate assessment of study quality. We evaluate such reporting practices, using tuberculosis as an example.

METHODS

For this systematic review, we initially searched MEDLINE, Embase Classic, and Embase on May 3, 2017, using the search terms "tuberculosis" and "genom* sequencing". We updated this initial search on April 23, 2019, and also included a search of bioRxiv at this time. We included studies in English, French, or Spanish that recruited patients with microbiologically confirmed tuberculosis and used whole genome sequencing for typing of strains. Non-human studies, conference abstracts, and literature reviews were excluded. For each included study, the number and proportion of fulfilled STROME-ID criteria were recorded by two reviewers. A comparison of the mean proportion of fulfilled STROME-ID criteria before and after publication of the STROME-ID guidelines (in 2014) was done using a two-tailed t test. Quasi-Poisson regression and tobit regression were used to examine associations between study characteristics and the number and proportion of fulfilled STROME-ID criteria. This study was registered with PROSPERO, CRD42017064395.

FINDINGS

976 titles and abstracts were identified by our primary search, with an additional 16 studies identified in bioRxiv. 114 full texts (published between 2009 and 2019) were eligible for inclusion. The mean proportion of STROME-ID criteria fulfilled was 50% (SD 12; range 16-75). The proportion of criteria fulfilled was similar before and after STROME-ID publication (51% [SD 11] vs 46% [14], p=0·26). The number of criteria reported (among those applicable to all studies) was not associated with impact factor, h-index, country of affiliation of senior author, or sample size of isolates. Similarly, the proportion of criteria fulfilled was not associated with these characteristics, with the exception of a sample size of isolates of 277 or more (the highest quartile). In terms of reproducibility, 100 (88%) studies reported which bioinformatic tools were used, but only 33 (33%) reported corresponding version numbers. Sequencing data were available for 86 (75%) studies.

INTERPRETATION

The reporting of STROME-ID criteria in genomic epidemiology studies of tuberculosis between 2009 and 2019 was low, with implications for assessment of study quality. The considerable proportion of studies without bioinformatics version numbers or sequencing data available highlights a key concern for reproducibility.

Design, Conduct, and Analysis of Surgical Randomized Controlled Trials: A Cross-sectional Survey

BACKGROUND

Randomized controlled trial (RCT) testing surgical intervention faced challenges due to complexities of surgery and made it more difficult for surgeons and methodologists than pharmaceutical providers to build a well-design, conduct RCT.

OBJECTIVE

We conducted a cross-sectional survey to address the methodological challenges of RCTs on surgical intervention and offer potential solutions.

METHODS

We searched PubMed in order to summarize 2-arm parallel randomized trials for surgical interventions published in 2013. The information regarding general characteristics, general methodological and special surgical characteristics related to surgical trials comparing alternative procedures was gathered.

RESULTS

Some 200 surgical trials were identified. The extent to which these trials in design, conduct and analysis differed substantially across items. The general information about sample size calculation (77.0%), lost to follow-up (71.5%), trial registration (55.5%), protocols of trials (56.0%), implementation of randomization (59.5%), concealment of randomization (56.0%); reporting of primary outcome as P value (67.0%). Surgery special information revealed that only 21.0% of trials considered surgeons' preference, approximately 12% to 50% of them controlled the quality of surgical interventions and none evaluated the effect of the learning curve.

CONCLUSION

There is much room for improvement concerning the reported designs, conduct, and analysis of surgical RCTs. Considering the difficulty of surgical RCTs, some other approaches, such as surgeons' eligibility, performance of pilot studies, or implementation of pragmatic RCTs/expertise-based trials, should be feasibly implemented to overcome the presented challenges.

Do pharmacy intervention reports adequately describe their interventions? A template for intervention description and replication analysis of reports included in a systematic review

INTRODUCTION

Scientific progress and translation of evidence into practice is impeded by poorly described interventions. The Template for Intervention Description and Replication (TIDieR) was developed to specify the minimal intervention elements that should be reported.

OBJECTIVES

(1) To assess the extent to which outpatient pharmacy interventions were adequately reported. (2) To examine the dimension(s) across which reporting quality varies. (3) To examine trial characteristics that predict better reporting.

METHODS

The sample comprised 86 randomised controlled trials identified in a Cochrane review of the effectiveness of pharmacist interventions on patient health outcomes. Duplicate, independent application of a modified 15-item TIDieR checklist was undertaken to assess the intervention reporting. The reporting/non-reporting of TIDieR items was analysed with principal component analysis to evaluate the dimensionality of reporting quality and regression analyses to assess predictors of reporting quality RESULTS: In total, 422 (40%) TIDieR items were fully reported, 395 (38%) were partially reported and 231 (22%) were not reported. A further 242 items were deemed not applicable to the specific trials. Reporting quality loaded on one component which accounted for 26% of the variance in TIDieR scores. More recent trials reported a slightly greater number of TIDieR items (0.07 (95% CI 0.02 to 0.13) additional TIDieR items per year of publication). Trials reported an 0.09 (95% CI 0.04 to 0.14) additional TIDieR items per unit increase in impact factor (IF) of the journal in which the main report was published.

CONCLUSIONS

Most trials lacked adequate intervention reporting. This diminished the applied and scientific value of their research. The standard of intervention reporting is, however, gradually increasing and appears somewhat better in journals with higher IFs. The use of the TIDieR checklist to improve reporting could enhance the utility and replicability of trials, and reduce research waste.

Use of the STROBE Checklist to Evaluate the Reporting Quality of Observational Research in Obstetrics

OBJECTIVE

To evaluate observational research manuscripts submitted to Obstetrics & Gynecology to determine the level of adherence to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist and highlight specific areas that could be improved.

METHODS

A scoring system based on the STROBE checklist was developed and validated for consistency by volunteer medical students or doctors. Using this scoring system, we performed a cross-sectional analysis on 198 observational research manuscripts submitted to Obstetrics & Gynecology from 2008 to 2016. Each manuscript was given a score based on the STROBE checklist. Comparisons were made among acceptance status, country of origin, and study type. Descriptive statistics (means, medians, and frequencies) were calculated for each manuscript category. The t test or Wilcoxon rank-sum test was used to compare differences between two groups and analysis of variance or the Kruskal-Wallis test was used to compare differences among three or more groups.

RESULTS

There was a statistically significant difference between the mean score for accepted (23.2±2.7) compared with rejected (19.7±4.1) manuscripts (P<.001). This difference was not seen when comparing country of origin and study type. Poor reporting was seen among all manuscript categories for objectives, study size, missing data, study participants, and translation of risk. Additionally, rejected manuscripts had poor reporting for eligibility criteria, variables, bias and confounding, statistical methods, unadjusted and adjusted estimates, and category boundaries.

CONCLUSION

Overall, accepted manuscripts show better adherence to the STROBE checklist, but there are several critical items that are poorly reported in all manuscripts.

Current Practices in Data Analysis Procedures in Psychology: What Has Changed?

This paper analyzes current practices in psychology in the use of research methods and data analysis procedures (DAP) and aims to determine whether researchers are now using more sophisticated and advanced DAP than were employed previously. We reviewed empirical research published recently in prominent journals from the USA and Europe corresponding to the main psychological categories of Journal Citation Reports and examined research methods, number of studies, number and type of DAP, and statistical package. The 288 papers reviewed used 663 different DAP. Experimental and correlational studies were the most prevalent, depending on the specific field of psychology. Two-thirds of the papers reported a single study, although those in journals with an experimental focus typically described more. The papers mainly used parametric tests for comparison and statistical techniques for analyzing relationships among variables. Regarding the former, the most frequently used procedure was ANOVA, with mixed factorial ANOVA being the most prevalent. A decline in the use of non-parametric analysis was observed in relation to previous research. Relationships among variables were most commonly examined using regression models, with hierarchical regression and mediation analysis being the most prevalent procedures. There was also a decline in the use of stepwise regression and an increase in the use of structural equation modeling, confirmatory factor analysis, and hierarchical linear modeling. Overall, the results show that recent empirical studies published in journals belonging to the main areas of psychology are employing more varied and advanced statistical techniques of greater computational complexity.

Prevalence of randomised controlled trials in Japanese cardiovascular journals: a descriptive study

Background

Little is known about the recent proportion and trends of randomised controlled trials (RCTs) published in Japanese cardiovascular journals.

Methods and results

This study retrospectively reviewed all original research articles published between 2006 and 2015 in six English-language journals issued by societies related to cardiovascular medicine in Japan. All these journals had been indexed in PubMed for more than 5 years until 2015. We examined the 2-year trends in the prevalence of RCTs, as well as the types of study subjects and interventions investigated. In addition, we applied a multivariable logistic regression analysis to assess the factors related to the publication of RCTs. A total of 7117 articles were eligible for our analyses. The proportion of RCTs among Japanese journals was 3.5% (252/7117). No significant change in the prevalence of RCTs among all included journals was noted (3.8% vs 3.1%, p=0.751). RCTs related to ischaemic heart disease were the most frequently published (85/252, 33.7%). In the multivariable analysis, a male first author (adjusted OR (AOR) 1.97; 95% CI 1.30 to 2.99) and reports from Asia (AOR 1.62; 95% CI 1.19 to 2.21) were significantly associated with the publication of RCTs.

Conclusions

In Japanese cardiovascular journals, RCTs accounted for 3.5% of all original research articles published, and there was no change in the prevalence of RCTs during the study period.

Advanced Research and Data Methods in Women's Health: Big Data Analytics, Adaptive Studies, and the Road Ahead

Technical advances in science have had broad implications in reproductive and women's health care. Recent innovations in population-level data collection and storage have made available an unprecedented amount of data for analysis while computational technology has evolved to permit processing of data previously thought too dense to study. "Big data" is a term used to describe data that are a combination of dramatically greater volume, complexity, and scale. The number of variables in typical big data research can readily be in the thousands, challenging the limits of traditional research methodologies. Regardless of what it is called, advanced data methods, predictive analytics, or big data, this unprecedented revolution in scientific exploration has the potential to dramatically assist research in obstetrics and gynecology broadly across subject matter. Before implementation of big data research methodologies, however, potential researchers and reviewers should be aware of strengths, strategies, study design methods, and potential pitfalls. Examination of big data research examples contained in this article provides insight into the potential and the limitations of this data science revolution and practical pathways for its useful implementation.

Deficiencies of effectiveness of intervention studies in veterinary medicine: a cross-sectional survey of ten leading veterinary and medical journals

The validity of studies that assess the effectiveness of an intervention (EoI) depends on variables such as the type of study design, the quality of their methodology, and the participants enrolled. Five leading veterinary journals and 5 leading human medical journals were hand-searched for EoI studies for the year 2013. We assessed (1) the prevalence of randomized controlled trials (RCTs) among EoI studies, (2) the type of participants enrolled, and (3) the methodological quality of the selected studies. Of 1707 eligible articles, 590 were EoI articles and 435 RCTs. Random allocation to the intervention was performed in 52% (114/219; 95%CI:45.2-58.8%) of veterinary EoI articles, against 87% (321/371; 82.5-89.7%) of human EoI articles (adjusted OR:9.2; 3.4-24.8). Veterinary RCTs were smaller (median: 26 animals versus 465 humans) and less likely to enroll real patients, compared with human RCTs (OR:331; 45-2441). Only 2% of the veterinary RCTs, versus 77% of the human RCTs, reported power calculations, primary outcomes, random sequence generation, allocation concealment and estimation methods. Currently, internal and external validity of veterinary EoI studies is limited compared to human medical ones. To address these issues, veterinary interventional research needs to improve its methodology, increase the number of published RCTs and enroll real clinical patients.

Randomized trials published in higher vs. lower impact journals differ in design, conduct, and analysis

OBJECTIVE

To compare methodological characteristics of randomized controlled trials (RCTs) published in higher vs. lower impact Core Clinical Journals.

STUDY DESIGN AND SETTING

We searched MEDLINE for RCTs published in 2007 in Core Clinical Journals. We randomly sampled 1,140 study reports in a 1:1 ratio in higher (five general medicine journals with the highest total citations in 2007) and lower impact journals.

RESULTS

Four hundred sixty-nine RCTs proved eligible: 219 in higher and 250 in lower impact journals. RCTs in higher vs. lower impact journals had larger sample sizes (median, 285 vs. 39), were more likely to receive industry funding (53% vs. 28%), declare concealment of allocation (66% vs. 36%), declare blinding of health care providers (53% vs. 41%) and outcome adjudicators (72% vs. 54%), report a patient-important primary outcome (69% vs. 50%), report subgroup analyses (64% vs. 26%), prespecify subgroup hypotheses (42% vs. 20%), and report a test for interaction (54% vs. 27%); P < 0.05 for all differences.

CONCLUSION

RCTs published in higher impact journals were more likely to report methodological safeguards against bias and patient-important outcomes than those published in lower impact journals. However, sufficient limitations remain such that publication in a higher impact journal does not ensure low risk of bias.

Matched case-control studies: a review of reported statistical methodology

BACKGROUND

Case-control studies are a common and efficient means of studying rare diseases or illnesses with long latency periods. Matching of cases and controls is frequently employed to control the effects of known potential confounding variables. The analysis of matched data requires specific statistical methods.

METHODS

The objective of this study was to determine the proportion of published, peer-reviewed matched case-control studies that used statistical methods appropriate for matched data. Using a comprehensive set of search criteria we identified 37 matched case-control studies for detailed analysis.

RESULTS

Among these 37 articles, only 16 studies were analyzed with proper statistical techniques (43%). Studies that were properly analyzed were more likely to have included case patients with cancer and cardiovascular disease compared to those that did not use proper statistics (10/16 or 63%, versus 5/21 or 24%, P = 0.02). They were also more likely to have matched multiple controls for each case (14/16 or 88%, versus 13/21 or 62%, P = 0.08). In addition, studies with properly analyzed data were more likely to have been published in a journal with an impact factor listed in the top 100 according to the Journal Citation Reports index (12/16 or 69%, versus 1/21 or 5%, P ≤ 0.0001).

CONCLUSION

The findings of this study raise concern that the majority of matched case-control studies report results that are derived from improper statistical analyses. This may lead to errors in estimating the relationship between a disease and exposure, as well as the incorrect adaptation of emerging medical literature.

Level of evidence in four selected rehabilitation journals

OBJECTIVE

To investigate the methodologic quality and level of evidence of publications in major peer-reviewed general rehabilitation journals (Archives of Physical Medicine and Rehabilitation [APMR], American Journal of Physical Medicine and Rehabilitation [AJPMR], Clinical Rehabilitation [CR], and Physical Therapy [PT]).

DESIGN

Descriptive, comparative.

MAIN OUTCOME MEASURES

All the articles published in AJPMR, APMR, CR, and PT between January 2005 and December 2009 were investigated. Type of study and level of evidence were recorded for all articles. Selection and assessment of articles were based on the title and abstract by 2 independent raters.

RESULTS

The most frequently published reports were randomized controlled trials (12.7%), followed by cross-sectional studies (12.1%), case reports/case series (10.3%), validation studies (9.3%), cohort studies (8.9%), clinical trials (7.5%), case-control studies (6.8%), and other study types (32.4%). When the articles were classified according to their level of evidence, level I studies most frequently appeared in CR (29.1%), followed by PT (11.0%), APMR (10.5%), and AJPMR (7.1%). Most of the meta-analyses (10) were in APMR, and there were none in AJPMR.

CONCLUSIONS

Randomized controlled trials and meta-analyses form only a small proportion of articles published in the current rehabilitation literature. The numbers of randomized controlled trials and meta-analysis are comparable with those in other fields.