A tutorial on methodological studies: the what, when, how and why

Reporting bias in breast reconstruction clinical trials: Which and when clinical trials get published

BACKGROUND

Reporting bias refers to the phenomenon in which the reporting of research findings is influenced by the nature of the results. Without the totality of evidence, clinical practice may be misguided. The objective of this work was to examine the extent of reporting bias in clinical trials of breast reconstruction surgery.

METHODS

We searched and extracted data from all completed breast reconstruction clinical trials published in ClinicalTrials.gov from database inception to August 2020. Investigators sought to identify published full manuscripts of the registered trials. The primary outcome was classified as positive or nonpositive and trials were classified as industry or nonindustry funded. Time to publication in a peer-reviewed journal was computed and compared using time-to-event analysis. Trial characteristics associated with publication were evaluated using logistic regression.

RESULTS

A total of 156 clinical trials were identified, of which, 53 trials were published. The median time to publication was 22 months (IQR, 13-35 months). Industry-funded studies were associated with a longer time to publication (HR = 2.4, p = 0.023) and publication in lower-impact journals (OR = 3.7, p = 0.048). Randomized clinical trials were associated with faster times to publication than nonrandomized studies (aHR = 3.2, p = 0.030). Statistical significance and the effect size were not associated with time to publication.

CONCLUSIONS

We found no evidence that industry-funded trials were more likely to report a positive primary outcome. However, industry-funded trials were associated with a longer time to publication and publication in lower-impact journals.

Patient and public involvement and engagement with underserved communities in dementia research: Reporting on a partnership to co-design a website for postdiagnostic dementia support

INTRODUCTION

Despite the advancements in Patient and Public Involvement and Engagement (PPIE), the voices of traditionally underserved groups are still poorly reflected in dementia research. This study aimed to report on a PPIE partnership between academics and members of the public from underserved communities to co-design Forward with Dementia-Social Care, a resource and information website supporting people receiving a dementia diagnosis.

METHODS

The PPIE partnership was set up in four stages: 1-identifying communities that have been under-represented from PPIE in dementia research; 2-recruiting PPIE partners from these communities; 3-supporting PPIE partners to become confident to undertake their research roles and 4-undertaking research co-design activities in an equitable fashion.

RESULTS

To address under-representation from PPIE in dementia research we recruited seven PPIE partners from Black, Asian and other minority ethnic groups; lesbian, gay, bisexual, transgender, queer+ communities; remote/rural area; religious minorities and partners living with rare forms of dementia. The partners met regularly throughout the project to oversee new sections for the study website, refine existing content and promote the website within their communities.

CONCLUSION

Strategies can be used to successfully recruit and involve PPIE partners from underserved communities in co-design activities. These include networking with community leaders, developing terms of reference, setting out 'rules of engagement', and investing adequate resources and time for accessible and equitable involvement. These efforts facilitate the co-design of research outputs that reflect the diversity and complexity of UK contemporary society.

PATIENT OR PUBLIC CONTRIBUTION

This study received support from seven members of the public with lived experience of dementia from communities that have been traditionally underserved in dementia research. These seven members of the public undertook the role of partners in the study. They all equally contributed to the study design, recruitment of participants, development and revision of topic guides for the interviews and development of the website. Three of these partners were also co-authors of this paper. On top of the activities shared with the other partners, they contributed to write independently of the academic team the section in this paper titled 'Partners' experiences, benefits and challenges of the partnership'. Further, they provided input in other sections of the paper on a par with the other (academic) co-authors.

Natural language processing techniques for studying language in pathological ageing: A scoping review

BACKGROUND

In the past few years there has been a growing interest in the employment of verbal productions as digital biomarkers, namely objective, quantifiable behavioural data that can be collected and measured by means of digital devices, allowing for a low-cost pathology detection, classification and monitoring. Numerous research papers have been published on the automatic detection of subtle verbal alteration, starting from written texts, raw speech recordings and transcripts, and such linguistic analysis has been singled out as a cost-effective method for diagnosing dementia and other medical conditions common among elderly patients (e.g., cognitive dysfunctions associated with metabolic disorders, dysarthria).

AIMS

To provide a critical appraisal and synthesis of evidence concerning the application of natural language processing (NLP) techniques for clinical purposes in the geriatric population. In particular, we discuss the state of the art on studying language in healthy and pathological ageing, focusing on the latest research efforts to build non-intrusive language-based tools for the early identification of cognitive frailty due to dementia. We also discuss some challenges and open problems raised by this approach.

METHODS & PROCEDURES

We performed a scoping review to examine emerging evidence about this novel domain. Potentially relevant studies published up to November 2021 were identified from the databases of MEDLINE, Cochrane and Web of Science. We also browsed the proceedings of leading international conferences (e.g., ACL, COLING, Interspeech, LREC) from 2017 to 2021, and checked the reference lists of relevant studies and reviews.

MAIN CONTRIBUTION

The paper provides an introductory, but complete, overview of the application of NLP techniques for studying language disruption due to dementia. We also suggest that this technique can be fruitfully applied to other medical conditions (e.g., cognitive dysfunctions associated with dysarthria, cerebrovascular disease and mood disorders).

CONCLUSIONS & IMPLICATIONS

Despite several critical points need to be addressed by the scientific community, a growing body of empirical evidence shows that NLP techniques can represent a promising tool for studying language changes in pathological aging, with a high potential to lead a significant shift in clinical practice.

WHAT THIS PAPER ADDS

What is already known on this subject Speech and languages abilities change due to non-pathological neurocognitive ageing and neurodegenerative processes. These subtle verbal modifications can be measured through NLP techniques and used as biomarkers for screening/diagnostic purposes in the geriatric population (i.e., digital linguistic biomarkers-DLBs). What this paper adds to existing knowledge The review shows that DLBs can represent a promising clinical tool, with a high potential to spark a major shift to dementia assessment in the elderly. Some challenges and open problems are also discussed. What are the potential or actual clinical implications of this work? This methodological review represents a starting point for clinicians approaching the DLB research field for studying language in healthy and pathological ageing. It summarizes the state of the art and future research directions of this novel approach.

The endorsement of general and artificial intelligence reporting guidelines in radiological journals: a meta-research study

BACKGROUND

Complete reporting is essential for clinical research. However, the endorsement of reporting guidelines in radiological journals is still unclear. Further, as a field extensively utilizing artificial intelligence (AI), the adoption of both general and AI reporting guidelines would be necessary for enhancing quality and transparency of radiological research. This study aims to investigate the endorsement of general reporting guidelines and those for AI applications in medical imaging in radiological journals, and explore associated journal characteristic variables.

METHODS

This meta-research study screened journals from the Radiology, Nuclear Medicine & Medical Imaging category, Science Citation Index Expanded of the 2022 Journal Citation Reports, and excluded journals not publishing original research, in non-English languages, and instructions for authors unavailable. The endorsement of fifteen general reporting guidelines and ten AI reporting guidelines was rated using a five-level tool: "active strong", "active weak", "passive moderate", "passive weak", and "none". The association between endorsement and journal characteristic variables was evaluated by logistic regression analysis.

RESULTS

We included 117 journals. The top-five endorsed reporting guidelines were CONSORT (Consolidated Standards of Reporting Trials, 58.1%, 68/117), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, 54.7%, 64/117), STROBE (STrengthening the Reporting of Observational Studies in Epidemiology, 51.3%, 60/117), STARD (Standards for Reporting of Diagnostic Accuracy, 50.4%, 59/117), and ARRIVE (Animal Research Reporting of In Vivo Experiments, 35.9%, 42/117). The most implemented AI reporting guideline was CLAIM (Checklist for Artificial Intelligence in Medical Imaging, 1.7%, 2/117), while other nine AI reporting guidelines were not mentioned. The Journal Impact Factor quartile and publisher were associated with endorsement of reporting guidelines in radiological journals.

CONCLUSIONS

The general reporting guideline endorsement was suboptimal in radiological journals. The implementation of reporting guidelines for AI applications in medical imaging was extremely low. Their adoption should be strengthened to facilitate quality and transparency of radiological study reporting.

Time and change: a typology for presenting research findings in qualitative longitudinal research

BACKGROUND

Qualitative longitudinal research (QLR) is an emerging methodology used in health research. The method literature states that the change in a phenomenon through time should be the focus of any QLR study, but in empirical studies, the analysis of changes through time is often poorly described, and the emphasis on time/change in the findings varies greatly. This inconsistency might depend on limitations in the existing method literature in terms of describing how QLR studies can present findings. The aim of this study was to develop and describe a typology of alternative approaches for integrating time and/or change in QLR findings.

METHODS

In this method study, we used an adapted scoping review design. Articles were identified using EBSCOhost. In total, methods and results sections from 299 QLR articles in the field of health research were analyzed with inspiration from content analysis.

RESULTS

We constructed a typology of three types and seven subtypes. The types were based on the underlying structural principles of how time/change was presented: Type A) Findings have a low utilization of longitudinal data, Type B) Findings are structured according to chronological time, and Type C) Findings focus on changes through time. These types differed in 1) the way the main focus was on time, change or neither; 2) the level of interpretation in the findings; and 3) how theoretical understandings of time/change were articulated in the articles. Each type encompassed two or three subtypes that represented distinct approaches to the aim and results presentation of QLR findings.

CONCLUSIONS

This method study is the first to describe a coherent and comprehensive typology of alternative approaches for integrating time/change into QLR findings in health research. By providing examples of various subtypes that can be used for results presentations, it can help researchers make informed decisions suitable to their research intent.

Quality of systematic reviews in African emergency medicine: a cross-sectional methodological study

Introduction

Reliable systematic reviews are essential to inform clinical practice guidelines, policies and further research priorities in Africa. For systematic review findings to be trustworthy, they need to be conducted with methodological rigour and reported transparently. We assessed the methodological quality of systematic reviews published in African emergency medicine journals, comparing them to those published in international emergency medicine journals. Additionally, we describe the types of review literature published in the African journals.

Methods

We performed a cross-sectional methodological study of systematic reviews published in selected African and international emergency medicine journals from 2012 to 2021. Studies were eligible if they were i) a systematic review on an emergency medicine topic, ii) published in one of the top five emergency medicine journals in the African region or internationally and iii) published between January 2012 and December 2021 in English or French. We searched PubMed, Web of Science and Scopus databases and hand-searched selected journals. Two authors screened titles, abstracts and full texts independently and in duplicate. Data extraction was performed by one reviewer, using a standardised form, after completing a calibration exercise. We described the characteristics of systematic reviews and assessed methodological quality using AMSTAR II.

Results

We identified 34 (37%) African and 511 (54%) international systematic reviews from 92 and 948 review articles respectively across 10 journals. We included all 34 African and a random sample of 100 international systematic reviews. Methodological quality was low or critically low for all the African systematic reviews (n=34, 100%) and all but three international systematic reviews (n=97, 97%). The median number of critical domain weaknesses was 4 (IQR 4;5) and 2 (IQR 2;4) for African and international systematic reviews respectively. The most common weaknesses across both African and international systematic reviews were i) not establishing a priori review protocols, ii) unclear selection of study designs iii) not providing a list of excluded studies and iv) unclear reporting on funding sources for included studies.

Conclusion

Emergency medicine systematic reviews published in African and international journals are lacking in methodological quality. Reporting an a priori protocol, developing a comprehensive search strategy, appropriate evidence synthesis and adequate assessment of risk of bias, heterogeneity and evidence certainty may improve the quality of systematic reviews.

Validation of the Duke Activity Status Index questionnaire by telephone In individuals after stroke

Background

Due to social restrictions caused by the pandemic, there was a need to validate outcome measures that could be administered by telephone call. Administration by telephone allows to remotely follow up stroke survivors since most of them have mobility restrictions. This study aims to investigate the validity of the Duke Activity Status Index (DASI) questionnaire administration to chronic stroke survivors through telephone call.

Methods

This is a cross-sectional study, developed according to COSMIN and GRRAS recommendations. It was recruited chronic stroke survivors, who answered the DASI questionnaire in two different time-points, in person and after a period of 5-7 days through a telephone call.

Results

Out of 260 subjects, 50 individuals (52% women) with a mean age of 56 ± 17 years were included. No statistically significant differences were observed (MD = -0.88; SD:4.14; 95% CI, -2.06 to 0.28; p = 0.13) on the total score of DASI administered in person and by telephone call. There was a very high agreement between the administration modes (ICC - 0.99; 95% CI, 0.94-0.98; p < 0.05). The Kappa coefficient ranged from 0.390 to 1.000, with the first item showing the best agreement (k = 1.000) and the fourth showing the worst agreement (k = 0.390).

Conclusions

The DASI questionnaire is valid to assess functional capacity and can be administered through telephone in chronic stroke survivors. Thus, clinicians and researchers may decide to avoid patient transportations administering DASI through telephone call, as a reliable measure for stroke survivors.

Prototyping Process and Usability Testing of a Serious Game for Brazilian Children With Type 1 Diabetes

This study aims to describe the prototype development and testing of a serious game designed for Brazilian children with diabetes. Following an approach of user-centered design, the researchers assessed game's preferences and diabetes learning needs to develop a Paper Prototype. The gameplay strategies included diabetes pathophysiology, self-care tasks, glycemic management, and food group learning. Diabetes and technology experts (n = 12) tested the prototype during audio-recorded sessions. Next, they answered a survey to evaluate the content, organization, presentation, and educational game aspects. The prototype showed a high content validity ratio (0.80), with three items not achieving the critical values (0.66). Experts recommended improving the game content and food illustrations. This evaluation contributed to the medium-fidelity prototype version, which after testing with diabetes experts (n = 12) achieved high content validity values (0.88). One item did not meet the critical values. Experts suggested increasing the options of outdoor activities and meals. Researchers also observed and video-recorded children with diabetes (n = 5) playing the game with satisfactory interaction. They considered the game enjoyable. The interdisciplinary team plays an important role guiding the designers in the use of theories and real needs of children. Prototypes are a low-cost usability and a successful method for evaluating games.

A Scoping Review of Adopted Information Extraction Methods for RCTs

Background

Randomized controlled trials (RCTs) provide the strongest evidence for therapeutic interventions and their effects on groups of subjects. However, the large amount of unstructured information in these trials makes it challenging and time-consuming to make decisions and identify important concepts and valid evidence. This study aims to explore methods for automating or semi-automating information extraction from reports of RCT studies.

Methods

We conducted a systematic search of PubMed, ACM Digital Library, and Web of Science to identify relevant articles published between January 1, 2010, and 2022. We focused on published Natural Language Processing (NLP), machine learning, and deep learning methods that automate or semi-automate key elements of information extraction in the context of RCTs.

Results

A total of 26 publications were included, which discussed the automatic extraction of key characteristics of RCTs using various PICO frameworks (PIBOSO and PECODR). Among these publications, 14 (53.8%) extracted key characteristics based on PICO, PIBOSO, and PECODR, while 12 (46.1%) discussed information extraction methods in RCT studies. Common approaches mentioned included word/phrase matching, machine learning algorithms such as binary classification using the Naïve Bayes algorithm and powerful BERT network for feature extraction, support vector machine for data classification, conditional random field, non-machine-dependent automation, and machine learning or deep learning approaches.

Conclusion

The lack of publicly available software and limited access to existing software makes it difficult to determine the most powerful information extraction system. However, deep learning models like Transformers and BERT language models have shown better performance in natural language processing.

Evaluating the Methodological Approaches of Cross-Cultural Adaptation of the Bedside Handover Attitudes and Behaviours Questionnaire into Portuguese

Nurse managers need culturally adapted assessment instruments to support the implementation of change to Nursing Bedside Handover (NBH) in healthcare institutions. This study aimed to cross-culturally adapt the Bedside Handover Attitudes and Behaviours (BHAB) questionnaire to the Portuguese context and evaluate the methodological approaches used for this purpose. To guide this study, we followed a guideline for cross-cultural translation and adaptation measurement instruments in healthcare. The results of the content validity testing suggested that the BHAB questionnaire is a valid instrument for use in the Portuguese context. To obtain these results we showed 1) using of a new methodological approach, the dual focus, to resolve the divergences and ambiguities in the translators' committee and the multi-professional committee; 2) the lack of a conceptual definition of the construct of the instrument as a requirement to retain items with I-CVI <0.70 after validity relevance pretesting and 3) the cognitive debriefing and relevance pretesting as methodological approaches which can be used alone or together to reinforce the evaluation of cultural relevance of the items. We concluded there is a need for guidelines to support the decision-making process of healthcare researchers with comprehensive information about the different methodological approaches they can follow.

Nonlinear effects and effect modification at the participant-level in IPD meta-analysis part 2: methodological guidance is available

OBJECTIVES

To review methodological guidance for nonlinear covariate-outcome associations (NL), and linear effect modification and nonlinear effect modification (LEM and NLEM) at the participant level in individual participant data meta-analyses (IPDMAs) and their power requirements.

STUDY DESIGN AND SETTING

We searched Medline, Embase, Web of Science, Scopus, PsycINFO and the Cochrane Library to identify methodology publications on IPDMA of LEM, NL or NLEM (PROSPERO CRD42019126768).

RESULTS

Through screening 6,466 records we identified 54 potential articles of which 23 full texts were relevant. Nine further relevant publications were published before or after the literature search and were added. Of these 32 references, 21 articles considered LEM, 6 articles NL or NLEM and 6 articles described sample size calculations. A book described all four. Sample size may be calculated through simulation or closed form. Assessments of LEM or NLEM at the participant level need to be based on within-trial information alone. Nonlinearity (NL or NLEM) can be modeled using polynomials or splines to avoid categorization.

CONCLUSION

Detailed methodological guidance on IPDMA of effect modification at participant-level is available. However, methodology papers for sample size and nonlinearity are rarer and may not cover all scenarios. On these aspects, further guidance is needed.

Nonlinear effects and effect modification at the participant-level in IPD meta-analysis part 1: analysis methods are often substandard

OBJECTIVES

To review analysis methods used for linear effect modification (LEM), nonlinear covariate-outcome associations (NL) and nonlinear effect modification (NLEM) at the participant-level in individual participant data meta-analysis (IPDMA).

STUDY DESIGN AND SETTING

We searched Medline, Embase, Web of Science, Scopus, PsycINFO and the Cochrane Library to identify IPDMA of randomized controlled trials (PROSPERO CRD42019126768). We investigated if and how IPDMA examined LEM, NL and NLEM, including whether aggregation bias was addressed and if power was considered.

RESULTS

We screened 6,466 records, randomly sampled 207 and identified 100 IPDMA of LEM, NL or NLEM. Power for LEM was calculated a priori in 3 IPDMA. Of 100 IPDMA, 94 analyzed LEM, 4 NLEM and 8 NL. One-stage models were favoured for all three (56%, 100%, 50%, respectively). Two-stage models were used in 15%, 0% and 25% of IPDMA with unclear descriptions in 30%, 0% and 25%, respectively. Only 12% of one-stage LEM and NLEM IPDMA provided sufficient detail to confirm they had addressed aggregation bias.

CONCLUSION

Investigation of effect modification at the participant-level is common in IPDMA projects, but methods are often open to bias or lack detailed descriptions. Nonlinearity of continuous covariates and power of IPDMA are rarely assessed.

Observed intervention effects for mortality in randomised clinical trials: a methodological study protocol

INTRODUCTION

It is essential to choose a realistic anticipated intervention effect when calculating a sample size for a randomised clinical trial. Unfortunately, anticipated intervention effects are often inflated, when compared with the 'true' intervention effects. This is documented for mortality in critical care trials. A similar pattern might exist across different medical specialties. This study aims to estimate the range of observed intervention effects for all-cause mortality in trials included in Cochrane Reviews, within each Cochrane Review Group.

METHODS AND ANALYSIS

We will include randomised clinical trials assessing all-cause mortality as an outcome. Trials will be identified from Cochrane Reviews published in the Cochrane Database of Systematic Reviews. Cochrane Reviews will be clustered according to the registered Cochrane Review Group (eg, Anaesthesia, Emergency and Critical Care) and the statistical analyses will be conducted for each Cochrane Review Group and overall. The median relative risk and IQR for all-cause mortality and the proportion of trials with a relative all-cause mortality risk within seven different ranges will be reported (relative risk below 0.70, 0.70-0.79, 0.80-0.89, 0.90-1.09, 1.10-1.19, 1.20-1.30 and above 1.30). Subgroup analyses will explore the effects of original design, sample size, risk of bias, disease, intervention type, follow-up length, participating centres, funding type, information size and outcome hierarchy.

ETHICS AND DISSEMINATION

Since we will use summary data from trials already approved by relevant ethical committees, this study does not require ethical approval. Regardless of our findings, the results will be published in an international peer-reviewed journal.

Development and Predictive Validation of the Brazilian Adductor Performance Test for Estimating the Chance of Hip Adductor Injuries in Elite Soccer Athletes

PURPOSE

To develop and validate the Brazilian Adductor Performance Test (BAPT) for predicting hip adductor muscle injuries in elite soccer athletes.

METHODS

A total of 108 soccer athletes were assessed, followed up for 3 months, and evaluated for a history of adductor injury 6 months before BAPT evaluation. The Shapiro-Wilk test was used as the normality test. The Mann-Whitney U test was used to compare BAPT scores between injured and uninjured athletes. Binary logistic regression was performed to identify the athletes' chances of injury based on their BAPT scores. A receiver operating characteristic curve was used to determine the cutoff point for the number of repetitions in the BAPT and Spearman bivariate correlation and identify factors potentially related to the test score. Furthermore, the intraclass correlation coefficient was used to determine interexaminer agreement. The level of significance was set at 95%.

RESULTS

The BAPT scores for hip adductor injury history did not differ significantly (P = .08). A significant deficit was identified in the BAPT scores of the injured athletes at the 3-month follow-up (P = .001). The cutoff point identified was 33 repetitions. Low BAPT scores increased the chance of injury by 20% (odds ratio, 1.20%; P = .001). The interexaminer agreement was .96 (P = .001).

CONCLUSION

BAPT can be used to identify athletes most likely to sustain hip adductor muscle injuries, indirectly reducing the rate of this injury in soccer clubs.

Quality of Reporting Using Good Reporting of A Mixed Methods Study Criteria in Chiropractic Mixed Methods Research: A Methodological Review

OBJECTIVE

The purpose of this review was to examine the reporting in chiropractic mixed methods research using Good Reporting of A Mixed Methods Study (GRAMMS) criteria.

METHODS

In this methodological review, we searched MEDLINE, Embase, CINAHL, and the Index to Chiropractic Literature from the inception of each database to December 31, 2020, for chiropractic studies reporting the use of both qualitative and quantitative methods or mixed qualitative methods. Pairs of reviewers independently screened titles, abstracts, and full-text studies, extracted data, and appraised reporting using the GRAMMS criteria and risk of bias with the Mixed Methods Appraisal Tool (MMAT). Generalized estimating equations were used to explore factors associated with reporting using GRAMMS criteria.

RESULTS

Of 1040 citations, 55 studies were eligible for review. Thirty-seven of these 55 articles employed either a multistage or convergent mixed methods design, and, on average, 3 of 6 GRAMMS items were reported among included studies. We found a strong positive correlation in scores between the GRAMMS and MMAT instruments (r = 0.78; 95% CI, 0.66-0.87). In our adjusted analysis, publications in journals indexed in Web of Science (adjusted odds ratio = 2.71; 95% CI, 1.48-4.95) were associated with higher reporting using GRAMMS criteria. Three of the 55 studies fully adhered to all 6 GRAMMS criteria, 4 studies adhered to 5 criteria, 10 studies adhered to 4 criteria, and the remaining 38 adhered to 3 criteria or fewer.

CONCLUSION

Our findings suggest that reporting in chiropractic mixed methods research using GRAMMS criteria was poor, particularly among studies with a higher risk of bias.

Methodological transparency of preoperative clinical practice guidelines for elective surgery. Systematic review

BACKGROUND

Clinical practice guidelines (CPG) are statements that provide recommendations regarding the approach to different diseases and aim to increase quality while decreasing the risk of complications in health care. Numerous guidelines in the field of perioperative care have been published in the previous decade but their methodological quality and transparency are relatively unknown.

OBJECTIVE

To critically evaluate the transparency and methodological quality of published CPG in the preoperative assessment and management of adult patients undergoing elective surgery.

DESIGN

Systematic review and methodological appraisal study.

DATA SOURCES

We searched for eligible CPG published in English or Spanish between January 1, 2010, and June 30, 2022, in Pubmed MEDLINE, TRIP Database, Embase, the Cochrane Library, as well as in representatives' medical societies of Anaesthesiology and developers of CPG.

ELIGIBILITY CRITERIA

CPG dedicated on preoperative fasting, cardiac assessment for non-cardiac surgery, and the use of routine preoperative tests were included. Methodological quality and transparency of CPG were assessed by 3 evaluators using the 6 domains of the AGREE-II tool.

RESULTS

We included 20 CPG of which 14 were classified as recommended guidelines. The domain of "applicability" scored the lowest (44%), while the domains "scope and objective" and "editorial interdependence" received the highest median scores of 93% and 97% respectively. The remaining domains received scores ranging from 44% to 84%. The top mean scored CPG in preoperative fasting was ASA 2017 (93%); among cardiac evaluation, CPG for non-cardiac surgery were CCS 2017 (91%), ESC-ESA 2014 (90%), and AHA-ACC 2014 (89%); in preoperative testing ICSI 2020 (97%).

CONCLUSIONS

In the last ten years, most published CPG in the preoperative assessment or management of adult patients undergoing elective surgery focused on preoperative fasting, cardiac assessment for non-cardiac surgery, and use of routine preoperative tests, present moderate to high methodological quality and can be recommended for their use or adaptation. Applicability and stakeholder involvement domains must be improved in the development of future guidelines.

Use of mixed methods research in intervention studies to increase young people's interest in STEM: A systematic methodological review

Introduction

Mixed methods research intervention studies integrate quantitative evaluation approaches, such as randomized controlled trials and quasi-experimental designs, with qualitative research to evaluate the effectiveness, efficacy, or other results of an intervention or program. These types of studies, which have attracted growing attention in recent years, enhance the scope and rigor of the evaluation. While various frameworks that summarize the justifications for carrying out these types of studies and provide implementation guidance have been published in the last few years in the health sciences, we do not know whether such frameworks have been properly implemented in the social and educational sciences. This review examined the methodological features and reporting practices of mixed methods intervention studies aimed at increasing young people's interest in STEM.

Methods

A systematic search was carried out in APA PsycNET, ERIC, ProQuest, Scopus, and Web of Science, and a hand search in 20 journals. We included peer-reviewed English-language articles that reported intervention studies with a quantitative component measuring outcomes specific to increasing secondary school students' interest in STEM fields, a qualitative component conducted before, during, or after the quantitative component, and evidence of integration of both components. Qualitative content analysis and ideal-type analysis were used to synthesize the findings.

Results

We found 34 studies; the majority published in the last ten years. Several patterns of mixed methods application were described in these studies, illustrating the unique insights that can be gained by employing this methodology. The reporting quality of the included studies was generally adequate, especially regarding the justification for using a mixed methods intervention design and the integration of the quantitative and qualitative components. Nonetheless, a few reporting issues were observed, such as a lack of detail in the presentation of the mixed methods design, an inadequate description of the qualitative sampling and analysis techniques, and the absence of joint displays for representing integration.

Discussion

Authors must pay attention to these issues to ensure that the insights obtained by the use of mixed methods research are effectively communicated.

Types and associated methodologies of overviews of reviews in health care: a methodological study with published examples

OBJECTIVES

To provide a descriptive insight into the different types of research questions/objectives and associated methodologies of overviews of reviews, supplemented by representative examples from the health care literature.

STUDY DESIGN AND SETTING

We searched in methodological articles for information on types and methodologies used in overviews and we explored the typology of reviews to identify similar types in literature of overviews. We categorized the types of overviews based on the research question/objective and the methodological approach used. Indicative examples for each category were selected from a sample of 2,121 overviews that were retrieved between 2000 and 2022 from MEDLINE, Scopus, and Cochrane Database of Systematic Reviews.

RESULTS

Based on type of research question, overviews were classified as overviews of reviews of interventions, associations, prediction, diagnostic accuracy, prevalence/incidence, experiences/views, economic evaluation, and measurement properties. Based on the methodological approach, we identified a variety of methods (systematic, living, rapid, scoping, evidence mapping, framework, and methodological) used in overviews.

CONCLUSION

The proposed classification and examples provide an essential starting point for future theory-building research on typologies and study designs of overviews of reviews. It is important for methodologists to make vigorous effort to create consensus-based methodological and reporting guidelines to cover these diverse types and key methodological challenges.

Methods for developing and reporting living evidence synthesis

BACKGROUND AND OBJECTIVES

Identify currently methodological aspects proposed for planning, conducting, and reporting living evidence (LE) synthesis. Develop a preliminary checklist of key LE synthesis elements.

METHODS

A survey of methodological articles describing or analyzing methods for the design, conduction, or reporting of LE synthesis.

RESULTS

Twelve methodological articles were identified and analyzed. Key elements were related to: i) definition of LE and characteristics of LE synthesis, ii) methods and tools for the living process, iii) new evidence integration (methods and considerations), iv) updates dissemination and publication, v) revisiting living parameters, and vi) protocol considerations for LE synthesis.

CONCLUSION

This survey displays basic methodological concepts that can drive the development of LE synthesis and identifies specific aspects with opportunities for development. The potential impact of the LE approach calls for a change in the current evidence synthesis updating processes to more open, collaborative, transparent, and efficient systems. LE approaches also challenge journal editors to shift toward more efficient processes for synthesis update dissemination, which minimizes the risks of reliability of published information.

Qualitative longitudinal research in health research: a method study

Background

Qualitative longitudinal research (QLR) comprises qualitative studies, with repeated data collection, that focus on the temporality (e.g., time and change) of a phenomenon. The use of QLR is increasing in health research since many topics within health involve change (e.g., progressive illness, rehabilitation). A method study can provide an insightful understanding of the use, trends and variations within this approach. The aim of this study was to map how QLR articles within the existing health research literature are designed to capture aspects of time and/or change.

Methods

This method study used an adapted scoping review design. Articles were eligible if they were written in English, published between 2017 and 2019, and reported results from qualitative data collected at different time points/time waves with the same sample or in the same setting. Articles were identified using EBSCOhost. Two independent reviewers performed the screening, selection and charting.

Results

A total of 299 articles were included. There was great variation among the articles in the use of methodological traditions, type of data, length of data collection, and components of longitudinal data collection. However, the majority of articles represented large studies and were based on individual interview data. Approximately half of the articles self-identified as QLR studies or as following a QLR design, although slightly less than 20% of them included QLR method literature in their method sections.

Conclusions

QLR is often used in large complex studies. Some articles were thoroughly designed to capture time/change throughout the methodology, aim and data collection, while other articles included few elements of QLR. Longitudinal data collection includes several components, such as what entities are followed across time, the tempo of data collection, and to what extent the data collection is preplanned or adapted across time. Therefore, there are several practices and possibilities researchers should consider before starting a QLR project.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01732-4.

Evaluating reporting of patient-reported outcomes in randomized controlled trials regarding inflammatory bowel disease: a methodological study

Patient-reported outcomes (PROs) in randomized controlled trials pertaining to inflammatory bowel disease are important in identifying patients' perspective of treatment. Incompletely reported PROs within trials could misrepresent information for clinicians and may contribute to treatment which lacks accommodation of patient input. Our study evaluates completeness of reporting of PROs and risk of bias (RoB) to identify how well trialists are adhering to known resources for trials. We used MEDLINE, Embase, and the Cochrane Central Register of Controlled Trials to identify eligible trials from 2006 to 2020 with at least 1 PRO measure related to inflammatory bowel disease. The trials were screened in duplicate using Rayyan. We then compared trial completion of reporting to the Consolidated Standards of Reporting Trials (CONSORT)-PRO adaptation, and assessed RoB using the Cochrane Collaboration RoB 2.0 tool. To measure trial and reporting characteristics, we performed bivariate regression analyses. Among a sample of 29 trials, the mean completion percentage for CONSORT-PRO was 46.77%. We found PROs as a secondary outcome had significantly lower CONSORT-PRO reporting (p<0.05). In addition, per cent completeness of reporting was significantly higher with both a 'therapy' intervention, and trials published following the development of CONSORT-PRO (p<0.05). Incomplete PRO reporting is common in trials focused on inflammatory bowel disease. This suboptimal reporting indicates the need for adherence to reporting guidelines. Trialists should use the CONSORT-PRO checklist, as endorsed by Patient-Reported Outcomes Tools: Engaging Users and Stakeholders, to assess their studies in order to enhance reporting adherence.

Artificial intelligence in COVID-19 evidence syntheses was underutilized, but impactful: a methodological study

OBJECTIVES

A rapidly developing scenario like a pandemic requires the prompt production of high-quality systematic reviews, which can be automated using artificial intelligence (AI) techniques. We evaluated the application of AI tools in COVID-19 evidence syntheses.

STUDY DESIGN

After prospective registration of the review protocol, we automated the download of all open-access COVID-19 systematic reviews in the COVID-19 Living Overview of Evidence database, indexed them for AI-related keywords, and located those that used AI tools. We compared their journals' JCR Impact Factor, citations per month, screening workloads, completion times (from pre-registration to preprint or submission to a journal) and AMSTAR-2 methodology assessments (maximum score 13 points) with a set of publication date matched control reviews without AI.

RESULTS

Of the 3,999 COVID-19 reviews, 28 (0.7%, 95% CI 0.47-1.03%) made use of AI. On average, compared to controls (n = 64), AI reviews were published in journals with higher Impact Factors (median 8.9 vs. 3.5, P < 0.001), and screened more abstracts per author (302.2 vs. 140.3, P = 0.009) and per included study (189.0 vs. 365.8, P < 0.001) while inspecting less full texts per author (5.3 vs. 14.0, P = 0.005). No differences were found in citation counts (0.5 vs. 0.6, P = 0.600), inspected full texts per included study (3.8 vs. 3.4, P = 0.481), completion times (74.0 vs. 123.0, P = 0.205) or AMSTAR-2 (7.5 vs. 6.3, P = 0.119).

CONCLUSION

AI was an underutilized tool in COVID-19 systematic reviews. Its usage, compared to reviews without AI, was associated with more efficient screening of literature and higher publication impact. There is scope for the application of AI in automating systematic reviews.

The Doctors' Effect on Patients' Physical Health Outcomes Beyond the Intervention: A Methodological Review

Background

Previous research suggests that when a treatment is delivered, patients’ outcomes may vary systematically by medical practitioner.

Objective

To conduct a methodological review of studies reporting on the effect of doctors on patients’ physical health outcomes and to provide recommendations on how this effect could be measured and reported in a consistent and appropriate way.

Methods

The data source was 79 included studies and randomized controlled trials from a systematic review of doctors’ effects on patients’ physical health. We qualitatively assessed the studies and summarized how the doctors’ effect was measured and reported.

Results

The doctors’ effects on patients’ physical health outcomes were reported as fixed effects, identifying high and low outliers, or random effects, which estimate the variation in patient health outcomes due to the doctor after accounting for all available variables via the intra-class correlation coefficient. Multivariable multilevel regression is commonly used to adjust for patient risk, doctor experience and other demographics, and also to account for the clustering effect of hospitals in estimating both fixed and random effects.

Conclusion

This methodological review identified inconsistencies in how the doctor’s effect on patients’ physical health outcomes is measured and reported. For grading doctors from worst to best performances and estimating random effects, specific recommendations are given along with the specific data points to report.

Point your SmartPhone at the code above. If you have a QR code reader the video abstract will appear. Or use:

https://youtu.be/rvHjVIEPVhI

Mixed methods intervention studies in children and adolescents with emotional and behavioral disorders: A methodological review

BACKGROUND

Mixed methods intervention studies can improve the accuracy of interventional evaluations in the field of emotional and behavioral disorders by helping researchers gain a more nuanced understanding of how a particular intervention works. However, no studies to date have systematically examined the ways in which this type of studies have been carried out and reported.

AIM

To examine the methodological features and reporting practices found in mixed methods intervention studies in children and adolescents with emotional and behavioral disorders.

METHOD

Methodological review based on a systematic search from inception to July 2021 in Embase, Medline, PsycINFO, and SCOPUS, and a hand search in seven journals.

RESULTS

We found 30 studies, most of them published since 2019. These studies reported several patterns of mixed methods use which illustrated the unique insights that researchers can gain by using this approach. We identified several ways that authors could more clearly report the justification for using a mixed methods approach, the description of the design used, and the evidence of integration of the quantitative and qualitative components.

CONCLUSION

We make recommendations for improving the reporting quality of mixed methods intervention studies in the field of emotional and behavioral disorders.

Causal assessment in evidence synthesis: A methodological review of reviews

In fields (such as population health) where randomised trials are often lacking, systematic reviews (SRs) can harness diversity in study design, settings and populations to assess the evidence for a putative causal relationship. SRs may incorporate causal assessment approaches (CAAs), sometimes called 'causal reviews', but there is currently no consensus on how these should be conducted. We conducted a methodological review of self-identifying 'causal reviews' within the field of population health to establish: (1) which CAAs are used; (2) differences in how CAAs are implemented; (3) how methods were modified to incorporate causal assessment in SRs. Three databases were searched and two independent reviewers selected reviews for inclusion. Data were extracted using a standardised form and summarised using tabulation and narratively. Fifty-three reviews incorporated CAAs: 46/53 applied Bradford Hill (BH) viewpoints/criteria, with the remainder taking alternative approaches: Medical Research Council guidance on natural experiments (2/53, 3.8%); realist reviews (2/53, 3.8%); horizontal SRs (1/53, 1.9%); 'sign test' of causal mechanisms (1/53, 1.9%); and a causal cascade model (1/53, 1.9%). Though most SRs incorporated BH, there was variation in application and transparency. There was considerable overlap across the CAAs, with a trade-off between breadth (BH viewpoints considered a greater range of causal characteristics) and depth (many alternative CAAs focused on one viewpoint). Improved transparency in the implementation of CAA in SRs in needed to ensure their validity and allow robust assessments of causality within evidence synthesis.

[Utilization and reporting of qualitative research methods in health services research in Germany: a scoping review on original research publications (2010-2019)]

BACKGROUND

Qualitative research methods offer a unique perspective on health care services. However, little is known about the actual application of qualitative methods in health services research. Therefore, the aim of this study was to gain an overview of volume and variety of the use of qualitative research methods in health services research in Germany.

METHODS

By means of a scoping review, a systematic literature search of the database PubMed was conducted in September 2020. We included (1) qualitative studies in (2) a health services setting (3) in Germany, (4) published in either German or English as (5) original research in a journal (6) between 2010 and 2019. After removing duplicates, tandem teams of researchers first performed a title and abstract screening, followed by a full text screening. Data was extracted by using a category grid considering research focus, study design and reporting.

RESULTS

In total, 759 articles were included in the title and abstract screening. After applying the exclusion criteria, 97 articles were included in the data extraction. The studies investigating mainly subjective perspectives of different stakeholders, especially physicians and patients, covered 13 areas of health care. Interviews were the dominant form of data collection (n=64). Data analysis was mainly conducted using content analysis (n=65).

CONCLUSION

A clear absolute increase in publications since the mid-2010s can be observed. At the same time, there has been a strong tendency towards certain methods being used for data collection and analysis. Compared to reporting standards and guidelines (e.g., COREQ), incomplete reporting of research methods has been noted. The results show that both an extension of the range of methods and the quality of reporting need to be discussed.

Design and methodological characteristics of studies using observational routinely collected health data for investigating the link between cancer and neurodegenerative diseases: protocol for a meta-research study

INTRODUCTION

Health services generate large amounts of routine health data (eg, administrative databases, disease registries and electronic health records), which have important secondary uses for research. Increases in the availability and the ability to access and analyse large amounts of data represent a major opportunity for conducting studies on the possible relationships between complex diseases. The objective of this study will be to evaluate the design, methods and reporting of studies conducted using observational routinely collected health data for investigating the link between cancer and neurodegenerative diseases.

METHODS AND ANALYSIS

This is the protocol for a meta-research study. We registered the study protocol within the Open Science Framework: https://osf.io/h2qjg. We will evaluate observational studies (eg, cohort and case-control) conducted using routinely collected health data for investigating the associations between cancer and neurodegenerative diseases (such as Alzheimer's disease, amyotrophic lateral sclerosis/motor neuron disease, Huntington's disease, multiple sclerosis and Parkinson's disease). The following electronic databases will be searched (from their inception onwards): MEDLINE, Embase and Web of Science Core Collection. Screening and selection of articles will be conducted by at least two researchers. Potential discrepancies will be resolved via discussion. Design, methods and reporting characteristics in each article will be extracted using a standardised data extraction form. Information on general, methodological and transparency items will be reported. We will summarise our findings with tables and graphs (eg, bar charts, forest plots).

ETHICS AND DISSEMINATION

Due to the nature of the proposed study, no ethical approval will be required. We plan to publish the full study in an open access peer-reviewed journal and disseminate the findings at scientific conferences and via social media. All data will be deposited in a cross-disciplinary public repository.

Systematic reviews and meta-analysis published in indexed Portuguese medical journals: time trends and critical appraisal

Introduction

Over the last years, the number of systematic reviews published is steadily increasing due to the global interest in this type of evidence synthesis. However, little is known about the characteristics of this research published in Portuguese medical journals. This study aims to evaluate the publication trends and overall quality of these systematic reviews.

Material and methods

This was a methodological study. We aimed the most visible Portuguese medical journals indexed in MEDLINE. Systematic reviews were identified through an electronic search (through PUBMED). We included systematic reviews published up to August 2020. Systematic reviews selection and data extraction were done independently by three authors. The overall quality critical appraisal using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2) was independently assessed by three authors. Disagreements were solved by consensus.

Results

Sixty-six systematic reviews published in 5 Portuguese medical journals were included. Most (n = 53; 80.3%) were systematic reviews without meta-analysis. Up to 2010 there was a steady increase in the number of systematic reviews published, followed by a period of great variability of publication, ranging from 1 to 10 in a given year. According to the systematic reviews’ typology, most have been predominantly conducted to assess the effectiveness/efficacy of health interventions (n = 27; 40.9%). General and Internal Medicine (n = 20; 30.3%) was the most addressed field. Most systematic reviews (n = 46; 69.7%) were rated as being of “critically low-quality”.

Conclusions

There were consistent flaws in the methodological quality report of the systematic reviews included, particularly in establishing a prior protocol and not assessing the potential impact of the risk of bias on the results.

Through the years, the number of systematic reviews published increased, yet their quality is suboptimal. There is a need to improve the reporting of systematic reviews in Portuguese medical journals, which can be achieved by better adherence to quality checklists/tools.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01591-z.

Pilot and Feasibility Studies in Rehabilitation Research: A Review and Educational Primer for the Physiatrist Researcher

ABSTRACT

Pilot and feasibility studies are conducted early in the clinical research pathway to evaluate whether a future, definitive study can or should be done and, if so, how. Poor planning and reporting of pilot and feasibility studies can compromise subsequent research efforts. Inappropriate labeling of studies as pilots also compromises education. In this review, first, a systematic survey of the current state of pilot and feasibility studies in rehabilitation research was performed, and second, recommendations were made for improvements to their design and reporting. In a random sample of 100 studies, half (49.5%) were randomized trials. Thirty (30.0%) and three (3.0%) studies used "pilot" and "feasibility" in the study title, respectively. Only one third (34.0%) of studies provided a primary objective related to feasibility. Most studies (92.0%) stated an intent for hypothesis testing. Although many studies (70.0%) mentioned outcomes related to feasibility in the methods, a third (30.0%) reported additional outcomes in the results and discussion only or commented on feasibility anecdotally. The reporting of progression plans to a main study (21.0%) and progression criteria (4.0%) was infrequent. Based on these findings, it is recommended that researchers correctly label studies as a pilot or feasibility design based on accepted definitions, explicitly state feasibility objectives, outcomes, and criteria for determining success of feasibility, justify the sample size, and appropriately interpret and report the implications of feasibility findings for the main future study.

Risk of bias in chiropractic mixed methods research: a secondary analysis of a meta-epidemiological review

OBJECTIVE

To examine the risk of bias in chiropractic mixed methods research.

METHODS

We performed a secondary analysis of a meta-epidemiological review of chiropractic mixed methods studies. We assessed risk of bias with the Mixed Methods Appraisal Tool (MMAT) and used generalized estimating equations to explore factors associated with risk of bias.

RESULTS

Among 55 eligible studies, a mean of 62% (6.8 [2.3]/11) of MMAT items were fulfilled. In our adjusted analysis, studies published since 2010 versus pre-2010 (adjusted odds ratio [aOR] = 2.26; 95% confidence interval [CI], 1.39 to 3.68) and those published in journals with an impact factor versus no impact factor (aOR = 2.21; 95% CI, 1.33 to 3.68) were associated with lower risk of bias.

CONCLUSION

Our findings suggest opportunities for improvement in the quality of conduct among published chiropractic mixed methods studies. Author compliance with the MMAT criteria may reduce methodological bias in future mixed methods research.

The use of cognitive task analysis in clinical and health services research — a systematic review

Background

At times, clinical case complexity and different types of uncertainty present challenges to less experienced clinicians or the naive application of clinical guidelines where this may not be appropriate. Cognitive task analysis (CTA) methods are used to elicit, document and transfer tacit knowledge about how experts make decisions.

Methods

We conducted a methodological review to describe the use of CTA methods in understanding expert clinical decision-making. We searched MEDLINE, EMBASE and PsycINFO from inception to 2019 for primary research studies which described the use of CTA methods to understand how qualified clinicians made clinical decisions in real-world clinical settings.

Results

We included 81 articles (80 unique studies) from 13 countries, published from 1993 to 2019, most commonly from surgical and critical care settings. The most common aims were to understand expert decision-making in particular clinical scenarios, using expert decision-making in the development of training programmes, understanding whether decision support tools were warranted and understanding procedural variability and error identification or reduction. Critical decision method (CDM) and CTA interviews were most frequently used, with hierarchical task analysis, task knowledge structures, think-aloud protocols and other methods less commonly used. Studies used interviews, observation, think-aloud exercises, surveys, focus groups and a range of more CTA-specific methodologies such as the systematic human error reduction and prediction approach. Researchers used CTA methods to investigate routine/typical (n = 64), challenging (n = 13) or more uncommon, rare events and anomalies (n = 3).

Conclusions

In conclusion, the elicitation of expert tacit knowledge using CTA has seen increasing use in clinical specialties working under challenging time pressures, complexity and uncertainty. CTA methods have great potential in the development, refinement, modification or adaptation of complex interventions, clinical protocols and practice guidelines.

Registration

PROSPERO ID CRD42019128418.

Supplementary Information

The online version contains supplementary material available at 10.1186/s40814-022-01002-6.

Self-reported sleep duration and timing: A methodological review of event definitions, context, and timeframe of related questions

Study Objectives

Clinical and population health recommendations are derived from studies that include self-report. Differences in question wording and response scales may significantly affect responses. We conducted a methodological review assessing variation in event definition(s), context (i.e., work- versus free-day), and timeframe (e.g., "in the past 4 weeks") of sleep timing/duration questions.

Methods

We queried databases of sleep, medicine, epidemiology, and psychology for survey-based studies and/or publications with sleep duration/timing questions. The text of these questions was thematically analyzed.

Results

We identified 53 surveys with sample sizes ranging from 93 to 1,185,106. For sleep duration, participants reported nocturnal sleep (24/44), sleep in the past 24-hours (14/44), their major sleep episode (3/44), or answered unaided (3/44). For bedtime, participants reported time into bed (19/47), first attempt to sleep (16/40), or fall-asleep time (12/47). For wake-time, participants reported wake-up time (30/43), the time they "get up" (7/43), or their out-of-bed time (6/43). Context guidance appeared in 18/44 major sleep duration, 35/47 bedtime, and 34/43 wake-time questions. Timeframe was provided in 8/44 major sleep episode duration, 16/47 bedtime, and 10/43 wake-time questions. One question queried the method of awakening (e.g., by alarm clock), 18 questions assessed sleep latency, and 12 measured napping.

Conclusion

There is variability in the event definition(s), context, and timeframe of questions relating to sleep. This work informs efforts at data harmonization for meta-analyses, provides options for question wording, and identifies questions for future surveys.

Virological measures and factors associated with outcomes, and missing outcome data in HIV clinical trials: a methodological study

BACKGROUND

To evaluate the definition of HIV virological outcomes in the literature and factors associated with outcomes and missing outcome data.

METHODS

We conducted a methodological review of HIV RCTs using a search (2009-2019) of PubMed, Embase and the Cochrane Central Register of Controlled Trials.Only full-text, peer-reviewed, randomised controlled trials (RCTs) that measured virological outcomes in people living with HIV, and published in English were included.We extracted study details and outcomes. We used logistic regression to identify factors associated with a viral threshold ≤50 copies/mL and linear regression to identify factors associated with missing outcome data.

RESULTS

Our search yielded 5847 articles; 180 were included. A virological outcome was the primary outcome in 73.5% of studies. 89 studies (49.4%) used virological success. The remaining used change in viral load (VL) (33 studies, 18.3%); virological failure (59 studies, 32.8%); or virological rebound (9 studies, 5.0%). 96 studies (53.3%) set the threshold at ≤50 copies/mL; and 33.1% used multiple measures.Compared with government and privately funded studies, RCTs with industry funding (adjusted OR 6.39; 95% CI 2.15 to 19.00; p<0.01) were significantly associated with higher odds of using a VL threshold of ≤50 copies/mL. Publication year, intervention type, income level and number of patients were not associated with a threshold of ≤50 copies/mL. Trials with pharmacological interventions had less missing data (β=-11.04; 95% CI -20.02 to -1.87; p=0.02).

DISCUSSION

Country source of funding was associated with VL threshold choice and studies with pharmacological interventions had less missing data, which may in part explain heterogeneous virological outcomes across studies. Multiple measures of VL were not associated with missing data. The development of formal guidelines on virological outcome reporting in RCTs is needed.

Quality of reporting in chiropractic mixed methods research: a methodological review protocol

Background

Mixed methods designs are increasingly used in health care research to enrich findings. However, little is known about the frequency of use of this methodology in chiropractic research, or the quality of reporting among chiropractic studies using mixed methods.

Objective

To quantify the use and quality of mixed methods in chiropractic research, and explore the association of study characteristics (e.g., authorship, expertise, journal impact factor, country and year of publication) with reporting quality.

Methods

We will conduct a systematic search of MEDLINE, EMBASE, CINAHL, and the Index to Chiropractic Literature to identify all chiropractic mixed methods studies published from inception of each database to December 31, 2020. Articles reporting the use of both qualitative and quantitative methods, or mixed qualitative methods, will be included. Pairs of reviewers will perform article screening, data extraction, risk of bias with the Mixed Methods Appraisal Tool (MMAT), and appraisal of reporting quality using the Good Reporting of A Mixed Methods Study (GRAMMS) guideline. We will explore the correlation between GRAMMS and MMAT scores, and construct generalized estimating equations to explore factors associated with reporting quality.

Discussion

This will be the first methodological review to examine the reporting quality of published mixed methods studies involving chiropractic research. The results of our review will inform opportunities to improve reporting in chiropractic mixed methods studies. Our results will be disseminated in a peer-reviewed publication and presented publicly at conferences and as part of a doctoral thesis.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12998-021-00395-0.

Examining the Public's Most Frequently Asked Questions Regarding COVID-19 Vaccines Using Search Engine Analytics in the United States: Observational Study

Background

The emergency authorization of COVID-19 vaccines has offered the first means of long-term protection against COVID-19–related illness since the pandemic began. It is important for health care professionals to understand commonly held COVID-19 vaccine concerns and to be equipped with quality information that can be used to assist in medical decision-making.

Objective

Using Google’s RankBrain machine learning algorithm, we sought to characterize the content of the most frequently asked questions (FAQs) about COVID-19 vaccines evidenced by internet searches. Secondarily, we sought to examine the information transparency and quality of sources used by Google to answer FAQs on COVID-19 vaccines.

Methods

We searched COVID-19 vaccine terms on Google and used the “People also ask” box to obtain FAQs generated by Google’s machine learning algorithms. FAQs are assigned an “answer” source by Google. We extracted FAQs and answer sources related to COVID-19 vaccines. We used the Rothwell Classification of Questions to categorize questions on the basis of content. We classified answer sources as either academic, commercial, government, media outlet, or medical practice. We used the Journal of the American Medical Association’s (JAMA’s) benchmark criteria to assess information transparency and Brief DISCERN to assess information quality for answer sources. FAQ and answer source type frequencies were calculated. Chi-square tests were used to determine associations between information transparency by source type. One-way analysis of variance was used to assess differences in mean Brief DISCERN scores by source type.

Results

Our search yielded 28 unique FAQs about COVID-19 vaccines. Most COVID-19 vaccine–related FAQs were seeking factual information (22/28, 78.6%), specifically about safety and efficacy (9/22, 40.9%). The most common source type was media outlets (12/28, 42.9%), followed by government sources (11/28, 39.3%). Nineteen sources met 3 or more JAMA benchmark criteria with government sources as the majority (10/19, 52.6%). JAMA benchmark criteria performance did not significantly differ among source types (χ²₄=7.40; P=.12). One-way analysis of variance revealed a significant difference in mean Brief DISCERN scores by source type (F_4,23=10.27; P<.001).

Conclusions

The most frequently asked COVID-19 vaccine–related questions pertained to vaccine safety and efficacy. We found that government sources provided the most transparent and highest-quality web-based COVID-19 vaccine–related information. Recognizing common questions and concerns about COVID-19 vaccines may assist in improving vaccination efforts.

Completeness of reporting of quality improvement studies in neonatology is inadequate: a systematic literature survey

INTRODUCTION

Quality improvement (QI) is a growing field of inquiry in healthcare, but the reporting quality of QI studies in neonatology remains unclear. We conducted a systematic survey of the literature to assess the reporting quality of QI studies and factors associated with reporting quality.

METHODS

We searched Medline for publications of QI studies from 2016 to 16 April 2020. Pairs of reviewers independently screened citations and assessed reporting quality using a 31-item modified Standards for Quality Improvement Reporting Excellence, 2nd edition (SQUIRE 2.0) checklist. We reported the number (percentage) of studies that reported each item and their corresponding 95% CIs. We used Poisson regression to explore factors associated with reporting quality, namely, journal endorsement of SQUIRE 2.0, declaration of funding sources, year of publication and number of authors. The results were reported as incidence rate ratio (IRR) and 95% CI.

RESULTS

Of 1921 citations, 336 were eligible; among them, we randomly selected 100 articles to assess reporting quality. The mean (standard deviation) number of SQUIRE 2.0 items adhered to was 22.0 (4.5). Percentage of articles reporting each item varied from 26% to 100%. Journal endorsement of SQUIRE 2.0 (IRR=1.11, 95% CI 1.02 to 1.21, p=0.015), declaration of funding sources and increasing number of authors were significantly associated with better reporting.

CONCLUSIONS

Reporting quality of QI studies in neonatology is inadequate. Endorsing the SQUIRE 2.0 guideline is a step that journals can implement to enhance the completeness of reporting.

Are randomized controlled trials in urology being conducted with justification?

CONTEXT

Considering the substantial increase in research funding in the field of urology, minimizing research waste should be a top priority. Systematic reviews (SRs) compile available evidence regarding a clinical question into a single critical resource. If properly utilized, SRs can help minimize redundant studies, focus attention to unsubstantiated treatments, and reduce research waste.

OBJECTIVES

To appraise the use of SRs as justification for conducting randomized controlled trials (RCTs) published in high impact urology journals, and to report the ways SRs were incorporated into RCT manuscripts published in the top four urology journals by h5 index.

METHODS

On December 13, 2019, a PubMed search was conducted for RCTs published in the top four urology journals according to the Google Scholar h5-index: European Urology, BJU International, The Journal of Urology, and Urology. For an article to be eligible for inclusion in this study, it must have been a full length RCT, published between November 30, 2014, and November 30, 2019 in one of the identified journals, reported only human subjects, and been accessible in English. The following data points were extracted independently by select investigators from each included RCT: manuscript title, year of publication, journal title, type of intervention (drug, medical device, procedure, other), funding source (government, hospital/university, industry, mixed) type of trial (parallel groups, crossover, cluster), and total number of participants reported in each RCT. The included RCTs were searched for reference to an SR, which was then recorded as "yes - verbatim," "yes - inferred," or "not used as justification" and the location in the manuscript where the SR was cited was recorded.

RESULTS

Of the 566 articles retrieved, 276 were included. Overall, 150 (54.3%) RCTs cited an SR as either verbatim (108; 39.1%) or inferred (42; 15.2%) trial justification, while 126 (45.7%) did not use an SR for RCT justification. Of those 126, 107 (84.9%) RCTs did not cite an SR to any extent. A significant association was noted between verbatim justification and type of intervention (x ²=20.23, p=0.017), with 18 of 31 (58.1%) "other" interventions (i.e. psychosocial intervention, exercise programs, and online therapy) having an SR cited as verbatim justification. Only 39 of 118 (33.1%) pharmaceutical trials referenced an SR as verbatim justification. Of 403 systematic review citations, 205 (50.8%) appeared in the Discussion section, while 15 (3.7%) were in the Methods section.

CONCLUSIONS

We found that RCTs published in four high impact urology journals inconsistently referenced an SR as justification and 39.1% of our entire sample did not reference an SR at all. These findings indicate that a divide exists between the instruction and implementation of evidence based medicine in the field of urology concerning RCTs published in the top four journals. Educating clinicians and researchers on the use of SR as justification for RCTs in urology may reduce research waste and increase the quality of RCTs in the field.

Suicide Prevention Among College Students Before and During the COVID-19 Pandemic: Protocol for a Systematic Review and Meta-analysis

BACKGROUND

Suicide is the second leading cause of death for college-aged individuals worldwide and in the United States. Recent studies have identified preliminary evidence of widening disparities in suicidal behaviors across sex, sexual orientation, race/ethnicity, age, and socioeconomic status among college students. Few systematic reviews and meta-analyses offer a comprehensive understanding of on-campus and off-campus suicide interventions, nor is collated information available for different types of screening, assessment, treatment, and postvention plans. Further challenges have been identified since the COVID-19 pandemic, calling for cost-effective and innovative interventions to address increased rates of suicidal behaviors among college students facing unprecedented stressors.

OBJECTIVE

This research protocol describes the first systematic review and meta-analysis to identify the most effective and cost-effective intervention components for universal and targeted (indicated and selected) suicide prevention among college students in a global context. Special attention will be placed on disparities in suicide prevention across sociodemographic subgroups, inclusive interventions beyond campus, global context, and intervention responses to the COVID-19 pandemic.

METHODS

A sensitive search strategy will be executed across MEDLINE (Ovid), EMBASE, PsycINFO (EBSCO), ERIC (EBSCO), Cochrane Library, Dissertations and Theses Global (ProQuest), Scopus, Global Index Medicus, SciELO, African Journals Online, Global Health (CABI), and Google Scholar. Data extraction and evaluation will be conducted by three independent researchers. Risk of bias will be assessed. A multilevel meta-regression model and subgroup analysis will be used to analyze the data and estimate effect sizes.

RESULTS

The initial search was completed in December 2020 and updated with additional other-language studies in March 2020. We expect the results to be submitted for publication in mid-2021.

CONCLUSIONS

Despite increasing rates of suicidal behaviors among college students, few preventative efforts have targeted this population, and fewer focus on factors that might place specific demographic groups at heightened risk. The impact of COVID-19 on suicidal behaviors among college students highlights and exacerbates the urgent need for rapid and effective interventions that might differ from traditional approaches. This equity-focused study will address these gaps and provide a valuable analysis of the effectiveness of suicide prevention programs and interventions. Findings will inform clinicians, researchers, policy makers, families, and organizations about evidence-based interventions for reducing the gaps in the suicide crisis among college students from different sociodemographic groups.

TRIAL REGISTRATION

PROSPERO CRD42020225429; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=225429.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/26948.

Guidelines are needed for studies of pre-treatment HIV drug resistance: a methodological study

BACKGROUND

The expansion of access to antiretroviral therapy (ART) has been accompanied by an increase in pre-treatment drug resistance (PDR). While it is critical to monitor the increasing prevalence of PDR across countries and populations to inform optimal regimen selection, the completeness of reporting is often suboptimal, limiting the interpretation and generalizability of the results. Indeed, there is no formal guidance on how studies investigating the prevalence of drug resistance should be reported. Thus, we sought to determine the completeness of reporting in studies of PDR and the factors associated with sub-optimal reporting to ascertain the need for guidelines.

METHODS

As part of a systematic review on the global prevalence of PDR in key populations (men who have sex with men, sex workers, transgender people, people who inject drugs and people in prisons), we searched 10 electronic databases until January 2019. We extracted information on selected study characteristics useful for interpreting prevalence data. Data were extracted in duplicate. Analyses of variance and correlation were used to explore factors that may explain the number of items reported.

RESULTS

We found 650 studies of which 387 were screened as full text and 234 were deemed eligible. The included studies were published between 1997 and 2019 and included a median of 239 (quartile 1 = 101; quartile 3 = 778) participants. Most studies originated from high-income countries (125/234; 53.0%). Of 23 relevant data items, including study design, setting, participant sociodemographic characteristics, HIV risk factors, type of resistance test conducted, definition of resistance, the mean (standard deviation) number of items reported was 13 (2.2). We found that more items were reported in studies published more recently (r = 0.20; p < 0.002) and in studies at low risk of bias (F [2231] = 8.142; p < 0.001).

CONCLUSIONS

Incomplete reporting in studies on PDR makes characterising levels of PDR in subpopulations across countries challenging. Hence, guidelines are needed to define a minimum set of variables to be included in such studies.

A methodological review protocol of the use of Bayesian factor analysis in primary care research

BACKGROUND

The development of questionnaires for primary care practice and research is of increasing interest in the literature. In settings where valuable prior knowledge or preliminary data is available, Bayesian factor analysis can be used to incorporate such information when conducting questionnaire construct validation. This protocol outlines a methodological review that will summarize evidence on the current use of Bayesian factor analysis in the primary care literature.

METHODS

A comprehensive search strategy has been developed and will be used to identify relevant literature (research studies in primary care) indexed in MEDLINE, Scopus, EMBASE, CINAHL, and Cochrane Library. The search strategy includes terms and synonyms for Bayesian factor analysis and primary care. The reference lists of relevant articles being identified will be screened to find further relevant studies. At least two reviewers will independently extract data and resolve discrepancies through consensus. Descriptive analyses will summarize the use and reporting of Bayesian factor analysis approaches for validating questionnaires applicable to primary care.

DISCUSSION

This methodological review will provide a comprehensive overview of the current use and reporting of Bayesian factor analysis in primary care and will provide recommendations for its proper future use.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42018114978.

Qualitative research methods in medical dissertations: an observational methodological study on prevalence and reporting quality of dissertation abstracts in a German university

Background

Qualitative methods offer a unique contribution to health research. Academic dissertations in the medical field provide an opportunity to explore research practice. Our aim was to assess the use of qualitative methods in dissertations in the medical field.

Methods

By means of a methodological observational study, an analysis of all academic medical dissertations’ abstracts between 1998 and 2018 in a repository databank of a large medical university faculty in Germany was performed. This included MD dissertations (Dr. med. (dent.)) and medical science dissertations (Dr. sc. hum.). All abstracts including “qualitativ*” were screened for studies using qualitative research methods. Data were extracted from abstracts using a category grid considering a) general characteristics (year, language, degree type), b) discipline, c) study design (mixed methods/qualitative only, data conduction, data analysis), d) sample (size and participants) and e) technologies used (data analysis software and recording technology). Thereby reporting quality was assessed.

Results

In total, 103 abstracts of medical dissertations between 1998 and 2018 (1.4% of N = 7619) were included, 60 of MD dissertations and 43 of medical sciences dissertations. Half of the abstracts (n = 51) referred to dissertations submitted since 2014. Most abstracts related to public health/hygiene (n = 27) and general practice (n = 26), followed by medical psychology (n = 19). About half of the studies (n = 47) used qualitative research methods exclusively, the other half (n = 56) used mixed methods. For data collection, primarily individual interviews were used (n = 80), followed by group interviews (n = 33) and direct observation (n = 11). Patients (n = 36), physicians (n = 36) and healthcare professionals (n = 17) were the most frequent research participants. Incomplete reporting of participants and data analysis was common (n = 67). Nearly half of the abstracts (n = 46) lacked information on how data was analysed, most of the remaining (n = 43) used some form of content analysis. In summary, 36 abstracts provided all crucial data (participants, sample size,; data collection and analysis method).

Conclusion

A small number of academic dissertations used qualitative research methods. About a third of these reported all key aspects of the methods used in the abstracts. Further research on the quality of choice and reporting of methods for qualitative research in dissertations is recommended.

Guidelines rarely used GRADE and applied methods inconsistently: A methodological study of Australian guidelines

OBJECTIVES

The Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach is accepted methodology to assess the certainty of the evidence included in systematic reviews and clinical practice guidelines. The GRADE approach is endorsed globally, in Australia, the National Health and Medical Research Council advocated for the use of the GRADE approach in 2011. The purpose of this methodological review was to assess how GRADE has been adopted for Australian practice guidelines.

STUDY DESIGN AND SETTING

This methodological review searched of the National Health and Medical Research Council Clinical Practice Guidelines Portal from 2011 to 2018, in an effort to retrieve all practice guidelines available via this medium.

RESULTS

240 guidelines were retrieved authored by 51 different organizations. 15 guidelines followed GRADE methodology. Application of GRADE methods varied between guidelines, some misreported and altered aspects of the GRADE process. Guidelines that closely adhered to the guidance from the GRADE Working Group scored higher in domain 3 (rigor of development) of the Appraisal of Guidelines for Research and Evaluation II tool, indicating a positive linear relationship between GRADE adherence and rigor of development scores.

CONCLUSION

The results of our project suggest that the use of GRADE in Australian guidelines is increasing, however, strategies to increase uptake and reporting within the guideline community need to be explored.