Reporting of methodological studies in health research: a protocol for the development of the MethodologIcal STudy reportIng Checklist (MISTIC)

Access to treatment protocols and manuals for evidence-based psychological interventions for severe mental disorders: a survey of randomised trials included in network meta-analyses

BACKGROUND

Evidence-based psychological interventions for mental disorders are described in treatment protocols and manuals, which detail treatment components and conditions of application. Systematic evaluations of the accessibility of treatment protocols and manuals across multiple mental disorders are absent.

OBJECTIVE

We assessed whether treatment protocols or manuals for psychological interventions for severe mental disorders are accessible and publicly available.

STUDY SELECTION AND ANALYSIS

We surveyed randomised controlled trials (RCTs) from six large network meta-analyses of psychological interventions for severe mental disorders (psychotic, borderline personality, substance use, bipolar, anorexia and bulimia nervosa). Between January 2024 and February 2025, we retrieved protocols and manuals of psychological intervention arms using a multipronged approach (published protocol, trial registries, author contact, commercial availability). We report the proportion of trials and intervention arms for which protocols or manuals were (1) Accessible, that is, retrievable by any method, and (2) Publicly versus commercially available.

FINDINGS

We included 260 RCTs, with 422 active intervention arms. We retrieved published protocols for 20 RCTs (8%, 95% CI 5% to 12%) and contacted 450 authors for the remaining 240. Authors shared protocols for 43/240 trials (18%, 95% CI 13% to 23%), refused to share for 73 (30%, 95% CI 25% to 37%) and did not respond for 101 (42%, 95% CI 36% to 49%). Protocols or manuals were retrievable for 364 psychological intervention arms (86%, 95% CI 83% to 89%), with 191 available commercially (45%, 95% CI 40% to 50%) and 106 (25%, 95% CI 21% to 30%) publicly.

CONCLUSIONS AND CLINICAL IMPLICATIONS

Retrieving detailed descriptions of psychological interventions used in trials, crucial for identifying treatment components, was challenging, resource-intensive and required multiple methods. Reliance on public availability and author sharing enabled access to about 40% of protocols or manuals.

Assessing the time required for qualitative analysis: A comparative methodological study of coding interview data in health services research

A reliable estimation of required resources is essential for sound research. So far, there have only been a few studies on researchers' time investment in qualitative studies. The aim of this study, therefore, was to provide an empirical account of the estimation of timescales of qualitative analysis.In this methodological study, time expenditure was documented and compared for the focused coding of transcripts of semi-structured interviews within five qualitative studies in health services research. Data were analyzed descriptively by means of absolute frequencies.Across studies, focused coding was assessed in 94 interviews with a total interview duration of 52 hours and 44 minutes. The number of interviews per study ranged from n=11 to n=27, with a mean duration of 36 minutes. Total coding time amounted to 76 hours, with a mean of 32 min per interview. Coding time per interview time ratio ranged from 0.75 to 1.52 minutes. On average, the time spent on focused coding roughly corresponds to the duration of the interviews. Focused coding tended to get quicker over time, though variation among studies was high.The results of this study provide a reference for estimating timescales of qualitative analysis and highlights the importance of considering factors such as composition of data and researchers' experience and involvement. In a specific research project, this effort must be balanced against the objective of the analysis, including the desired accuracy, detail and depth. Further research is needed to specify how specific parameters (i. e. nature of the study population, method of data analysis and use of concepts and theories) affect coding in qualitative analysis.

Measures of diabetic retinopathy treatment coverage: protocol for a methodological review

INTRODUCTION

Diabetic retinopathy is one of the leading causes of vision impairment globally. Alongside the systemic control of diabetes and timely detection of diabetic retinopathy, the prompt initiation and completion of treatment is essential to prevent vision loss. Routine monitoring of access to retinal screening services for the detection of diabetic retinopathy is common, while monitoring of coverage of subsequent treatment services is far less common. When diabetic retinopathy treatment coverage is assessed, there is great variability in how it is defined and reported. If a definition of treatment coverage could be standardised, the monitoring of the quality of diabetes eye care could more readily be compared between settings and over time. The aim of this review is to summarise how diabetic retinopathy treatment coverage has been measured in published studies and the extent to which these have been disaggregated by population groups.

METHODS AND ANALYSIS

A search will be conducted on Medline and Embase without any language restrictions, for cohort and cross-sectional studies published from 1 January 2015 that report diabetic retinopathy treatment coverage for adults with diabetic retinopathy and/or macular oedema. We will include studies from any world region reporting diabetic retinopathy treatment coverage for one or more of: (1) laser photocoagulation; (2) intravitreal injections of antivascular endothelial growth factor agents; (3) intravitreal injections of corticosteroids; (4) vitrectomy. The PROGRESS framework (place of residence, race/ethnicity/culture/language, occupation, gender/sex, religion, education, socioeconomic status and social capital) will be used to assess disaggregation by population groups. Two investigators will independently screen studies and extract relevant data. Data will be synthesised descriptively to outline the full range of definitions of diabetic retinopathy treatment coverage in the literature and identify the common sources of data used.

ETHICS AND DISSEMINATION

This review will only include published data; thus, no ethical approval will be sought. The findings of this review will be published in a peer-reviewed journal and presented at relevant conferences. The findings will also be considered in conjunction with an ongoing review on retinal screening for diabetic retinopathy to develop indicators for monitoring of services along the diabetes eye care pathway, which may include an indicator of effective service coverage.

REGISTRATION

Open Science Framework registration 6/08/2024: https://osf.io/5b93m.

Therapeutic quality of exercise interventions for chronic low back pain: a meta-research study using i-CONTENT tool

OBJECTIVE

To assess the therapeutic quality of exercise interventions delivered in chronic low back pain (cLBP) trials using the international Consensus on Therapeutic Exercise aNd Training (i-CONTENT) tool and its inter-rater agreement.

METHODS

We performed a meta-research study, starting from the trials' arms included in the published Cochrane review (2021) 'Exercise therapy for chronic low back pain'. Two pairs of independent reviewers applied the i-CONTENT tool, a standardised tool designed to ensure the quality of exercise therapy intervention, in a random sample of 100 different exercise arms. We assessed the inter-rater agreement of each category calculating the specific agreement. A percentage of 70% was considered satisfactory.

RESULTS

We included 100 arms from 68 randomised controlled trials published between 1991 and 2019. The most assessed exercise types were core strengthening (n=27 arms) and motor control (n=13 arms). Among alternative approaches, yoga (n=11) and Pilates (n=7) were the most representative. Overall, most exercise interventions were rated as having a low risk of ineffectiveness for patient selection (100%), exercise type (92%), outcome type and timing (89%) and qualified supervisor (84%). Conversely, some items showed more uncertainty: the safety of exercise programmes was rated as 'probably low risk' in 58% of cases, exercise dosage in 34% and adherence to exercise in 44%. The items related to exercise dosage (31%) and adherence (29%) had heterogenous judgements, scoring as high risk of ineffectiveness or probably not done. Among all exercise types, Pilates scored best in all domains. A satisfactory specific agreement for 'low risk category' was achieved in all items, except dosage of exercise (60%) and adherence to exercise (54%).

CONCLUSION

Exercises delivered for patients with cLBP generally demonstrate favourable therapeutic quality, although some exercise modalities may present poor therapeutic quality related to dosage and adherence. While the i-CONTENT judgements generally showed satisfactory specific agreement between raters, disagreements arose in evaluating some crucial items.

Methods for assessing peri-implant marginal bone levels on digital periapical radiographs: a meta-research

OBJECTIVES

This meta-research assessed methodologies used for evaluating peri-implant marginal bone levels on digital periapical radiographs in randomized clinical trials published between 2019 and 2023.

METHODS

Articles were searched in four databases. Data on methods for assessing peri-implant marginal bone levels were extracted. Risk of bias assessment was performed.

RESULTS

During full-text reading, 108 out of 162 articles were excluded. Methodological issues accounted for these exclusions, including the absence of radiograph-type information, the lack of radiographic positioners, the missing anatomical references, and the use of panoramic radiographs or tomography. Fifty-four articles were included, most from Europe (70%) and university-based (74%). Radiographic positioners were specified in 54% of articles. Examiner calibration was unreported in 54%, with 69% lacking details. In 59%, no statistical measure assessed examiner agreement. Blinding was unreported or unused in 50%. Marginal bone level changes were the primary outcome of 61%. Most articles (59.3%) raised "some concerns" regarding bias, while 37% showed a high risk of bias, and only two articles (3.7%) demonstrated a low risk of bias.

CONCLUSIONS

Several limitations and areas for improvement were identified. Future studies should prioritize protocol registration, standardize radiographic acquisitions, specify examiner details, implement calibration and statistical measures for agreement, introduce blinding protocols, and maintain geometric calibration standards.

Addressing equity, diversity, and inclusion in JBI qualitative systematic reviews: a methodological scoping review

OBJECTIVE

The objective of this methodological scoping review was to investigate ways in which qualitative review teams are addressing equity, diversity, and inclusion (EDI) in the process of conducting and reporting qualitative systematic reviews that use JBI guidelines.

INTRODUCTION

To promote health equity, there is a need for evidence synthesis processes and practices to develop approaches that incorporate EDI. Some guidance is available to guide equity-focused review methods and reporting, but this is primarily oriented to quantitative systematic reviews. There is currently limited knowledge about how review teams are addressing EDI within qualitative evidence syntheses.

INCLUSION CRITERIA

This review included English-language qualitative systematic reviews, published in 2022, that used all the stjpg outlined in the JBI guidance for qualitative reviews.

METHODS

A 1-year sample of published reviews was identified from a search undertaken on March 17, 2023, of 2 health care databases: MEDLINE (Ovid) and CINAHL (EBSCOhost). Data extraction followed a framework approach, using an adapted pre-existing equity template. This included attention to i) the reporting of a range of characteristics associated with EDI, ii) search approaches, and iii) analytical approaches (including reflexivity, intersectionality, and knowledge user engagement). Data were analyzed using descriptive statistics and narrative summary.

RESULTS

Forty-three reviews met the inclusion criteria. The majority of the reviews (n = 30) framed their questions and aims in a generic/universal (rather than EDI-focused) way. Six reviews justified their population focus in terms of an EDI-related issue. Only 1 review included a knowledge user. The sociodemographic and other key characteristics of the samples in underpinning studies were poorly reported, making it hard to discern EDI-related issues or to undertake EDI-related analyses. Thirteen of the reviews included non-English-language evidence sources, and 31 reviews included gray literature sources. Ten reviews demonstrated an element of intersectional or otherwise critical approach within their analyses of categories and synthesized findings (whereby issues of power and/or representation were explicitly considered). Only 8 reviews included discussions of review team composition and reflexivity within the review process.

CONCLUSIONS

This EDI-focused methodological enquiry has highlighted some limitations within current qualitative evidence synthesis practice. Without closer attention to EDI, there is a danger that systematic reviews may simply serve to amplify, rather than illuminate, existing gaps, silences, and inequitable knowledge claims based on dominant representations. This review sets out a range of suggestions to help qualitative evidence synthesis teams to more systematically embed EDI within their methods and practices.

REVIEW REGISTRATION

Open Science Framework https://osf.io/wy5kv/.

Proposed scenario for post-suicide attempt care for individuals undergoing gender transition

OBJECTIVE

To develop and validate a simulated scenario for post-suicide attempt care for a person undergoing gender transition.

METHOD

This was a methodological study conducted in two stages: (i) development of the scenario based on the literature and clinical simulation recommendations; (ii) validation by ten experts, through an online assessment using a Likert scale and suggestions. Descriptive analyses and content validity index were applied.

RESULTS

: All items met the minimum acceptance criterion (≥0.70), and adjustments were made to incorporate a more inclusive approach and reflect the diversity of gender identities.

CONCLUSION

The study developed and validated a clinical simulation scenario for post-suicide attempt care for a person undergoing gender transition. The expert validation indicated agreement on the scenario's items, which can be applied in accessible training and qualification for healthcare professionals across different categories and contexts.

Use and application of geographical restrictions in systematic reviews with the aim of including studies about Germany: An update of a methodological review

BACKGROUND

In systematic reviews (SRs), geographical limitations in literature searches can aid in focussing research efforts. A methodological review published in 2016 examined the approaches SR authors use to identify studies about Germany, analysing 36 SRs.

OBJECTIVE

The aim of this study was to update the original review.

METHODS

We conducted a literature search on PubMed for SRs synthesising evidence from studies about Germany published between 22 January 2016 and 7 June 2022. Two reviewers independently performed study selection and data extraction. We evaluated the application of search syntax for restricting studies to those about Germany using the peer review of electronic search strategies criteria. The updated findings were reported and summarised alongside those of the original review.

RESULTS

Thirty-two additional SRs were newly included (total = 68). Geographic restrictions were applied in 57 SRs, representing 72% in the original review and increasing to 97% in the newly included SRs. Moreover, there was an increased use of truncations and field tags.

CONCLUSION

Although geographical restriction methods are increasingly utilised, additional tools are necessary to enhance the robustness of search strategies. The development of a dedicated geographical search filter would facilitate the identification of studies about Germany.

Serious adverse events reporting in recent randomised clinical trials in intensive care medicine - A methodological study protocol

BACKGROUND

Serious adverse events (SAEs) are common in intensive care unit (ICU) patients. Reporting of SAEs in randomised clinical trials (RCTs) varies why underreporting is likely. We aim to describe the reporting of SAEs from 2020 onwards and to illustrate the recent reporting of SAEs published in major medical journals.

METHODS

We will conduct a methodological study assessing pharmacological interventions in RCTs including adult ICU patients. We will search 10 general medical and critical care journals in PubMed. We will include all RCTs published from 2020 onwards. The primary research question is how many RCTs report SAEs in the primary publication. Secondary research questions include how SAEs are reported in the primary publication either as (1) proportion of patients experiencing one or more SAE, (2) all single events occurred, or (3) both strategies combined. We will assess the association between the proportion of patients with reported SAEs and the following trial characteristics: multicentred versus single-centre RCTs, industry-sponsored versus academic-sponsored, published trial protocol versus unpublished work, blinding, trials sample size, and RCTs focusing on COVID-19 patients versus other populations.

DISCUSSION

The outlined methodological study will provide important information on the reporting of SAEs in recent drug trials in adult ICU patients.

The quality of reporting in randomized controlled trials investigating exercise for individuals with whiplash-associated disorders; a systematic review

BACKGROUND

Whiplash-associated disorders are a common sequela of road traffic accidents. Exercise therapy is considered an effective intervention, and it is recommended for the management of such condition. However, the application of research findings to everyday clinical practice is dependent on sufficient details being reported.

OBJECTIVES

To explore the quality of reporting in studies investigating the effectiveness of exercise for whiplash-associated disorders.

METHODS

A literature search was conducted to identify studies testing the effectiveness of exercise for whiplash-associated disorders. Two reporting checklists were used to evaluate reporting completeness. The median positive scores for each study and overall percentage of positive scores for each item were calculated. Percentage agreement and the Cohen's Kappa coefficient were calculated.

RESULTS

Twenty-one studies were included. According to the Template for Intervention Description and Replication checklist, items were reported appropriately with a median of 29% (range 0-95%, IQR 40.5). The median number of adequately reported items per study was 5 (range 1-10, IQR 3). For the Consensus on Exercise Reporting Template checklist, items were reported appropriately with a median of 29% (range 0-57%, IQR 29). The median number of adequately reported items per study was 4 (range 0-16, IQR 8). Percentage agreement ranged from 57% to 100% while Cohen's Kappa from -0.17 to 1.00.

CONCLUSIONS

The study reveals significant gaps in the quality of reporting in studies investigating exercise for whiplash-associated disorders as both checklists showed a median reporting adequacy of only 29%. Overall, the inter-rater agreement for both checklists was acceptable.

Substantial discrepancies exist between registered protocol and published manuscript in trials on exercise interventions for chronic low back pain: a metaresearch study

OBJECTIVES

Reporting bias, prevalent in biomedical fields, can undermine evidence credibility. Our objective was to evaluate the proportion of discrepancies between registered protocols and published manuscripts in randomized controlled trials (RCTs) on exercise interventions for patients with chronic low back pain (CLBP).

STUDY DESIGN AND SETTING

We conducted a cross-sectional meta-research study, starting from the 2021 "Exercise therapy for CLBP" Cochrane Review. We selected all RCTs reporting a protocol registration on a primary register of the World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP) or in ClinicalTrials.gov. We extracted data from both registered protocol and published manuscript of RCTs, collecting recruitment and administrative information (eg, record dates) and details of trial characteristics (eg, outcomes, arms, statistical analysis plan details [SAPs]). Independent pairs of reviewers assessed discrepancies between registered protocol and published manuscript for the reporting of primary and secondary outcomes domains, measurement instruments, time-points, number of arms and SAPs(if attached). Outcome discrepancies were characterized as addition, omission, upgrade or downgrade.

RESULTS

We included 116 RCTs reporting an available protocol registration. Overall, 100 RCTs (86.2%) distinguished between primary and secondary outcomes. Of these, 39 RCTs (39.0%) reported one or more discrepancies in primary outcomes, and 78 RCTs (78.0%) reported one or more discrepancies in secondary outcomes. Focusing on discrepancies for the primary outcome, 64.5% of added, upgraded or downgraded outcomes favored statistically significant effects. Few RCTs (n = 6) reported discrepancies in the number of arms. SAPs were poorly reported in the registered protocols (n = 3) for being compared to the publications.

CONCLUSION

We found substantial outcome discrepancies comparing registered protocols and published manuscripts in RCTs assessing exercise interventions for patients with CLBP, with some impacting the statistical significance of the effects. Readers are encouraged to approach RCTs results in this field with caution.

Underserved groups could be better considered within population-based eye health surveys: a methodological study

OBJECTIVES

In pursuit of health equity, the World Health Organization has recently called for more extensive monitoring of inequalities in eye health. Population-based eye health surveys can provide this information, but whether underserved groups are considered in the design, implementation, and reporting of surveys is unknown. We conducted a systematic methodological review of surveys published since 2000 to examine how many population-based eye health surveys have considered underserved groups in their design, implementation, or reporting.

STUDY DESIGN AND SETTING

We identified all population-based cross-sectional surveys reporting the prevalence of objectively measured vision impairment or blindness. Using the PROGRESS + framework to identify underserved groups, we assessed whether each study considered underserved groups within 15 items across the rationale, sampling or recruitment methods, or the reporting of participation and prevalence rates.

RESULTS

388 eye health surveys were included in this review. Few studies prospectively considered underserved groups during study planning or implementation, for example within their sample size calculations (n = 5, ∼1%) or recruitment strategies (n = 70, 18%). The most common way that studies considered underserved groups was in the reporting of prevalence estimates (n = 374, 96%). We observed a modest increase in the number of distinct PROGRESS + factors considered by a publication over the study period. Gender/sex was considered within at least one item by 95% (n = 367) of studies. Forty-three percent (n = 166) of included studies were conducted primarily on underserved population groups, particularly for subnational studies of people living in rural areas, and we identified examples of robust population-based studies in socially excluded groups.

CONCLUSION

More effort is needed to improve the design, implementation, and reporting of surveys to monitor inequality and promote equity in eye health. Ideally, national-level monitoring of vision impairment and service coverage would be supplemented with smaller-scale studies to understand the disparities experienced by the most underserved groups.

The policies on the use of large language models in radiological journals are lacking: a meta-research study

OBJECTIVE

To evaluate whether and how the radiological journals present their policies on the use of large language models (LLMs), and identify the journal characteristic variables that are associated with the presence.

METHODS

In this meta-research study, we screened Journals from the Radiology, Nuclear Medicine and Medical Imaging Category, 2022 Journal Citation Reports, excluding journals in non-English languages and relevant documents unavailable. We assessed their LLM use policies: (1) whether the policy is present; (2) whether the policy for the authors, the reviewers, and the editors is present; and (3) whether the policy asks the author to report the usage of LLMs, the name of LLMs, the section that used LLMs, the role of LLMs, the verification of LLMs, and the potential influence of LLMs. The association between the presence of policies and journal characteristic variables was evaluated.

RESULTS

The LLM use policies were presented in 43.9% (83/189) of journals, and those for the authors, the reviewers, and the editor were presented in 43.4% (82/189), 29.6% (56/189) and 25.9% (49/189) of journals, respectively. Many journals mentioned the aspects of the usage (43.4%, 82/189), the name (34.9%, 66/189), the verification (33.3%, 63/189), and the role (31.7%, 60/189) of LLMs, while the potential influence of LLMs (4.2%, 8/189), and the section that used LLMs (1.6%, 3/189) were seldomly touched. The publisher is related to the presence of LLM use policies (p < 0.001).

CONCLUSION

The presence of LLM use policies is suboptimal in radiological journals. A reporting guideline is encouraged to facilitate reporting quality and transparency.

CRITICAL RELEVANCE STATEMENT

It may facilitate the quality and transparency of the use of LLMs in scientific writing if a shared complete reporting guideline is developed by stakeholders and then endorsed by journals.

KEY POINTS

The policies on LLM use in radiological journals are unexplored. Some of the radiological journals presented policies on LLM use. A shared complete reporting guideline for LLM use is desired.

The confidence in the results of physiotherapy systematic reviews in the musculoskeletal field is not increasing over time: a meta-epidemiological study using AMSTAR 2 tool

OBJECTIVES

To assess the confidence in the results of systematic reviews on the effectiveness of physiotherapy for musculoskeletal conditions in the past 10 years and to analyze trends and factors associated.

METHODS

This is a metaepidemiological study on systematic reviews (SRs) with meta-analysis of randomized controlled trials (RCTs). MEDLINE, Cochrane Database of Systematic Reviews, CINAHL, and PEDro were searched for SRs of RCT on physiotherapy interventions for musculoskeletal disorders from December 2012 to December 2022. Two researchers independently screened the records based on the inclusion criteria; a random sample of 100 studies was selected, and each journal, author, and study variable was extracted. The methodological quality of SRs was independently assessed with the AMSTAR 2 tool. Any disagreement was solved by consensus.

RESULTS

The confidence in SRs results was critically low in 90% of the studies, and it did not increase over time. Cochrane reviews are predominantly represented in the higher AMSTAR 2 confidence levels, with a statistically significant difference compared to non-Cochrane reviews. The last author's H-index is the only predictor of higher confidence among the variables analyzed (OR 1.04; 95% CI: 1.01, 1.06).

CONCLUSION

The confidence in SRs results is unacceptably low. Given the relevance of musculoskeletal disorders and the impact of evidence synthesis on the clinical decision-making process, there is an urgent need to improve the quality of secondary research by adopting more rigorous methods.

Instructional and didactic support tool for teaching-learning Post-Partum Haemorrhage care in simulated settings: Creation and validation

OBJECTIVE

To describe the creation and validation process of an instructional, didactic and self-applied support tool for teaching-learning Post-Partum Haemorrhage care in simulated settings.

BACKGROUND

Students frequently face difficulties performing the actions in the proper sequence in Post-Partum Haemorrhage simulated cases. Even in a controlled environment, anxiety, nervousness and fear of making mistakes are evident, which render the simulated experience highly stressful. Having a tool with a guideline can help students perform these actions more assertively.

DESIGN

A methodological study to develop a didactic tool.

METHODS

The creation process of the didactic tool, called Instructional Disk for the Management of Post-Partum Haemorrhage, was divided into five phases: Analysis, Design, Development, Implementation and Evaluation. Nine experts specialised in Obstetrics and Midwifery validated the tool by answering a questionnaire with nine items; in turn, 32 undergraduate Midwifery program students carried out the evaluation using a questionnaire on applicability, functionality, clarity, coherence and usability of the tool in Post-Partum Haemorrhage simulated stations. The data were analysed descriptively, considering absolute agreement when the answers to all questions in the five-point Likert scale corresponded to 5 (I totally agree). The Content Validity Index was calculated for the experts' questionnaires.

RESULTS

In the validation stage, the experts agreed or totally agreed with all nine items, reaching a Content Validity Index = 1. In the total scores assigned by the experts, there was a variation between 80.0% and 100% absolute agreement, with a mean of 95.6%. In the students' assessment, the variation was between 87.5% and 100% absolute agreement, with a mean of 97.7%, in the eight items evaluated. The agreement level above 90% among experts and students was considered high.

CONCLUSIONS

The Instructional Disk for the Management of Post-Partum Haemorrhage was validated by experts with extensive experience in Obstetrics and Midwifery care and teaching, ensuring that the content included in the guidelines for the management of Post-Partum Haemorrhage adopted in Brazil is covered. The students positively evaluated this support tool for learning the care to be provided in Post-Partum Haemorrhage cases in the simulated stations.

Reasons for missing evidence in rehabilitation meta-analyses: a cross-sectional meta-research study

BACKGROUND

Systematic reviews of randomized controlled trials are the best evidence for informing on intervention effectiveness. Their results, however, can be biased due to omitted evidence in the quantitative analyses. We aimed to assess the proportion of randomized controlled trials omitted from meta-analyses in the rehabilitation field and explore related reasons.

METHODS

This is a cross-sectional meta-research study. For each systematic review included in a published selected sample in the rehabilitation field, we identified an index meta-analysis on the primary outcome and the main comparison. We then looked at all the studies considered eligible for the chosen comparison in the systematic review and identified those trials that have been omitted (i.e., not included) from each index meta-analysis. Reasons for omission were collected based on an eight-reason classification. We used descriptive statistics to describe the proportion of omitted trials overall and according to each reason.

RESULTS

Starting from a cohort of 827 systematic reviews, 131 index meta-analyses comprising a total of 1761 eligible trials were selected. Only 16 index meta-analyses included all eligible studies while 15 omitted studies without providing references. From the remaining 100 index meta-analyses, 717 trials (40,7%) were omitted overall. Specific reasons for omission were: "unable to distinguish between selective reporting and inadequate planning" (39,3%, N = 282), "inadequate planning" (17%, N = 122), "justified to be not included" (15,1%, N = 108), "incomplete reporting" (8,4%, N = 60), "selective reporting" (3,3%, N = 24) and other situations (e.g., outcome present but no motivation for omission) (5,2%, N = 37). The 11,7% (N = 84) of omitted trials were not assessed due to non-English language or full text not available.

CONCLUSIONS

Almost half of the eligible trials were omitted from their index meta-analyses. Better reporting, protocol registration, definition and adoption of core outcome sets are needed to prevent omission of evidence in systematic reviews.

Racial and Ethnic Representation in Preventive Intervention Research: a Methodological Study

Individuals who are Asian or Asian American, Black or African American, Native American or American Indian or Alaska Native, Native Hawaiian or Pacific Islander, and Hispanic or Latino (i.e., presently considered racial ethnic minoritized groups in the USA) lacked equal access to resources for mitigating risk during COVID-19, which highlighted public health disparities and exacerbated inequities rooted in structural racism that have contributed to many injustices, such as failing public school systems and unsafe neighborhoods. Minoritized groups are also vulnerable to climate change wherein the most severe harms disproportionately fall upon underserved communities. While systemic changes are needed to address these pervasive syndemic conditions, immediate efforts involve examining strategies to promote equitable health and well-being-which served as the impetus for this study. We conducted a descriptive analysis on the prevalence of culturally tailored interventions and reporting of sample characteristics among 885 programs with evaluations published from 2010 to 2021 and recorded in the Blueprints for Healthy Youth Development registry. Inferential analyses also examined (1) reporting time trends and (2) the relationship between study quality (i.e., strong methods, beneficial effects) and culturally tailored programs and racial ethnic enrollment. Two percent of programs were developed for Black or African American youth, and 4% targeted Hispanic or Latino populations. For the 77% of studies that reported race, most enrollees were White (35%) followed by Black or African American (28%), and 31% collapsed across race or categorized race with ethnicity. In the 64% of studies that reported ethnicity, 32% of enrollees were Hispanic or Latino. Reporting has not improved, and there was no relationship between high-quality studies and programs developed for racial ethnic youth, or samples with high proportions of racial ethnic enrollees. Research gaps on racial ethnic groups call for clear reporting and better representation to reduce disparities and improve the utility of interventions.

Using evidence to decision frameworks led to guidelines of better quality and more credible and transparent recommendations

BACKGROUND AND OBJECTIVES

To determine whether the use of Evidence to Decision (EtD) frameworks is associated to higher quality of both guidelines and individual recommendations.

METHODS

We identified guidelines recently published by international organizations that have methodological guidance documents for their development. Pairs of researchers independently extracted information on the use of these frameworks, appraised the quality of the guidelines using the Appraisal of Guidelines, Research and Evaluation II Instrument (AGREE-II), and assessed the clinical credibility and implementability of the recommendations with the Appraisal of Guidelines for REsearch & Evaluation Recommendations Excellence (AGREE-REX) tool. We conducted both descriptive and inferential analyses.

RESULTS

We included 66 guidelines from 17 different countries, published in the last 5 years. Thirty guidelines (45%) used an EtD framework to formulate their recommendations. Compared to those that did not use a framework, those using an EtD framework scored higher in all domains of both AGREE-II and AGREE-REX (P < 0.05). Quality scores did not differ between the use of the The Grading of Recommendations Assessment, Development and Evaluation-EtD framework (17 guidelines) or another EtD framework (13 guidelines) (P > 0.05).

CONCLUSION

The use of EtD frameworks is associated with guidelines of better quality, and more credible and transparent recommendations. Endorsement of EtD frameworks by guideline developing organizations will likely increase the quality of their guidelines.

REPCAN: Guideline for REporting Population-based CANcer Registry Data

Background: The objective of this study was to develop a guideline on how to report result of a population-based cancer registry. Methods: The guideline's development involved a core working committee and a scientific committee comprising experts from diverse domains. The process comprised three steps: 1) a comprehensive review of existing tools and guidelines and the development of the initial draft of the guideline based on a review of literature, 2) refinement items through several rounds of focus group discussion among the core group, and development initial draft, and 3) Evaluation of the initial draft by scientific committee members. Items in the guideline were organized to accommodate reports of population-based cancer registries as a scientific manuscript. Results: The core committee developed 47 items distributed in the major heading of a scientific manuscript presented as a checklist. The evaluation of the scientific committee led to a consensus on the majority of the items included in the checklist. Among 10 committee members, 7 provided unreserved approval, validating each item's necessity, applicability, and comprehensibility in the checklist. Feedback from the remaining 3 members was carefully analyzed and integrated to enhance the guideline's robustness. Incorporating feedback, a first final draft was presented in a meeting of scientific and core working committee members. Collaborative discussion ensured clarity of expression for each items and a final checklist was developed. Conclusion: The guideline abbreviated as REPCAN offers a standardized framework for reporting population-based cancer registry, fostering transparency, comparability, and comprehensive data presentation. The guideline encourages flexibility while promoting comprehensive and robust reporting practices.

Evaluating the Methodological Approaches of Cross-Cultural Adaptation of the Bedside Handover Attitudes and Behaviours Questionnaire into Portuguese

Nurse managers need culturally adapted assessment instruments to support the implementation of change to Nursing Bedside Handover (NBH) in healthcare institutions. This study aimed to cross-culturally adapt the Bedside Handover Attitudes and Behaviours (BHAB) questionnaire to the Portuguese context and evaluate the methodological approaches used for this purpose. To guide this study, we followed a guideline for cross-cultural translation and adaptation measurement instruments in healthcare. The results of the content validity testing suggested that the BHAB questionnaire is a valid instrument for use in the Portuguese context. To obtain these results we showed 1) using of a new methodological approach, the dual focus, to resolve the divergences and ambiguities in the translators' committee and the multi-professional committee; 2) the lack of a conceptual definition of the construct of the instrument as a requirement to retain items with I-CVI <0.70 after validity relevance pretesting and 3) the cognitive debriefing and relevance pretesting as methodological approaches which can be used alone or together to reinforce the evaluation of cultural relevance of the items. We concluded there is a need for guidelines to support the decision-making process of healthcare researchers with comprehensive information about the different methodological approaches they can follow.

Methods and results of studies on reporting guideline adherence are poorly reported: a meta-research study

OBJECTIVES

We investigated recent meta-research studies on adherence to four reporting guidelines to determine the proportion that provided (1) an explanation for how adherence to guideline items was rated and (2) results from all included individual studies. We examined conclusions of each meta-research study to evaluate possible repetitive and similar findings.

STUDY DESIGN AND SETTING

A cross-sectional meta-research study. MEDLINE (Ovid) was searched on July 5, 2022 for studies that used any version of the Consolidated Standards of Reporting Trials, Preferred Reporting Items for Systematic Reviews and Meta-Analyses, Standards for the Reporting of Diagnostic Accuracy Studies, or Strengthening the Reporting of Observational Studies in Epidemiology reporting guidelines or their extensions to evaluate reporting.

RESULTS

Of 148 included meta-research studies published between August 2020 and June 2022, 14 (10%, 95% confidence interval [CI] 6%-15%) provided a fully replicable explanation of how they coded the adherence ratings and 49 (33%, 95% CI 26%-41%) completely reported individual study results. Of 90 studies that classified reporting as adequate or inadequate in the study abstract, six (7%, 95% CI 3%-14%) concluded that reporting was adequate, but none of those six studies provided information on how items were coded or provided item-level results for included studies.

CONCLUSION

Almost all included meta-research studies found that reporting in health research is suboptimal. However, few of these reported enough information for verification or replication.

Observed intervention effects for mortality in randomised clinical trials: a methodological study protocol

INTRODUCTION

It is essential to choose a realistic anticipated intervention effect when calculating a sample size for a randomised clinical trial. Unfortunately, anticipated intervention effects are often inflated, when compared with the 'true' intervention effects. This is documented for mortality in critical care trials. A similar pattern might exist across different medical specialties. This study aims to estimate the range of observed intervention effects for all-cause mortality in trials included in Cochrane Reviews, within each Cochrane Review Group.

METHODS AND ANALYSIS

We will include randomised clinical trials assessing all-cause mortality as an outcome. Trials will be identified from Cochrane Reviews published in the Cochrane Database of Systematic Reviews. Cochrane Reviews will be clustered according to the registered Cochrane Review Group (eg, Anaesthesia, Emergency and Critical Care) and the statistical analyses will be conducted for each Cochrane Review Group and overall. The median relative risk and IQR for all-cause mortality and the proportion of trials with a relative all-cause mortality risk within seven different ranges will be reported (relative risk below 0.70, 0.70-0.79, 0.80-0.89, 0.90-1.09, 1.10-1.19, 1.20-1.30 and above 1.30). Subgroup analyses will explore the effects of original design, sample size, risk of bias, disease, intervention type, follow-up length, participating centres, funding type, information size and outcome hierarchy.

ETHICS AND DISSEMINATION

Since we will use summary data from trials already approved by relevant ethical committees, this study does not require ethical approval. Regardless of our findings, the results will be published in an international peer-reviewed journal.

The interpretation of clinical relevance in randomised clinical trials in patients with chronic low back pain: protocol for a meta-research study

Chronic low back pain (cLBP) is one of the leading worldwide causes of disability. The smallest worthwhile effect (SWE) parameter has been proposed to find a threshold of clinical relevance. Specific values of the SWE have been calculated in patients with cLBP for pain intensity, physical functioning and time to recovery for physiotherapy compared with no intervention. Our objectives are 1) To evaluate how authors have interpreted the clinical relevance of the effect of physiotherapy compared to no-intervention on pain, physical functioning and time to recovery; 2) To reinterpret the clinical relevance of these between-group differences based on the available SWE estimates; 3) To evaluate, for descriptive purposes, whether the studies are adequately powered or underpowered considering the published SWE values and a power threshold of 80%. A systematic search in Medline, PEDro, Embase and Cochrane CENTRAL will be conducted. We will search for RCT investigating the effectiveness of physiotherapy as compared to no interventions in people with cLBP. We will compare the authors' interpretation of results for clinical relevance with their results to determine if they meet their a-priori definitions. Then, we will perform a re-interpretation of the between-group differences based on SWE values published for cLBP.

Adherence to the PRISMA statement and its association with risk of bias in systematic reviews published in rehabilitation journals: A meta-research study

BACKGROUND

Systematic reviews (SRs) and meta-analyses are essential resources for the clinicians. They allow to evaluate the strengths and the weaknesses of the evidence to support clinical decision-making if they are adequately reported. Little is known in the rehabilitation field about the completeness of reporting of SRs and its relationship with the risk of bias (ROB).

OBJECTIVES

Primary: 1) To evaluate the completeness of reporting of systematic reviews (SRs) published in rehabilitation journals by evaluating their adherence to the PRISMA 2009 checklist, 2) To investigate the relationship between ROB and completeness of reporting. Secondary: To study the association between completeness of reporting and journals and study characteristics.

METHODS

A random sample of 200 SRs published between 2011 and 2020 in 68 rehabilitation journals was indexed under the "rehabilitation" category in the InCites database. Two independent reviewers evaluated adherence to the PRISMA checklist and assessed ROB using the ROBIS tool. Overall adherence and adherence to each PRISMA item and section were calculated. Regression analyses investigated the association between completeness of reporting, ROB, and other characteristics (impact factor, publication options, publication year, and study protocol registration).

RESULTS

The mean overall PRISMA adherence across the 200 studies considered was 61.4%. Regression analyses show that having a high overall ROB is a significant predictor of lower adherence (B=-7.1%; 95%CI -12.1, -2.0). Studies published in fourth quartile journals displayed a lower overall adherence (B= -7.2%; 95%CI -13.2, -1.3) than those published in first quartile journals; the overall adherence increased (B= 11.9%; 95%CI 5.9, 18.0) if the SR protocol was registered. No association between adherence, publication options, and publication year was found.

CONCLUSION

Reporting completeness in rehabilitation SRs is suboptimal and is associated with ROB, impact factor, and study registration. Authors of SRs should improve adherence to the PRISMA guideline, and journal editors should implement strategies to optimize the completeness of reporting.

Cost-Effectiveness of HIV Retention and Re-engagement Interventions in High-Income Countries: A Systematic Literature Review

Engagement in lifelong HIV care is critical for both patient and public health, yet there are limited resources to invest in improving HIV outcomes. We systematically reviewed evidence on the cost-effectiveness of retention and re-engagement interventions. We searched five databases for peer-reviewed studies published between 2010 and 2020. We assessed reporting and methods quality, extracted data on target populations, interventions, and cost-effectiveness, and evaluated overall strength of evidence. Eleven studies met inclusion criteria, and eight had moderate-high quality. Cost-effectiveness estimates ranged from cost-saving to over $1,000,000/quality-adjusted life year (QALY) gained. Of the 73 cost-effectiveness ratios reported, 64% were < $100,000/QALY gained. Interventions were more likely to be cost-effective when targeted to high-risk groups, implemented in locations where baseline retention levels were low, and when used in combination with other high-impact HIV interventions (such as prevention). Overall, existing evidence is moderately strong that retention and/or re-engagement interventions can be cost-effective in high-income countries.

Completeness of exercise reporting among randomized controlled trials on pelvic floor muscle training for women with pelvic organ prolapse: A systematic review

AIMS

This systematic review aimed to assess the completeness of exercise reporting in randomized controlled trials (RCTs) on pelvic floor muscle training (PFMT) for women with pelvic organ prolapse (POP).

METHODS

MEDLINE, Cochrane Central, CINHAL, Embase, SCOPUS, and PEDro databases were searched up to October 2020. Full-text RCTs comparing PFMT to any type of intervention among women with any type and stage of POP were eligible for inclusion. Completeness of intervention was evaluated with t20he template for intervention description and replication (TIDieR) and the consensus on exercise reporting template (CERT). Inter-rater agreement for each item of the tools was calculated.

RESULTS

Twenty-six RCTs were included. None of the studies completely reported all intervention descriptors. On average 57.1% (6.8 ± 2.4; out of 12) of the overall TIDieR items and 35.3% (6.7 ± 2.9; out of 19) of the CERT were well described. In particular, 7 and 5 items were completely reported more than 50% of the time for the TIDieR and CERT, respectively. Frequent shortcomings were the undetailed reporting of information regarding tailoring and modifications of exercises and their adherence. Detailed descriptions of exercise repetitions to enable replication were missing in 53.8%. According to the CERT, only 11.5% of the RCTs sufficiently described the main providers' characteristics.

CONCLUSION

The completeness of PFMT reporting for women with POP is still below desirable standards and it is insufficient to ensure transferability into practice. The present results may add relevant knowledge and contribute to improving adequate reporting of exercise.