Common methodological pitfalls in ICI pneumonitis risk prediction studies

Background

Pneumonitis is one of the most common adverse events induced by the use of immune checkpoint inhibitors (ICI), accounting for a 20% of all ICI-associated deaths. Despite numerous efforts to identify risk factors and develop predictive models, there is no clinically deployed risk prediction model for patient risk stratification or for guiding subsequent monitoring. We believe this is due to systemic suboptimal approaches in study designs and methodologies in the literature. The nature and prevalence of different methodological approaches has not been thoroughly examined in prior systematic reviews.

Methods

The PubMed, medRxiv and bioRxiv databases were used to identify studies that aimed at risk factor discovery and/or risk prediction model development for ICI-induced pneumonitis (ICI pneumonitis). Studies were then analysed to identify common methodological pitfalls and their contribution to the risk of bias, assessed using the QUIPS and PROBAST tools.

Results

There were 51 manuscripts eligible for the review, with Japan-based studies over-represented, being nearly half (24/51) of all papers considered. Only 2/51 studies had a low risk of bias overall. Common bias-inducing practices included unclear diagnostic method or potential misdiagnosis, lack of multiple testing correction, the use of univariate analysis for selecting features for multivariable analysis, discretization of continuous variables, and inappropriate handling of missing values. Results from the risk model development studies were also likely to have been overoptimistic due to lack of holdout sets.

Conclusions

Studies with low risk of bias in their methodology are lacking in the existing literature. High-quality risk factor identification and risk model development studies are urgently required by the community to give the best chance of them progressing into a clinically deployable risk prediction model. Recommendations and alternative approaches for reducing the risk of bias were also discussed to guide future studies.

Stem Cells as Therapy for Necrotizing Enterocolitis: A Systematic Review and Meta-Analysis of Preclinical Studies

Background: Necrotizing enterocolitis (NEC) is the most common life-threatening gastrointestinal condition among very and extremely preterm infants. Stem cell therapy has shown some promising protective effects in animal models of intestinal injury, including NEC, but no systematic review has yet evaluated the preclinical evidence of stem cell therapy for NEC prevention or treatment. Methods: PubMed and EMBASE databases were searched for studies using an animal model of NEC with stem cells or their products. The SYRCLE tool was used for the assessment of risk of bias. A random-effects model was used to pool odds ratios (ORs) and 95% confidence interval (CI). Results: We screened 953 studies, of which nine (eight rat and one mouse models) met the inclusion criteria. All animal models induced NEC by a combination of hypothermia, hypoxia, and formula feeding. Risk of bias was evaluated as unclear on most items for all studies included. Meta-analysis found that both mesenchymal and neural stem cells and stem cell-derived exosomes reduced the incidence of all NEC (OR 0.22, 95% CI 0.16-0.32, k = 16), grade 2 NEC (OR 0.41, 95% CI 0.24-0.70, k = 16), and grade 3-4 NEC (OR 0.28, 95% CI 0.19-0.42, k = 16). k represents the number of independent effect sizes included in each meta-analysis. The effect of the exosomes was similar to that of the stem cells. Stem cells and exosomes also improved 4-day survival (OR 2.89 95% CI 2.07-4.04, k = 9) and 7-day survival (OR 3.96 95% CI 2.39-6.55, k = 5) after experimental NEC. Meta-analysis also found that stem cells reduced other indicators of intestinal injury. Conclusion: The data from this meta-analysis suggest that both stem cells and stem cell-derived exosomes prevented NEC in rodent experimental models. However, unclear risk of bias and incomplete reporting underline that poor reporting standards are common and hamper the reliable interpretation of preclinical evidence for stem cell therapy for NEC.

The Synaptic Scaling Literature: A Systematic Review of Methodologies and Quality of Reporting

The maintenance of the excitability of neurons and circuits is a fundamental process for healthy brain functions. One of the main homeostatic mechanisms responsible for such regulation is synaptic scaling. While this type of plasticity is well-characterized through a robust body of literature, there are no systematic evaluations of the methodological and reporting features from these studies. Our review yielded 168 articles directly investigating synaptic scaling mechanisms, which display relatively high impact, with a median impact factor of 7.76 for the publishing journals. Our methodological analysis identified that 86% of the articles made use of inhibitory interventions to induce synaptic scaling, while only 41% of those studies contain excitatory manipulations. To verify the effects of synaptic scaling, the most assessed outcome was miniature excitatory postsynaptic current (mEPSC) recordings, performed in 71% of the articles. We could also observe that the field is mostly focused on mechanistic studies of the synaptic scaling pathways (70%), rather than the interaction with other types of plasticity, such as Hebbian processes (4%). We found that more than half of the articles failed to describe simple features, such as regulatory compliance statements, ethics committee approval, or statements of conflict of interests. In light of these results, we discuss the strengths and pitfalls existing in synaptic scaling literature.

Use of rituximab in adults with relapsing-remitting multiple sclerosis: a systematic literature review

Objective: Rituximab is used as an off-label treatment for relapsing-remitting multiple sclerosis (RRMS); however, the comparative efficacy and safety of rituximab versus currently licensed disease-modifying drugs (DMDs) for RRMS is unknown. A systematic literature review was conducted to evaluate the available data pertaining to efficacy and safety of rituximab in adult patients with RRMS and highly active relapsing multiple sclerosis (HA-RMS); data quality was critically assessed via risk of bias (RoB) assessment.Methods: Biomedical literature databases were searched until mid-2018 and key proceedings were searched from 2016 to 2018. Critical appraisal of non-randomized studies was conducted using the Cochrane RoB assessment tool; randomized controlled trials (RCTs) were appraised using comprehensive assessment criteria based on the NICE guidelines.Results: Thirty-eight unique studies based on 49 publications were identified: 25 RRMS studies (one RCT) and 13 HA-RMS studies (no RCTs). The evidence among patients with RRMS generally favored rituximab in comparison to placebo (relapse rate) and interferons/glatiramer acetate (relapse rate and disability progression), although much of the non-randomized data were descriptive and/or not statistically significant. In comparison to placebo, rituximab was associated with a greater risk of adverse events. Two-thirds of the non-randomized RRMS studies were associated with critical/serious RoB; the single RCT was associated with low RoB. Furthermore, all of the non-randomized HA-RMS studies were associated with critical/serious RoB.Conclusions: Available evidence of off-label rituximab use for the treatment of patients with RRMS suggests generally favorable efficacy versus placebo and interferons/glatiramer acetate; however, the poor quality of the included studies limits any robust conclusions.

Practical guidance for using multiple data sources in systematic reviews and meta-analyses (with examples from the MUDS study)

Data for individual trials included in systematic reviews may be available in multiple sources. For example, a single trial might be reported in 2 journal articles and 3 conference abstracts. Because of differences across sources, source selection can influence the results of systematic reviews. We used our experience in the Multiple Data Sources in Systematic Reviews (MUDS) study, and evidence from previous studies, to develop practical guidance for using multiple data sources in systematic reviews. We recommend the following: (1) Specify which sources you will use. Before beginning a systematic review, consider which sources are likely to contain the most useful data. Try to identify all relevant reports and to extract information from the most reliable sources. (2) Link individual trials with multiple sources. Write to authors to determine which sources are likely related to the same trials. Use a modified Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) flowchart to document both the selection of trials and the selection of sources. (3) Follow a prespecified protocol for extracting trial characteristics from multiple sources. Identify differences among sources, and contact study authors to resolve differences if possible. (4) Prespecify outcomes and results to examine in the review and meta-analysis. In your protocol, describe how you will handle multiple outcomes within each domain of interest. Look for outcomes using all eligible sources. (5) Identify which data sources were included in the review. Consider whether the results might have been influenced by data sources used. (6) To reduce bias, and to reduce research waste, share the data used in your review.

A systematic review of Chinese randomized clinical trials of SSRI treatment of depression

BACKGROUND

Selective serotonin reuptake inhibitors (SSRIs) have become the most frequently used antidepressants in China in recent decades. This systematic review and meta-analysis examined the efficacy and tolerability of SSRIs in Chinese studies and the quality of Chinese randomized controlled trials.

METHODS

Major Western and Chinese electronic databases were searched for double-blind, parallel group randomised controlled trials (RCTs) comparing SSRIs (fluoxetine, citalopram, escitalopram, fluvoxamine, paroxetine, or sertraline) with other antidepressants such as SSRI, Selective Noradrenaline Reuptake Inhibitor (SNRI), tricyclic antidepressant (TCA), Traditional Chinese Medicine (TCM) and/or placebo. Response, remission, and dropout rates due to side effects were defined as primary outcomes. Mean total Hamilton Rating Scale of Depression (HAMD) scores at endpoint, overall dropout rates and total Treatment Emergent Symptom Scale (TESS) scores were defined as secondary outcomes. Data were combined with random effects models. Risk of bias was assessed by the Cochrane evaluation tool. Quality of reports was assessed by the fulfilment of Consolidated Standards of Reporting Trial (CONSORT) items.

RESULTS

A total of 71 studies were included. Only one study was listed in both Chinese and Western databases. SSRIs were found to be more effective than TCAs. No significant differences were observed regarding dropout rates due to side effects. Using the Cochrane risk of bias tool, adequate methods of sequence generation were described in 16 (23%) studies. All authors failed to report trial registration. Informed consent, sources of funding, email address, protocol, and limitations were also not mentioned in most studies. However, reporting quality improved steadily between 1996 and 2013.

CONCLUSIONS

In light of the low trial quality, the findings of a significant advantage of SSRI over TCA in terms of response rate and remission rate should be replicated by large high-quality Chinese studies.