Current practice in analysing and reporting binary outcome data-a review of randomised controlled trial reports

Optimal reconstruction methods after distal gastrectomy for gastric cancer: a protocol for a systematic review and network meta-analysis update

BACKGROUND

Distal gastrectomy (DG) is a commonly used surgical procedure for gastric cancer (GC), with three reconstruction methods available: Billroth I, Billroth II, and Roux-en-Y. In 2018, our team published a systematic review to provide guidance for clinical practice on the optimal reconstruction method after DG for GC. However, since then, new evidence from several randomized controlled trials (RCTs) has emerged, prompting us to conduct an updated systematic review and network meta-analysis to provide the latest comparative estimates of the efficacy and safety of the three reconstruction methods after DG for GC.

METHOD

This systematic review and network meta-analysis update followed the PRISMA-P guidelines and will include a search of PubMed, Embase, and the Cochrane Library for RCTs comparing the outcomes of Billroth I, Billroth II, or Roux-en-Y reconstruction after DG for patients with GC. Two independent reviewers will screen the titles and abstracts based on predefined eligibility criteria, and two reviewers will assess the full texts of relevant studies. The Bayesian network meta-analysis will evaluate various outcomes, including quality of life after surgery, anastomotic leakage within 30 days after surgery, operation time, intraoperative blood loss, major postoperative complications within 30 days after surgery, incidence and severity of bile reflux, and loss of body weight from baseline.

ETHICS AND DISSEMINATION

The review does not require ethical approval. The findings of the review will be disseminated through publication in an academic journal, presentations at conferences, and various media outlets.

INPLASY REGISTRATION NUMBER

INPLASY2021100060.

Statistical inference and effect measures in abstracts of randomized controlled trials, 1975-2021. A systematic review

OBJECTIVE

To examine the time trend of statistical inference, statistical reporting style of results, and effect measures from the abstracts of randomized controlled trials (RCTs).

STUDY DESGIN AND SETTINGS

We downloaded 385,867 PubMed abstracts of RCTs from 1975 to 2021. We used text-mining to detect reporting of statistical inference (p-values, confidence intervals, significance terminology), statistical reporting style of results, and effect measures for binary outcomes, including time-to-event measures. We validated the text mining algorithms by random samples of abstracts.

RESULTS

A total of 320 676 abstracts contained statistical inference. The percentage of abstracts including statistical inference increased from 65% (1975) to 87% (2006) and then decreased slightly. From 1975 to 1990, the sole reporting of language regarding statistical significance was predominant. Since 1990, reporting of p-values without confidence intervals has been the most common reporting style. Reporting of confidence intervals increased from 0.5% (1975) to 29% (2021). The two most common effect measures for binary outcomes were hazard ratios and odds ratios. Number needed to treat and number needed to harm are reported in less than 5% of abstracts with binary endpoints.

CONCLUSIONS

Reporting of statistical inference in abstracts of RCTs has increased over time. Increasingly, p-values and confidence intervals are reported rather than just mentioning the presence of "statistical significance". The reporting of odds ratios comes with the liability that the untrained reader will interpret them as risk ratios, which is often not justified, especially in RCTs.

Effect of 13-valent pneumococcal conjugate vaccine on experimental carriage of Streptococcus pneumoniae serotype 6B in Blantyre, Malawi: a randomised controlled trial and controlled human infection study

BACKGROUND

The effect of childhood pneumococcal conjugate vaccine implementation in Malawi is threatened by absence of herd effect. There is persistent vaccine-type pneumococcal carriage in both vaccinated children and the wider community. We aimed to use a human infection study to measure 13-valent pneumococcal conjugate vaccine (PCV13) efficacy against pneumococcal carriage.

METHODS

We did a double-blind, parallel-arm, randomised controlled trial investigating the efficacy of PCV13 or placebo against experimental pneumococcal carriage of Streptococcus pneumoniae serotype 6B (strain BHN418) among healthy adults (aged 18-40 years) from Blantyre, Malawi. We randomly assigned participants (1:1) to receive PCV13 or placebo. PCV13 and placebo doses were prepared by an unmasked pharmacist to maintain research team and participant masking with identification only by a randomisation identification number and barcode. 4 weeks after receiving either PCV13 or placebo, participants were challenged with 20 000 colony forming units (CFUs) per naris, 80 000 CFUs per naris, or 160 000 CFUs per naris by intranasal inoculation. The primary endpoint was experimental pneumococcal carriage, established by culture of nasal wash at 2, 7, and 14 days. Vaccine efficacy was estimated per protocol by means of a log-binomial model adjusting for inoculation dose. The trial is registered with the Pan African Clinical Trials Registry, PACTR202008503507113, and is now closed.

FINDINGS

Recruitment commenced on April 27, 2021 and the final visit was completed on Sept 12, 2022. 204 participants completed the study protocol (98 PCV13, 106 placebo). There were lower carriage rates in the vaccine group at all three inoculation doses (0 of 21 vs two [11%] of 19 at 20 000 CFUs per naris; six [18%] of 33 vs 12 [29%] of 41 at 80 000 CFUs per naris, and four [9%] of 44 vs 16 [35%] of 46 at 160 000 CFUs per naris). The overall carriage rate was lower in the vaccine group compared with the placebo group (ten [10%] of 98 vs 30 [28%] of 106; Fisher's p value=0·0013) and the vaccine efficacy against carriage was estimated at 62·4% (95% CI 27·7-80·4). There were no severe adverse events related to vaccination or inoculation of pneumococci.

INTERPRETATION

This is, to our knowledge, the first human challenge study to test the efficacy of a pneumococcal vaccine against pneumococcal carriage in Africa, which can now be used to establish vaccine-induced correlates of protection and compare alternative strategies to prevent pneumococcal carriage. This powerful tool could lead to new means to enhance reduction in pneumococcal carriage after vaccination.

FUNDING

Wellcome Trust.

A systematic review and meta-analysis of predictive and prognostic models for outcome prediction using positron emission tomography radiomics in head and neck squamous cell carcinoma patients

BACKGROUND

Positron emission tomography (PET) images of head and neck squamous cell carcinoma (HNSCC) patients can assess the functional and biochemical processes at cellular levels. Therefore, PET radiomics-based prediction and prognostic models have the potentials to understand tumour heterogeneity and assist clinicians with diagnosis, prognosis and management of the disease. We conducted a systematic review of published modelling information to evaluate the usefulness of PET radiomics in the prediction and prognosis of HNSCC patients.

METHODS

We searched bibliographic databases (MEDLINE, Embase, Web of Science) from 2010 to 2021 and considered 31 studies with pre-defined inclusion criteria. We followed the CHARMS checklist for data extraction and performed quality assessment using the PROBAST tool. We conducted a meta-analysis to estimate the accuracy of the prediction and prognostic models using the diagnostic odds ratio (DOR) and average C-statistic, respectively.

RESULTS

Manual segmentation method followed by 40% of the maximum standardised uptake value (SUVmax ) thresholding is a commonly used approach. The area under the receiver operating curves of externally validated prediction models ranged between 0.60-0.87, 0.65-0.86 and 0.62-0.75 for overall survival, distant metastasis and recurrence, respectively. Most studies highlighted an overall high risk of bias (outcome definition, statistical methodologies and external validation of models) and high unclear concern in terms of applicability. The meta-analysis showed the estimated pooled DOR of 6.75 (95% CI: 4.45, 10.23) for prediction models and the C-statistic of 0.71 (95% CI: 0.67, 0.74) for prognostic models.

CONCLUSIONS

Both prediction and prognostic models using clinical variables and PET radiomics demonstrated reliable accuracy for detecting adverse outcomes in HNSCC, suggesting the prospect of PET radiomics in clinical settings for diagnosis, prognosis and management of HNSCC patients. Future studies of prediction and prognostic models should emphasise the quality of reporting, external model validation, generalisability to real clinical scenarios and enhanced reproducibility of results.

Congruence couple therapy for alcohol use and gambling disorders with comorbidities (part I): Outcomes from a randomized controlled trial

A nonblinded randomized trial was conducted at two Canadian provincial outpatient addiction clinics that tested the effectiveness of a systemic congruence couple therapy (CCT) versus individual-based treatment-as-usual (TAU) on nine clinical outcomes: (1) primary outcomes-alcohol use and gambling, psychiatric symptoms, and couple adjustment; (2) secondary outcomes-emotion regulation, substance use, depression, post-traumatic stress symptoms, and life stress. Data of primary clients and partners (N = 46) were analyzed longitudinally across baseline, posttreatment (5 months), and follow-up (8 months). Alcohol use disorder (95%) and gambling disorder (5%) were in the severe range at baseline, and co-addiction was 27%. Psychiatric comorbidity was 100%, and 18% of couples were jointly addicted. Between-group comparison favored CCT in primary outcomes with medium-to-large effect sizes (Cohen's h = 0.74-1.44). Secondary outcomes were also significantly stronger for CCT (Cohen's h = 0.27-1.53). Within-group, for all primary outcomes, a significant proportion of symptomatic CCT clients and partners improved, converging with ANOVA results of large effect sizes (0.14-0.29). All secondary outcomes improved significantly in CCT with large effect sizes (0.14-0.50). TAU showed significant within-group improvement in alcohol use, other substance use, and life stress with large effect sizes (0.16-0.40). Primary clients and partners made largely equivalent improvement within CCT and within TAU. Results were triangulated with clients' satisfaction ratings and counselors' reports. Overall, significant within-group effects were detected for CCT both clinically and statistically and between-group difference favored CCT. Future trials are required to validate these promising findings.

Assessing the robustness of negative vascular surgery randomized controlled trials using their reverse fragility index

OBJECTIVE

The reverse fragility index (RFI) describes the number of event conversions needed to convert a statistically nonsignificant dichotomous outcome to a significant one. The objective of the present study was to assess the RFI of vascular surgery randomized controlled trials (RCTs) comparing endovascular vs open surgery for the treatment of abdominal aortic aneurysms (AAAs), carotid artery stenosis (CAS), and peripheral artery disease (PAD).

METHODS

MEDLINE and Embase were searched for RCTs that had investigated AAAs, CAS, or PAD with statistically nonsignificant binary primary outcomes. The primary outcome for the present study was the median RFI. Calculation of the RFI was performed by creating two-by-two contingency tables and subtracting events from the group with fewer events and adding nonevents to the same group until a two-tailed Fisher exact test had produced a statistically significant result (P ≤ .05).

RESULTS

Of 4187 reports, 49 studies reporting 103 different primary end points were included. The overall median RFI was 7 (interquartile range [IQR], 5-13). The specific RFIs for AAA, CAS, and PAD were 10 (IQR, 6-15.5), 6 (IQR, 5-9.5), and 7 (IQR, 5.5-10), respectively. Of the 103 end points, 42 (47%) had had a loss to follow-up greater than the RFI, of which 10 were AAA trials (24%), 23 were CAS trials (55%), and 9 were PAD trials (21%). The Pearson correlation demonstrated a significant positive relationship between a study's RFI and the impact factor of its publishing journal (r = 0.38; 95% confidence interval [CI], 0.20-0.54; P < .01), length of follow-up (r = 0.43; 95% CI, 0.26-0.58; P < .01), and sample size (r = 0.28; 95% CI, 0.09-0.45; P < .01).

CONCLUSIONS

A small number of events (median, 7) was required to change the outcome of negative RCTs from statistically nonsignificant to significant, with 47% of the studies having missing data that could have reversed the finding of its primary outcome. Reporting of the RFI relative to the loss to follow-up could be of benefit in future trials and provide confidence regarding the robustness of the P value.

The reporting and handling of missing data in longitudinal studies of older adults is suboptimal: a methodological survey of geriatric journals

Background

Missing data are common in longitudinal studies, and more so, in studies of older adults, who are susceptible to health and functional decline that limit completion of assessments. We assessed the extent, current reporting, and handling of missing data in longitudinal studies of older adults.

Methods

Medline and Embase databases were searched from 2015 to 2019 for publications on longitudinal observational studies conducted among persons ≥55 years old. The search was restricted to 10 general geriatric journals published in English. Reporting and handling of missing data were assessed using questions developed from the recommended standards. Data were summarised descriptively as frequencies and proportions.

Results

A total of 165 studies were included in the review from 7032 identified records. In approximately half of the studies 97 (62.5%), there was either no comment on missing data or unclear descriptions. The percentage of missing data varied from 0.1 to 55%, with a 14% average among the studies that reported having missing data. Complete case analysis was the most common method for handling missing data with nearly 75% of the studies (n = 52) excluding individual observations due to missing data, at the initial phase of study inclusion or at the analysis stage. Of the 10 studies where multiple imputation was used, only 1 (10.0%) study followed the guideline for reporting the procedure fully using online supplementary documents.

Conclusion

The current reporting and handling of missing data in longitudinal observational studies of older adults are inadequate. Journal endorsement and implementation of guidelines may potentially improve the quality of missing data reporting. Further, authors should be encouraged to use online supplementary files to provide additional details on how missing data were addressed, to allow for more transparency and comprehensive appraisal of studies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01605-w.

Increased transparency was required when reporting imputation of primary outcome data in clinical trials

OBJECTIVE

To explore the transparency of reporting primary outcome data within randomised controlled trials (RCTs) in the presence of missing data.

STUDY DESIGN

/ Setting: A cohort examination of RCTs published in the four major medical journals (NEJM, JAMA, BMJ, Lancet) in 2013 and the first quarter of 2018. Data was extracted on reporting quality, the number of randomised participants and the number of participants included within the primary outcome analysis with observed or imputed data.

RESULTS

91/159 (57%) of the studies analysed from 2013 and 19/46 (41%) from 2018 included imputed data within the primary outcome analysis. Of these, only 13/91 (14%) studies from 2013 and 1/19 (5%) studies from 2018 explicitly reported the number of imputed values in the CONSORT diagram. Results tables included levels of imputed data in 12/91 (13%) studies in 2013 and 4/19 (21%) in 2018. Consequently, identification of imputed data was a time-consuming task requiring extensive cross-referencing of all manuscript elements.

CONCLUSION

Imputed primary outcome data is poorly reported. Participant flow diagrams frequently reported participant status which does not necessarily correlate to availability of data. We recommended that the number of imputed values are explicitly reported within CONSORT flow diagrams to increase transparency.

Sensitivity to missing not at random dropout in clinical trials: Use and interpretation of the trimmed means estimator

Outcome values in randomized controlled trials (RCTs) may be missing not at random (MNAR), if patients with extreme outcome values are more likely to drop out (eg, due to perceived ineffectiveness of treatment, or adverse effects). In such scenarios, estimates from complete case analysis (CCA) and multiple imputation (MI) will be biased. We investigate the use of the trimmed means (TM) estimator for the case of univariable missingness in one continuous outcome. The TM estimator operates by setting missing values to the most extreme value, and then “trimming” away equal fractions of both groups, estimating the treatment effect using the remaining data. The TM estimator relies on two assumptions, which we term the “strong MNAR” and “location shift” assumptions. We derive formulae for the TM estimator bias resulting from the violation of these assumptions for normally distributed outcomes. We propose an adjusted TM estimator, which relaxes the location shift assumption and detail how our bias formulae can be used to establish the direction of bias of CCA and TM estimates, to inform sensitivity analyses. The TM approach is illustrated in a sensitivity analysis of the CoBalT RCT of cognitive behavioral therapy (CBT) in 469 individuals with 46 months follow‐up. Results were consistent with a beneficial CBT treatment effect, with MI estimates closer to the null and TM estimates further from the null than the CCA estimate. We propose using the TM estimator as a sensitivity analysis for data where extreme outcome value dropout is plausible.

Analyses of repeatedly measured continuous outcomes in randomized controlled trials needed substantial improvements

OBJECTIVES

Systematic understanding is lacking regarding how current trials handle repeated measure data and the extent to which appropriate statistical methods are used for such data set. This study investigated the current practice of analyzing the repeated measure data among randomized controlled trials (RCTs).

STUDY DESIGN AND SETTING

We searched the Core Clinical Journals indexed in PubMed for RCTs published in 2019 and included a continuous primary outcome with repeated measures. We randomly sampled RCTs from the eligible trials. Team of methods trained investigators screened studies for eligibility and collected data using the pilot-tested, standardized questionnaires. We thoroughly documented statistical analyses of the continuous primary outcome with repeated measures and particularly recorded how statistically advanced methods were used to handle these repeated measures.

RESULTS

In total, 200 trials were included. Of these trials, the mean number of repeated measures for the continuous primary outcome was 5.46 (SD = 3.4); 58 (29.0%) trials did not specify the time point of primary outcome in the method; 113 (56.5%) trials did not use statistically advanced methods for handling repeated measure data in the primary analyses. Among187 trials included the baseline values, 88 (47.1%) trials did not adjust for outcome value at baseline. Among 87 trials using statistically advanced methods, 49 (56.3%) did not specify correlation structure for model.

CONCLUSIONS

The statistical analyses of repeatedly measured continuous outcomes in RCTs need substantial improvements. Careful planning of the primary outcome and the use of statistically advanced methods for analyzing data are warranted.

Completeness of reporting and risks of overstating impact in cluster randomised trials: a systematic review

Overstating the impact of interventions through incomplete or inaccurate reporting can lead to inappropriate scale-up of interventions with low impact. Accurate reporting of the impact of interventions is of great importance in global health research to protect scarce resources. In global health, the cluster randomised trial design is commonly used to evaluate complex, multicomponent interventions, and outcomes are often binary. Complete reporting of impact for binary outcomes means reporting both relative and absolute measures. We did a systematic review to assess reporting practices and potential to overstate impact in contemporary cluster randomised trials with binary primary outcome. We included all reports registered in the Cochrane Central Register of Controlled Trials of two-arm parallel cluster randomised trials with at least one binary primary outcome that were published in 2017. Of 73 cluster randomised trials, most (60 [82%]) showed incomplete reporting. Of 64 cluster randomised trials for which it was possible to evaluate, most (40 [63%]) reported results in such a way that impact could be overstated. Care is needed to report complete evidence of impact for the many interventions evaluated using the cluster randomised trial design worldwide.

Time Window for Ischemic Stroke First Mobilization Effectiveness: Protocol for an Investigator-Initiated Prospective Multicenter Randomized 3-Arm Clinical Trial

OBJECTIVE

The purpose of this study is to investigate the optimal time window for initiating mobilizing after acute ischemic stroke.

METHODS

The TIME Trial is a pragmatic, investigator-initiated, multi-center, randomized, 3-arm parallel group, clinical trial. This trial will be conducted in 57 general hospitals in mainland China affiliated with the China Stroke Databank Center and will enroll 6033 eligible patients with acute ischemic stroke. Participants will be randomly allocated to either (1) the very early mobilization group in which mobilization is initiated within 24 hours from stroke onset, (2) the early mobilization group in which mobilization begins between 24 and 72 hours poststroke, or (3) the late mobilization group in which mobilization is started after 72 hours poststroke. The mobilization protocol is otherwise standardized and identical for each comparison group. Mobilization is titrated by baseline mobility level and progress of patients throughout the intervention period. The primary outcome is death or disability assessed with the modified Rankin scale at 3 months poststroke. Secondary outcomes include impairment score of the National Institutes of Health Stroke Scale, dependence in activities of daily living as measured using the modified Barthel Index, cognitive ability assessed with the Mini-Mental State Examination, incidence of adverse events, hospital length of stay, and total medical costs.

IMPACT

The TIME Trial is designed to answer the question "when is the best time to start mobilization after stroke?" The effect of timing is isolated from the effect of type and dose of mobilization by otherwise applying a standard mobilization protocol across groups. The TIME Trial may, therefore, contribute to increasing the knowledge base regarding the optimal time window for initiating mobilization after acute ischemic stroke.

Caregiving burnout of community-dwelling people with dementia in Hong Kong and New Zealand: a cross-sectional study

Background

Informal caregiving for people with dementia can negatively impact caregivers’ health. In Asia-Pacific regions, growing dementia incidence has made caregiver burnout a pressing public health issue. A cross-sectional study with a representative sample helps to understand how caregivers experience burnout throughout this region. We explored the prevalence and contributing factors of burnout of caregivers of community-dwelling older people with dementia in Hong Kong (HK), China, and New Zealand (NZ) in this study.

Methods

Analysis of interRAI Home Care Assessment data for care-recipients (aged ≥65 with Alzheimer’s disease/other dementia) who had applied for government-funded community services and their caregivers was conducted. The sample comprised 9976 predominately Chinese in HK and 16,725 predominantly European in NZ from 2013 to 2016. Caregiver burnout rates for HK and NZ were calculated. Logistic regression was used to determine the adjusted odds ratio (AOR) of the significant factors associated with caregiver burnout in both regions.

Results

Caregiver burnout was present in 15.5 and 13.9% of the sample in HK and NZ respectively. Cross-regional differences in contributing factors to burnout were found. Care-recipients’ ADL dependency, fall history, and cohabitation with primary caregiver were significant contributing factors in NZ, while primary caregiver being child was found to be significant in HK. Some common contributing factors were observed in both regions, including care-recipients having behavioural problem, primary caregiver being spouse, providing activities-of-daily-living (ADL) care, and delivering more than 21 h of care every week. In HK, allied-health services (physiotherapy, occupational therapy and speech therapy) protected caregiver from burnout. Interaction analysis showed that allied-health service attenuates the risk of burnout contributed by care-recipient’s older age (85+), cohabitation with child, ADL dependency, mood problem, and ADL care provision by caregivers.

Conclusions

This study highlights differences in service delivery models, family structures and cultural values that may explain the cross-regional differences in dementia caregiving experience in NZ and HK. Characteristics of caregiving dyads and their allied-health service utilization are important contributing factors to caregiver burnout. A standardized needs assessment for caregivers could help policymakers and healthcare practitioners to identify caregiving dyads who are at risk of burnout and provide early intervention.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-021-02153-6.