Interpretation of subgroup results in clinical trial publications: insights from a survey of medical specialists in Ontario, Canada

An approach to explore for a sweet spot in randomized trials

OBJECTIVE

The objective of the study was to demonstrate how a conventional randomized trial can be analyzed through a stratified or a matched approach to identify a potential sweet spot where observed differences might be accentuated in the mid range of disease severity.

DESIGN AND SETTING

We review a landmark randomized trial of heart failure patients that tested whether implantable defibrillators reduce mortality (n = 2,521).

RESULTS

Overall, 22% (182/829) of the patients in the defibrillator group died compared with 29% (484/1,692) of patients in the control group. Proportional hazards analysis yielded a modest 25% survival benefit (hazard ratio = 0.75, 95% confidence interval: 0.63 to 0.89). Stratified analysis of the trial yielded a larger 52% survival benefit for those in the middle quintile of disease severity (hazard ratio = 0.48, 95% confidence interval: 0.29 to 0.79). In contrast, little of the survival benefit was explained by patients with the greatest disease severity (hazard ratio = 0.89, 95% confidence interval: 0.69 to 1.15). The discrepancy between crude and stratified analyses could be visualized by graphical displays and replicated with matched comparisons.

CONCLUSION

Our approach for analyzing a randomized trial could help identify a potential sweet spot of an accentuated treatment effect.

Prescribers' Knowledge and Skills for Interpreting Research Results: A Systematic Review

INTRODUCTION

Appropriate medication prescribing may be influenced by a prescriber's ability to understand and interpret medical research. The objective of this review was to synthesize the research related to prescribers' critical appraisal knowledge and skills-defined as the understanding of statistical methods, biases in studies, and relevance and validity of evidence.

METHODS

We searched PubMed and other databases from January 1990 through September 2015. Two reviewers independently screened and selected studies of any design conducted in the United States, the United Kingdom, or Canada that involved prescribers and that objectively measured critical appraisal knowledge, skills, understanding, attitudes, or prescribing behaviors. Data were narratively synthesized.

RESULTS

We screened 1204 abstracts, 72 full-text articles, and included 29 studies. Study populations included physicians. Physicians' extant knowledge and skills were in the low to middle of the possible score ranges and demonstrated modest increases in response to interventions. Physicians with formal education in epidemiology, biostatistics, and research demonstrated higher levels of knowledge and skills. In hypothetical scenarios presenting equivalent effect sizes, the use of relative effect measures was associated with greater perceptions of medication effectiveness and intent to prescribe, compared with the use of absolute effect measures. The evidence was limited by convenience samples and study designs that limit internal validity.

DISCUSSION

Critical appraisal knowledge and skills are limited among physicians. The effect measure used can influence perceptions of treatment effectiveness and intent to prescribe. How critical appraisal knowledge and skills fit among the myriad of influences on prescribing behavior is not known.

Assessing Heterogeneity of Treatment Effects: Are Authors Misinterpreting Their Results?

OBJECTIVE

To determine whether investigations of heterogeneity of treatment effects (HTE) in randomized-controlled trials (RCTs) are prespecified and whether authors' interpretations of their analyses are consistent with the objective evidence.

DATA SOURCES/STUDY SETTING

Trials published in Annals of Internal Medicine, British Medical Journal, Journal of the American Medical Association, Lancet, and New England Journal of Medicine in 1994, 1999, and 2004.

STUDY DESIGN

We reviewed 87 RCTs that reported formal tests for statistical interaction or heterogeneity (HTE analyses), derived from a probability sample of 541 articles.

DATA COLLECTION/EXTRACTION

We recorded reasons for performing HTE analysis; an objective classification of evidence for HTE (termed "clinicostatistical divergence" [CSD]); and authors' interpretations of findings. Authors' interpretations, compared with CSD, were coded as understated, overstated, or adequately stated.

PRINCIPLE FINDINGS

Fifty-three RCTs (61 percent) claimed prespecified covariates for HTE analyses. Trials showed strong (6), moderate (11), weak (25), or negligible (16) evidence for CSD (29 could not be classified due to inadequate information). Authors stated that evidence for HTE was sufficient to support differential treatment in subgroups (10); warranted more research (31); was absent (21); or provided no interpretation (25). HTE was overstated in 22 trials, adequately stated in 57 trials, and understated in 8 trials.

CONCLUSIONS

Inconsistencies in performance and reporting may limit the potential of HTE analysis as a tool for identifying HTE and individualizing care in diverse populations. Recommendations for future studies on the reporting and interpretation of HTE analyses are provided.