Bayesian network meta-analysis for unordered categorical outcomes with incomplete data

Twenty years of network meta-analysis: Continuing controversies and recent developments

Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.

Network meta-analysis for an ordinal outcome when outcome categorization varies across trials

BACKGROUND

Binary outcomes are likely the most common in randomized controlled trials, but ordinal outcomes can also be of interest. For example, rather than simply collecting data on diseased versus healthy study subjects, investigators may collect information on the severity of disease, with no disease, mild, moderate, and severe disease as possible levels of the outcome. While some investigators may be interested in all levels of the ordinal variable, others may combine levels that are not of particular interest. Therefore, when research synthesizers subsequently conduct a network meta-analysis on a network of trials for which an ordinal outcome was measured, they may encounter a network in which outcome categorization varies across trials.

METHODS

The standard method for network meta-analysis for an ordinal outcome based on a multinomial generalized linear model is not designed to accommodate the multiple outcome categorizations that might occur across trials. In this paper, we propose a network meta-analysis model for an ordinal outcome that allows for multiple categorizations. The proposed model incorporates the partial information provided by trials that combine levels through modification of the multinomial likelihoods of the affected arms, allowing for all available data to be considered in estimation of the comparative effect parameters. A Bayesian fixed effect model is used throughout, where the ordinality of the outcome is accounted for through the use of the adjacent-categories logit link.

RESULTS

We illustrate the method by analyzing a real network of trials on the use of antibiotics aimed at preventing liver abscesses in beef cattle and explore properties of the estimates of the comparative effect parameters through simulation. We find that even with the categorization of the levels varying across trials, the magnitudes of the biases are relatively small and that under a large sample size, the root mean square errors become small as well.

CONCLUSIONS

Our proposed method to conduct a network meta-analysis for an ordinal outcome when the categorization of the outcome varies across trials, which utilizes the adjacent-categories logit link, performs well in estimation. Because the method considers all available data in a single estimation, it will be particularly useful to research synthesizers when the network of interest has only a limited number of trials for each categorization of the outcome.

Conduct and reporting of multivariate network meta-analyses: a scoping review

OBJECTIVES

Combining multivariate and network meta-analysis methods simultaneously in a multivariate network meta-analysis (MVNMA) provides the methodological framework to analyze the largest amount of evidence relevant to decision-makers (i.e., from indirect evidence and correlated outcomes). The objectives of this scoping review were to summarize the characteristics of MVNMAs published in the health sciences literature and map the methodological guidance available for MVNMA.

STUDY DESIGN AND SETTING

We searched MEDLINE, Embase, and the Cumulative Index to Nursing and Allied Health Literature from inception to 28 August 2023, along with citations of included studies, for quantitative evidence syntheses that applied MVNMA and articles addressing MVNMA methods. Pairs of reviewers independently screened potentially eligible studies. Collected data included bibliographic, methodological, and analytical characteristics of included studies. We reported results as total numbers, frequencies, and percentages for categorical variables and medians and interquartile ranges for continuous variables that were not normally distributed.

RESULTS

After screening 1,075 titles and abstracts, and 112 full texts, we included 38 unique studies, of which, 10 were quantitative evidence syntheses that applied MVNMA and 28 were articles addressing MVNMA methods. Among the 10 MVNMAs, the first was published in 2013, four used studies identified from already published systematic reviews, and eight addressed pharmacological interventions, which were the most common interventions. They evaluated interventions for metastatic melanoma, colorectal cancer, prostate cancer, oral hygiene, disruptive behavior disorders, rheumatoid arthritis, narcolepsy, type 2 diabetes, and overactive bladder syndrome. Five MVNMAs analyzed two outcomes simultaneously, and four MVNMAs analyzed three outcomes simultaneously. Among the articles addressing MVNMA methods, the first was published in 2007 and the majority provided methodological frameworks for conducting MVNMAs (26/28, 93%). One study proposed criteria to standardize reporting of MVNMAs and two proposed items relevant to the quality assessment of MVNMAs. Study authors used data from 18 different illnesses to provide illustrative examples within their methodological guidance.

CONCLUSIONS

The application of MVNMA in the health sciences literature is uncommon. Many methodological frameworks are published; however, standardization and specific criteria to guide reporting and quality assessment are lacking. This overview of the current landscape may help inform future conduct of MVNMAs and research on MVNMA methods.

Impacts of a perinatal exposure to manganese coupled with maternal stress in rats: Tests of untrained behaviors

Manganese (Mn), an element that naturally occurs in the environment, has been shown to produce neurotoxic effects on the developing young when levels exceed physiological requirements. To evaluate the effects of this chemical in combination with non-chemical factors pregnant Long-Evans rats were treated with 0, 2, or 4 mg/mL Mn in their drinking water from gestational day (GD) 7 to postnatal day (PND) 22. Half of the dams received a variable stress protocol from GD13 to PND9, that included restraint, small cage with reduced bedding, exposure to predator odor, intermittent intervals of white noise, lights on for 24 h, intermittent intervals of lights on during dark cycle and cages with grid floors and reduced bedding. One male and one female offspring from each litter were tested to assess untrained behavior. Ultrasonic vocalizations (USV) were recorded from PND13 pups while they were isolated from the litter. Locomotor activity (MA) was measured in figure-eight mazes at PND 17, 29, and 79 (different set of rats at each time point). Social approach (SA) was tested at PND48. Acoustic startle response (ASR) and pre-pulse inhibition (PPI) were measured starting at PND58. At PND53 a sweetness preference for a chocolate flavored milk solution was assessed. There were sex related differences on several parameters for the USVs. There was also a Mn by stress by sex interaction with the females from the 4 mg/mL stressed dams having more frequency modulated (FM) call elements than the 4 mg/mL non-stressed group. There was an effect of Mn on motor activity but only at PND29 with the 2 mg/mL group having higher counts than the 0 mg/mL group. The social approach test showed sex differences for both the habituation and test phase. There was an effect of Mn, with the 4 mg/mL males having a greater preference for the stimulus rat than did the 0 mg/mL males. There was also a stress by sex interaction. The ASR and PPI had only a sex effect. Thus, with only the FM call elements having a Mn by stress effect, and the PND29 MA and SA preference index having a Mn effect but at different doses requires further investigation.

Methodological approaches for analysing data from therapeutic efficacy studies

Several anti-malarial drugs have been evaluated in randomized clinical trials to treat acute uncomplicated Plasmodium falciparum malaria. The outcome of anti-malarial drug efficacy studies is classified into one of four possible outcomes defined by the World Health Organization: adequate clinical and parasitological response, late parasitological failure, late clinical failure, early treatment failure. These four ordered categories are ordinal data, which are reduced to either a binary outcome (i.e., treatment success and treatment failure) to calculate the proportions of treatment failure or to time-to-event outcome for KaplanMeier survival analysis. The arbitrary transition from 4-level ordered categories to 2-level type categories results in a loss of statistical power. In the opinion of the authors, this outcome can be considered as ordinal at a fixed endpoint or at longitudinal endpoints. Alternative statistical methods can be applied to 4-level ordinal categories of therapeutic response to optimize data exploitation. Furthermore, network meta-analysis is useful not only for direct comparison of drugs which were evaluated together in a randomized design, but also for indirect comparison of different artemisinin-based combinations across different clinical studies using a common drug comparator, with the aim to determine the ranking order of drug efficacy. Previous works conducted in Cameroonian children served as data source to illustrate the feasibility of these novel statistical approaches. Data analysis based on ordinal end-point may be helpful to gain further insight into anti-malarial drug efficacy.

Leveraging Cumulative Network Meta-analysis and Value of Information Analysis to Understand the Evolving Value of Medical Research

BACKGROUND

Leveraging cumulative network meta-analysis (NMA) and value of information (VOI) analysis, this article aims to understand the evolving value of medical research and to identify gaps in the evidence for future research.

METHODS

As an illustration, we identified 31 randomized controlled trials (RCT) from 1980 to 2013 that examined a network of 3 interventions for coronary artery disease: medical therapy (MED), percutaneous coronary intervention (PCI), and coronary artery bypass graft (CABG) surgery. We conducted Bayesian NMA to combine evidence from a new RCT with existing knowledge. Then, using the Duke Databank for Cardiovascular Diseases database, we developed an accelerated failure time model to estimate the joint effects of patient characteristics and treatment choices on survival outcomes. With the estimated coefficients and covariance matrices, we projected survival benefits and its surrounding uncertainty among 50,000 simulated patients treated with MED, PCI, or CABG. The value of resolving residual uncertainty from future trials was quantified through the VOI analysis. We repeated these steps for each published RCT to estimate dynamic changes in VOI estimates.

RESULTS

Our cumulative NMA found that CABG conferred a lower, but not statistically significant, mortality than PCI (hazard ratio [HR], 0.90; 95% uncertainty interval, 0.80-1.05). MED had a nonsignificantly higher long-term mortality than PCI (HR, 1.11; 0.98-1.31) but significantly higher than CABG (HR, 1.07; 1.23-1.41). The greatest potential gains from future research would come from additional head-to-head trials between CABG v. PCI with the value of future research equaling 0.27 life years per patient.

CONCLUSIONS

The combination of cumulative NMA and VOI approaches can improve the efficiency of comparative effectiveness research by using all of the available evidence to determine future research priorities.

Personalized (N-of-1) Trials for Depression: A Systematic Review

PURPOSE/BACKGROUND

Personalized (N-of-1) trials are single-patient, crossover-design trials that may be useful for personalizing the selection of depression treatments. We conducted a systematic review of published N-of-1 trials for depression to determine the feasibility and suitability of this methodology for personalizing depression care.

METHODS/PROCEDURES

Electronic databases were searched from database inception through October 2016. Studies were selected if they enrolled depressed patients, included a within-subject crossover design, and systematically assessed depressive symptoms during the N-of-1 trial.

FINDINGS/RESULTS

Five eligible studies reporting on 47 depressed patients (range, 1-18 patients) were identified. Two studies were conducted among adults with treatment-resistant depression, 1 study among depressed inpatients, and 2 studies among patients from special populations (geriatric nursing home, human immunodeficiency virus-associated encephalopathy). All studies evaluated the effects of pharmacologic treatments (methylphenidate, D-amphetamine, ketamine, and sulpiride). Three studies compared an off-label treatment with placebo, 1 study compared 2 off-label treatments, and 1 study compared escalating doses of an off-label treatment with placebo. All 4 studies with more than 1 participant demonstrated heterogeneous treatment effects. All studies produced data that could personalize treatment selection for individual patients. No studies reported on recruitment challenges, compliance with self-tracking, or satisfaction with participation.

IMPLICATIONS/CONCLUSIONS

The feasibility of N-of-1 trials for depression was demonstrated for a limited number of second-line pharmacologic treatments in treatment-resistant patients or in patients with comorbidities that would have excluded them from conventional randomized controlled trials. Additional research is needed to determine whether N-of-1 trials are suitable for improving the selection of depression treatments in clinical practice.

A Bayesian missing data framework for generalized multiple outcome mixed treatment comparisons

Bayesian statistical approaches to mixed treatment comparisons (MTCs) are becoming more popular because of their flexibility and interpretability. Many randomized clinical trials report multiple outcomes with possible inherent correlations. Moreover, MTC data are typically sparse (although richer than standard meta-analysis, comparing only two treatments), and researchers often choose study arms based upon which treatments emerge as superior in previous trials. In this paper, we summarize existing hierarchical Bayesian methods for MTCs with a single outcome and introduce novel Bayesian approaches for multiple outcomes simultaneously, rather than in separate MTC analyses. We do this by incorporating partially observed data and its correlation structure between outcomes through contrast-based and arm-based parameterizations that consider any unobserved treatment arms as missing data to be imputed. We also extend the model to apply to all types of generalized linear model outcomes, such as count or continuous responses. We offer a simulation study under various missingness mechanisms (e.g., missing completely at random, missing at random, and missing not at random) providing evidence that our models outperform existing models in terms of bias, mean squared error, and coverage probability then illustrate our methods with a real MTC dataset. We close with a discussion of our results, several contentious issues in MTC analysis, and a few avenues for future methodological development.

GetReal in network meta-analysis: a review of the methodology

Pairwise meta-analysis is an established statistical tool for synthesizing evidence from multiple trials, but it is informative only about the relative efficacy of two specific interventions. The usefulness of pairwise meta-analysis is thus limited in real-life medical practice, where many competing interventions may be available for a certain condition and studies informing some of the pairwise comparisons may be lacking. This commonly encountered scenario has led to the development of network meta-analysis (NMA). In the last decade, several applications, methodological developments, and empirical studies in NMA have been published, and the area is thriving as its relevance to public health is increasingly recognized. This article presents a review of the relevant literature on NMA methodology aiming to pinpoint the developments that have appeared in the field. Copyright © 2016 John Wiley & Sons, Ltd.

Multivariate meta-analysis using individual participant data

When combining results across related studies, a multivariate meta-analysis allows the joint synthesis of correlated effect estimates from multiple outcomes. Joint synthesis can improve efficiency over separate univariate syntheses, may reduce selective outcome reporting biases, and enables joint inferences across the outcomes. A common issue is that within-study correlations needed to fit the multivariate model are unknown from published reports. However, provision of individual participant data (IPD) allows them to be calculated directly. Here, we illustrate how to use IPD to estimate within-study correlations, using a joint linear regression for multiple continuous outcomes and bootstrapping methods for binary, survival and mixed outcomes. In a meta-analysis of 10 hypertension trials, we then show how these methods enable multivariate meta-analysis to address novel clinical questions about continuous, survival and binary outcomes; treatment–covariate interactions; adjusted risk/prognostic factor effects; longitudinal data; prognostic and multiparameter models; and multiple treatment comparisons. Both frequentist and Bayesian approaches are applied, with example software code provided to derive within-study correlations and to fit the models.