Analyzing center specific outcomes in hematopoietic cell transplantation

Benchmarking survival outcomes: A funnel plot for survival data

Benchmarking is commonly used in many healthcare settings to monitor clinical performance, with the aim of increasing cost-effectiveness and safe care of patients. The funnel plot is a popular tool in visualizing the performance of a healthcare center in relation to other centers and to a target, taking into account statistical uncertainty. In this paper, we develop a methodology for constructing funnel plots for survival data. The method takes into account censoring and can deal with differences in censoring distributions across centers. Practical issues in implementing the methodology are discussed, particularly in the setting of benchmarking clinical outcomes for hematopoietic stem cell transplantation. A simulation study is performed to assess the performance of the funnel plots under several scenarios. Our methodology is illustrated using data from the European Society for Blood and Marrow Transplantation benchmarking project.

COVID-19 and Hematopoietic Cell Transplantation Center-Specific Survival Analysis: Can We Adjust for the Impact of the Pandemic? Recommendations of the COVID-19 Task Force of the 2020 Center for International Blood and Marrow Transplantation Research Center Outcomes Forum

COVID-19 has significantly impacted the practice of hematopoietic cell transplantation (HCT) and likely affected outcomes of HCT recipients. Early reports document substantially higher case fatality rates for HCT recipients than seen in faced by the general population. Currently we do not have a clear picture of how much of this threat is present within the first year after HCT and how infection rates and outcomes vary with time after HCT. There are important because center-specific survival estimates for reporting purposes focus on 1-year post-HCT mortality. Transplantation centers have dramatically changed their practices in response to the pandemic. At many centers, quality assurance processes and procedures were disrupted, changes that likely affected team performance. Centers have been affected unevenly by the pandemic through time, location, and COVID-19 burdens. Assessment of center-specific survival depends on the ability to adjust for risk factors, such as COVID-19, that are outside center control using consistent methods so that team performance based on controllable risk factors can be ascertained. The Center for International Blood and Marrow Transplantation Research (CIBMTR) convened a working group for the 2020 Center Outcomes Forum to assess the impact of COVID-19 on both patient-specific risks and center-specific performance. This committee reviewed the factors at play and developed recommendations for a process to determine whether adjustments in the methodology to assess center-specific performance are needed.

Community health status and outcomes after allogeneic hematopoietic cell transplantation in the United States

BACKGROUND

The association of community factors and outcomes after hematopoietic cell transplantation (HCT) has not been comprehensively described. Using the County Health Rankings and Roadmaps (CHRR) and the Center for International Blood and Marrow Transplant Research (CIBMTR), this study evaluated the impact of community health status on allogeneic HCT outcomes.

METHODS

This study included 18,544 adult allogeneic HCT recipients reported to the CIBMTR by 170 US centers in 2014-2016. Sociodemographic, environmental, and community indicators were derived from the CHRR, an aggregate community risk score was created, and scores were assigned to each patient (patient community risk score [PCS]) and transplant center (center community risk score [CCS]). Higher scores indicated less healthy communities. The impact of PCS and CCS on patient outcomes after allogeneic HCT was studied.

RESULTS

The median age was 55 years (range, 18-83 years). The median PCS was -0.21 (range, -1.37 to 2.10; standard deviation [SD], 0.42), and the median CCS was -0.13 (range, -1.04 to 0.96; SD, 0.40). In multivariable analyses, a higher PCS was associated with inferior survival (hazard ratio [HR] per 1 SD increase, 1.04; 99% CI, 1.00-1.08; P = .0089). Among hematologic malignancies, a tendency toward inferior survival was observed with a higher PCS (HR, 1.04; 99% CI, 1.00-1.08; P = .0102); a higher PCS was associated with higher nonrelapse mortality (NRM; HR, 1.08; 99% CI, 1.02-1.15; P = .0004). CCS was not significantly associated with survival, relapse, or NRM.

CONCLUSIONS

Patients residing in counties with a worse community health status have inferior survival as a result of an increased risk of NRM after allogeneic HCT. There was no association between the community health status of the transplant center location and allogeneic HCT outcomes.

Transplant center characteristics and survival after allogeneic hematopoietic cell transplantation in adults

Allogeneic hematopoietic cell transplantation (alloHCT) is a highly specialized procedure. We surveyed adult transplant centers in the United States (US) and then used data reported to the Center for International Blood and Marrow Transplant Research (CIBMTR) (2008–2010) to evaluate associations of center volume, infrastructure, and care delivery models with survival post alloHCT. Based on their 2010 alloHCT volume, centers were categorized as low-volume (≤40 alloHCTs; N=42 centers, 1,900 recipients) or high-volume (>40 alloHCTs; N=41 centers, 9,637 recipients). 100-day survival was 86% (95% CI, 85–87%) in high-volume compared to 83% (95% CI, 81–85%) in low-volume centers (difference 3%; P<0.001). One-year survival was 62% (95% CI, 61–63%) and 56% (95% CI, 54–58%), respectively (difference 6%; P < 0.001). Logistic regression analyses adjusted for patient and center characteristics; alloHCT at high-volume centers (odds ratio [OR] 1.32; P<0.001) and presence of a survivorship program dedicated to HCT recipients (OR 1.23; P=0.009) were associated with favorable 1-year survival compared to low-volume centers. Similar findings were observed in a CIBMTR validation cohort (2012–2014); high-volume centers had better 1-year survival (OR 1.24, P<0.001). Among US adult transplant centers, alloHCT at high-volume centers and at centers with survivorship programs is associated with higher 1-year survival.

Pre-transplantation Risks and Transplant-Techniques in Haematopoietic Stem Cell Transplantation for Acute Leukaemia

BACKGROUND

The role of conditioning intensity and stem cell source on modifying pre-transplantation risk in allogeneic haematopoietic stem cell transplantation (HSCT) is a matter of debate, but crucial when benchmarking centres.

METHODS

This Retrospective, multicenter exploratory-validation analysis of 9103 patients, (55.5% male, median age 50 years; 1-75 years range) with an allogeneic HSCT between 2010 and 2016 from a matched sibling (N = 8641; 95%) or matched unrelated donor (N = 462; 5%) for acute myeloid (N = 6432; 71%) or acute lymphoblastic (N = 2671; 29%) leukaemia in first complete remission, and reported by 240 centres in 30 countries to the benchmark database of the European Society for Blood and Marrow Transplantation (EBMT) searched for factors associated with use of transplant techniques (standard N = 6375;70% or reduced intensity conditioning N = 2728;30%, respectively bone marrow N = 1945;21% or peripheral blood N = 7158;79% as stem cell source), and their impact on outcome.

FINDINGS

Treatment groups differed significantly from baseline population (p < 0.001), and within groups regarding patient-, disease-, donor-, and centre-related pre-transplantation risk factors (p < 0.001); choice of technique did depend on pre-transplantation risk factors and centre (p < 0.001). Probability of overall survival at 5 years decreased systematically and significantly with increasing pre-transplantation risk score (score 2 vs 0/1 HR: 1·2, 95% c.i. [1·1-1·.3], p = 0.002; score 3 vs 0/1 HR: 1·5, 95% c.i. [1·3-1·7], p < 0.001; score 4/5/6 vs 0/1 HR: 1·9, 95% c.i. [1·6-2·2], p < 0.001) with no significant differences between treatment groups (likelihood ratio test on interaction: p = 0.40). Overall survival was significantly associated with selection steps and completeness of information (p < 0.001).

INTERPRETATION

Patients' pre-transplantation risk factors determine survival, independent of transplant techniques. Transplant techniques should be regarded as centre policy, not stratification factor in benchmarking. Selection criteria and completeness of data bias outcome. Outcomes may be improved more effectively through better identifying pre-transplantation factors as opposed to refinement of transplant techniques.

FUNDING

The study was funded by EBMT.

Providing both autologous and allogeneic hematopoietic stem cell transplants (HSCT) may have a stronger impact on the outcome of autologous HSCT in adult patients than activity levels or implementation of JACIE at Belgian transplant centres

While performance since the introduction of the JACIE quality management system has been shown to be improved for allogeneic hematopoietic stem cell transplants (HSCT), impact on autologous-HSCT remains unclear in Europe. Our study on 2697 autologous-HSCT performed in adults in 17 Belgian centres (2007-2013) aims at comparing the adjusted 1 and 3-yr survival between the different centres & investigating the impact of 3 centre-related factors on performance (time between JACIE accreditation achievement by the centre and the considered transplant, centre activity volume and type of HSCT performed by centres: exclusively autologous vs both autologous & allogeneic). We showed a relatively homogeneous performance between Belgian centres before national completeness of JACIE implementation. The 3 centre-related factors had a significant impact on the 1-yr survival, while activity volume and type of HSCT impacted the 3-yr survival of autologous-HSCT patients in univariable analyses. Only activity volume (impact on 1-yr survival only) and type of HSCT (impact on 1 and 3-yr survivals) remained significant in multivariable analysis. This is explained by the strong relationship between these 3 variables. An extended transplantation experience, i.e., performing both auto & allo-HSCT, appears to be a newly informative quality indicator potentially conveying a multitude of underlying complex factors.

Country-Level Macroeconomic Indicators Predict Early Post-Allogeneic Hematopoietic Cell Transplantation Survival in Acute Lymphoblastic Leukemia: A CIBMTR Analysis

For patients with acute lymphoblastic leukemia (ALL), allogeneic hematopoietic cell transplantation (alloHCT) offers a potential cure. Life-threatening complications can arise from alloHCT that require the application of sophisticated health care delivery. The impact of country-level economic conditions on post-transplantation outcomes is not known. Our objective was to assess whether these variables were associated with outcomes for patients transplanted for ALL. Using data from the Center for Blood and Marrow Transplant Research, we included 11,261 patients who received a first alloHCT for ALL from 303 centers across 38 countries between the years of 2005 and 2013. Cox regression models were constructed using the following macroeconomic indicators as main effects: Gross national income per capita, health expenditure per capita, and Human Development Index (HDI). The outcome was overall survival at 100 days following transplantation. In each model, transplants performed within lower resourced environments were associated with inferior overall survival. In the model with the HDI as the main effect, transplants performed in the lowest HDI quartile (n = 697) were associated with increased hazard for mortality (hazard ratio, 2.42; 95% confidence interval, 1.64 to 3.57; P < .001) in comparison with transplants performed in the countries with the highest HDI quartile. This translated into an 11% survival difference at 100 days (77% for lowest HDI quartile versus 88% for all other quartiles). Country-level macroeconomic indices were associated with lower survival at 100 days after alloHCT for ALL. The reasons for this disparity require further investigation.

Payment and Care for Hematopoietic Cell Transplantation Patients: Toward a Specialized Medical Home for Complex Care Patients

Patient-centered medical home models are fundamental to the advanced alternative payment models defined in the Medicare Access and Children's Health Insurance Plan Reauthorization Act (MACRA). The patient-centered medical home is a model of healthcare delivery supported by alternative payment mechanisms and designed to promote coordinated medical care that is simultaneously patient-centric and population-oriented. This transformative care model requires shifting reimbursement to include a per-patient payment intended to cover services not previously reimbursed such as disease management over time. Payment is linked to quality measures, including proportion of care delivered according to predefined pathways and demonstrated impact on outcomes. Some medical homes also include opportunities for shared savings by reducing overall costs of care. Recent proposals have suggested expanding the medical home model to specialized populations with complex needs because primary care teams may not have the facilities or the requisite expertise for their unique needs. An example of a successful care model that may provide valuable lessons for those creating specialty medical home models already exists in many hematopoietic cell transplantation (HCT) centers that deliver multidisciplinary, coordinated, and highly specialized care. The integration of care delivery in HCT centers has been driven by the specialty care their patients require and by the payment methodology preferred by the commercial payers, which has included bundling of both inpatient and outpatient care in the peritransplant interval. Commercial payers identify qualified HCT centers based on accreditation status and comparative performance, enabled in part by center-level comparative performance data available within a national outcomes database mandated by the Stem Cell Therapeutic and Research Act of 2005. Standardization across centers has been facilitated via voluntary accreditation implemented by Foundation for the Accreditation of Cell Therapy. Payers have built on these community-established programs and use public outcomes and program accreditation as standards necessary for inclusion in specialty care networks and contracts. Although HCT centers have not been described as medical homes, most HCT providers have already developed the structures that address critical requirements of MACRA for medical homes.

ROBUST MIXED EFFECTS MODEL FOR CLUSTERED FAILURE TIME DATA: APPLICATION TO HUNTINGTON'S DISEASE EVENT MEASURES

An important goal in clinical and statistical research is properly modeling the distribution for clustered failure times which have a natural intraclass dependency and are subject to censoring. We handle these challenges with a novel approach that does not impose restrictive modeling or distributional assumptions. Using a logit transformation, we relate the distribution for clustered failure times to covariates and a random, subject-specific effect. The covariates are modeled with unknown functional forms, and the random effect may depend on the covariates and have an unknown and unspecified distribution. We introduce pseudovalues to handle censoring and splines for functional covariate effects, and frame the problem into fitting an additive logistic mixed effects model. Unlike existing approaches for fitting such models, we develop semiparametric techniques that estimate the functional model parameters without specifying or estimating the random effect distribution. We show both theoretically and empirically that the resulting estimators are consistent for any choice of random effect distribution and any dependency structure between the random effect and covariates. Last, we illustrate the method's utility in an application to a Huntington's disease study where our method provides new insights into differences between motor and cognitive impairment event times in at-risk subjects.

Optimizing Quality and Efficiency of Healthcare Delivery in Hematopoietic Cell Transplantation

Hematopoietic cell transplantation is a complex and resource intense procedure that can be associated with high risks of treatment failure due to disease relapse or complications. There also exists considerable variability among transplant centers with respect to the number of procedures performed, available resources and personnel, patient selection, transplant practices, and supportive care. Hematopoietic cell transplantation as a specialty has been a pioneer in incorporating the constructs of quality and efficiency routinely in patient care. However, several challenges still remain. Harmonization of data collection and reporting, use of innovative technological tools, evidence-based practice supported by clinical trials, better efforts towards care coordination and transition of care, and reduction of variation will facilitate these efforts and will lead to improved experience and outcomes for hematopoietic cell transplant recipients.

Comparing center-specific cumulative incidence functions

The competing risks data structure arises frequently in clinical and epidemiologic studies. In such settings, the cumulative incidence function is often used to describe the ultimate occurrence of a particular cause of interest. If the objective of the analysis is to compare subgroups of patients with respect to cumulative incidence, imbalance with respect to group-specific covariate distributions must generally be factored out, particularly in observational studies. This report proposes a measure to contrast center- (or, more generally group-) specific cumulative incidence functions (CIF). One such application involves evaluating organ procurement organizations with respect to the cumulative incidence of kidney transplantation. In this case, the competing risks include (i) death on the wait-list and (ii) removal from the wait-list. The proposed method assumes proportional cause-specific hazards, which are estimated through Cox models stratified by center. The proposed center effect measure compares the average CIF for a given center to the average CIF that would have resulted if that particular center had covariate pattern-specific cumulative incidence equal to that of the national average. We apply the proposed methods to data obtained from a national organ transplant registry.

Economics and Outcome After Hematopoietic Stem Cell Transplantation: A Retrospective Cohort Study

Hematopoietic stem cell transplantation (HSCT) is a lifesaving expensive medical procedure. Hence, more transplants are performed in more affluent countries. The impact of economic factors on patient outcome is less defined. We analyzed retrospectively a defined cohort of 102,549 patients treated with an allogeneic (N = 37,542; 37%) or autologous (N = 65,007; 63%) HSCT. They were transplanted by one of 404 HSCT centers in 25 European countries between 1999 and 2006. We searched for associations between center-specific microeconomic or country-specific macroeconomic factors and outcome. Center patient-volume and center program-duration were significantly and systematically associated with improved survival after allogeneic HSCT (HR 0·87; 0·84-0·91 per 10 patients; p < 0·0001; HR 0·90;0·85-0·90 per 10 years; p < 0·001) and autologous HSCT (HR 0·91;0·87-0·96 per 10 patients; p < 0·001; HR 0·93;0·87-0·99 per 10 years; p = 0·02). The product of Health Care Expenditures by Gross National Income/capita was significantly associated in multivariate analysis with all endpoints (R(2) = 18%; for relapse free survival) after allogeneic HSCT. Data indicate that country- and center-specific economic factors are associated with distinct, significant, systematic, and clinically relevant effects on survival after HSCT. They impact on center expertise in long-term disease and complication management. It is likely that these findings apply to other forms of complex treatments.

Methods for comparing center-specific survival outcomes using direct standardization

The evaluation of center-specific outcomes is often through survival analysis methods. Such evaluations must account for differences in the distribution of patient characteristics across centers. In the context of censored event times, it is also important that the measure chosen to evaluate centers not be influenced by imbalances in the center-specific censoring distributions. The practice of using center indicators in a hazard regression model is often invalid, inconvenient, or undesirable to carry out. We propose a semiparametric version of the standardized rate ratio (SRR) useful for the evaluation of centers with respect to a right-censored event time. The SRR for center j can be interpreted as the ratio of the expected number of deaths in the total population (if the total population were in fact subject to the center j mortality hazard) to the observed number of events. The proposed measure is not affected by differences in center-specific covariate or censoring distributions. Asymptotic properties of the proposed estimators are derived, with finite-sample properties examined through simulation studies. The proposed methods are applied to national kidney transplant data.

A Gaussian Copula Model for Multivariate Survival Data

We consider a Gaussian copula model for multivariate survival times. Estimation of the copula association parameter is easily implemented with existing software using a two-stage estimation procedure. Using the Gaussian copula, we are able to test whether the association parameter is equal to zero. When the association term is positive, the model can be extended to incorporate cluster-level frailty terms. Asymptotic properties are derived under the two-stage estimation scheme. Simulation studies verify finite sample utility. We apply the method to a Children's Oncology Group multi-center study of acute lymphoblastic leukemia. The analysis estimates marginal treatment effects and examines potential clustering within treatment institution.