Rates of false flagging due to statistical artifact in CMS evaluations of transplant programs: results of a stochastic simulation

Evaluations of statistical methods for outlier detection when benchmarking in clinical registries: a systematic review

OBJECTIVES

Benchmarking is common in clinical registries to support the improvement of health outcomes by identifying underperforming clinician or health service providers. Despite the rise in clinical registries and interest in publicly reporting benchmarking results, appropriate methods for benchmarking and outlier detection within clinical registries are not well established, and the current application of methods is inconsistent. The aim of this review was to determine the current statistical methods of outlier detection that have been evaluated in the context of clinical registry benchmarking.

DESIGN

A systematic search for studies evaluating the performance of methods to detect outliers when benchmarking in clinical registries was conducted in five databases: EMBASE, ProQuest, Scopus, Web of Science and Google Scholar. A modified healthcare modelling evaluation tool was used to assess quality; data extracted from each study were summarised and presented in a narrative synthesis.

RESULTS

Nineteen studies evaluating a variety of statistical methods in 20 clinical registries were included. The majority of studies conducted application studies comparing outliers without statistical performance assessment (79%), while only few studies used simulations to conduct more rigorous evaluations (21%). A common comparison was between random effects and fixed effects regression, which provided mixed results. Registry population coverage, provider case volume minimum and missing data handling were all poorly reported.

CONCLUSIONS

The optimal methods for detecting outliers when benchmarking clinical registry data remains unclear, and the use of different models may provide vastly different results. Further research is needed to address the unresolved methodological considerations and evaluate methods across a range of registry conditions.

PROSPERO REGISTRATION NUMBER

CRD42022296520.

Kidney transplant program specific reporting and transplant metrics

PURPOSE OF REVIEW

Kidney transplantation is a heavily regulated medical procedure with the Secretary of HHS ultimately responsible for oversight and authority derived from the NOTA and the Final Rule. Transplant Programs undergo publicly reported evaluations every 6 months based on outcomes from a 2-and-a-half-year period. The current Bayesian metrics for kidney transplant programs were created such that over ten percentage of programs are deemed underperformers, or 'flag', every 6 months. Newly suggested transplant metrics have been released for public comment in Summer 2021. In addition to graft outcomes, waiting list mortality and organ acceptance rate ratios are proposed.

RECENT FINDINGS

Under the newly proposed kidney transplant metrics, over 10% of programs are expected to be deemed underperformers or 'flagged'. Transplant Center flagging is well correlated with decreased transplantation due to the transplant centres move to more conservative organ and patient acceptance. Death on the waiting list is a proposed metric over which transplant centres have little influence.

SUMMARY

In the USA, the harsh regulation continued by Health Resources and Services Administration (HRSA) through the national organ procurement and transplant network (OPTN) and Scientific Registry for Transplant Recipients (SRTR) leads directly to high organ discard rates and limitations to transplanting patients with perceived unadjusted risks. Instead of loosening regulation in a highly functioning industry that achieves remarkable outcomes in end stage kidney patients, the OPTN with the SRTR persist in increasing potential penalties through more proposed metrics that continue to deem 10% of US kidney transplant programs as underperformers. HRSA must establish a reasonable regulatory environment that allows for innovation and increased transplant opportunities for US end-stage renal disease patients.

Textbook surgical outcome in lung transplantation: Analysis of a US national registry

Textbook surgical outcome (TO) is a novel composite quality measure in lung transplantation (LTx). Compared to one-year survival metrics, TO may better differentiate center performance, and motivate improvements in care. To understand the feasibility of implementing this metric, we defined TO in LTx using US national data, and evaluated its ability to predict post-transplant outcomes and differentiate center performance. Adult patients who underwent isolated LTx between 2016-2019 were included. TO was defined as freedom from post-transplant length of stay >30 days, 90-day mortality, intubation or extracorporeal membrane oxygenation at 72 hours post-transplant, post-transplant ventilator support lasting ≥5 days, postoperative airway dehiscence, inpatient dialysis, pre-discharge acute rejection, and grade 3 primary graft dysfunction at 72 hours. Recipient and donor characteristics and post-transplant outcomes were compared between patients who achieved and failed TO. Of 8959 lung transplant recipients, 4664 (52.1%) achieved TO. Patient and graft survival were improved among patients who achieved TO (both log-rank p<0.0001). Among 62 centers, adjusted rates of TO ranged from 27.0% to 72.4% reflecting a wide variability in center-level performance. TO defined using national data may represent a novel composite metric to guide quality improvement in LTx across US transplant centers. Summary: In this study we defined textbook outcome (TO) for lung transplantation (LTx) using US national data. We found that achievement of TO was associated with improved post-transplant survival, and wide variability in center-level LTx performance. These findings suggest that TO could be readily implemented to compare quality of care among US LTx centers. This article is protected by copyright. All rights reserved.

Long-term kidney transplant graft survival-Making progress when most needed

Current short-term kidney post-transplant survival rates are excellent, but longer-term outcomes have historically been unchanged. This study used data from the national Scientific Registry of Transplant Recipients (SRTR) and evaluated 1-year and 5-year graft survival and half-lives for kidney transplant recipients in the US. All adult (≥18 years) solitary kidney transplants (n = 331,216) from 1995 to 2017 were included in the analysis. Mean age was 49.4 years (SD +/-13.7), 60% male, and 25% Black. The overall (deceased and living donor) adjusted hazard of graft failure steadily decreased from 0.89 (95%CI: 0.88, 0.91) in era 2000-2004 to 0.46 (95%CI: 0.45, 0.47) for era 2014-2017 (1995-1999 as reference). Improvements in adjusted hazards of graft failure were more favorable for Blacks, diabetics and older recipients. Median survival for deceased donor transplants increased from 8.2 years in era 1995-1999 to an estimated 11.7 years in the most recent era. Living kidney donor transplant median survival increased from 12.1 years in 1995-1999 to an estimated 19.2 years for transplants in 2014-2017. In conclusion, these data show continuous improvement in long-term outcomes with more notable improvement among higher-risk subgroups, suggesting a narrowing in the gap for those disadvantaged after transplantation.

Current status: meeting the regulatory goals of your liver transplant program

PURPOSE OF REVIEW

The regulatory framework set by the Organ Procurement and Transplantation Network (OPTN) and Center for Medicare and Medicaid Services (CMS) for practice of liver transplantation in US is periodically updated and risk adjusted. Therefore, it is prudent for transplant centers to know the rules of engagement as it pertains to their practice.

RECENT FINDINGS

OPTN besides providing the regulatory oversight for safe and continued practice of transplant centers, provides necessary tools like: advanced statistical models and technological platforms to aid, and guide transplant centers including the necessary safeguards for high-quality transplant care.CMS regulations although had different thresholds to flag underperformance, often covered common grounds similar to the OPTN, therefore considered duplicative and unnecessary. But with much deliberation and consideration CMS undertook a major overhaul to the final rule for re-approval applications, a giant leap in the positive direction for transplant innovation and growth.

SUMMARY

The duplicative regulatory framework of OPTN and CMS has although achieved the goal of improving 1-year patient outcomes, it has proven costly in terms of slowing innovation, increasing organ discard and stunting growth of transplant volume. But the new updates in effect and also in the pipeline are a long-awaited opportunity for waiting transplant patients.

Mortality and Access to Kidney Transplantation in Patients with Sickle Cell Disease-Associated Kidney Failure

BACKGROUND AND OBJECTIVES

Patients with sickle cell disease-associated kidney failure have high mortality, which might be lowered by kidney transplantation. However, because they show higher post-transplant mortality compared with patients with other kidney failure etiologies, kidney transplantation remains controversial in this population, potentially limiting their chance of receiving transplantation. We aimed to quantify the decrease in mortality associated with transplantation in this population and determine the chance of receiving transplantation with sickle cell disease as the cause of kidney failure as compared with other etiologies of kidney failure.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

Using a national registry, we studied all adults with kidney failure who began maintenance dialysis or were added to the kidney transplant waiting list in 1998-2017. To quantify the decrease in mortality associated with transplantation, we measured the absolute risk difference and hazard ratio for mortality in matched pairs of transplant recipients versus waitlisted candidates in the sickle cell and control groups. To compare the chance of receiving transplantation, we estimated hazard ratios for receiving transplantation in the sickle cell and control groups, treating death as a competing risk.

RESULTS

Compared with their matched waitlisted candidates, 189 transplant recipients with sickle cell disease and 220,251 control recipients showed significantly lower mortality. The absolute risk difference at 10 years post-transplant was 20.3 (98.75% confidence interval, 0.9 to 39.8) and 19.8 (98.75% confidence interval, 19.2 to 20.4) percentage points in the sickle cell and control groups, respectively. The hazard ratio was also similar in the sickle cell (0.57; 95% confidence interval, 0.36 to 0.91) and control (0.54; 95% confidence interval, 0.53 to 0.55) groups (interaction P=0.8). Nonetheless, the sickle cell group was less likely to receive transplantation than the controls (subdistribution hazard ratio, 0.73; 95% confidence interval, 0.61 to 0.87). Similar disparities were found among waitlisted candidates (subdistribution hazard ratio, 0.62; 95% confidence interval, 0.53 to 0.72).

CONCLUSIONS

Patients with sickle cell disease-associated kidney failure exhibited similar decreases in mortality associated with kidney transplantation as compared with those with other kidney failure etiologies. Nonetheless, the sickle cell population was less likely to receive transplantation, even after waitlist registration.

Now is the time for the Organ Procurement and Transplantation Network to change regulatory policy to effectively increase transplantation in the United States; Carpe Diem

With the Centers for Medicare and Medicaid Services proposing to remove outcome measures from the transplant centers' renewal for Conditions of Participation an exciting opportunity surfaces for the Organ Procurement and Transplantation Network to make an equally bold change and allow for increased transplantation options for patients in the United States.

Time for reform in transplant program-specific reporting: AST/ASTS transplant metrics taskforce

In accordance with the National Organ Transplant Act and Department of Health and Human Services' Final Rule, the Scientific Registry of Transplant Recipients (SRTR) publicly releases biannual program-specific reports that include analyses of transplant centers' risk-adjusted waitlist mortality, organ acceptance ratios, transplant rates, and graft and patient survival. Since the inception of these center metrics, 1-year posttransplant graft and patient survival have improved, and center variation has decreased, casting uncertainty on their clinical relevance. The SRTR has recently modified center evaluations by ranking centers into 5 tiers rather than 3 tiers in an attempt to discriminate between programs performing within a tight range, further exacerbating this uncertainty. The American Society of Transplantation/American Society of Transplant Surgeons convened an expert taskforce to examine both the utility and unintended consequences of transplant center metrics. Estimates of center variation in outcomes in adjacent tiers are imprecise and fleeting, but can result in consequential changes in clinician and center behavior. The taskforce has concerns that current metrics, based principally on 1-year graft and patient survival, provide minimal if any benefit in informing patient choice and access to transplantation, with the untoward effect of decreased utilization of organs and restriction of research and innovation.

Dramatic secular changes in prognosis for kidney transplant candidates in the United States

Over recent decades, numerous clinical advances and policy changes have affected outcomes for candidates of kidney transplantation in the United States. We examined the national Scientific Registry for Transplant Recipients for adult (18+) solitary kidney transplant candidates placed on the waiting list for primary listing from 2001 to 2015. We evaluated rates of mortality, transplantation, and waitlist removal. Among 340 115 candidates there were significant declines in mortality (52 deaths/1000 patient years in 2001-04 vs 38 deaths/1000 patient years in 2012-15) and transplant rates (304 transplants/1000 patient years in 2001-04 vs 212 transplants/1000 patient years in 2012-15) and increases in waitlist removals (15 removals/1000 patient years in 2001-04 vs 25/1000 patient years in 2012-15) within the first year after listing. At 5 years an estimated 37% of candidates listed in 2012-15 were alive without transplant as compared to 22% in 2001-04. Declines in mortality over time were significantly more pronounced among African Americans, candidates with longer dialysis duration, and those with diabetes (P < .001). Cumulatively, results indicate dramatic changes in prognoses for adult kidney transplant candidates, likely impacted by selection criteria, donor availability, regulatory oversight, and clinical care. These trends are important considerations for prospective policy development and research, clinical and patient decision-making, and evaluating the impact on access to care.

Poor Performance Flagging Is Associated With Fewer Transplantations at Centers Flagged Multiple Times

BACKGROUND

Lung transplantation outcomes are heavily scrutinized, given the high stakes of these operations, yet the Center for Medicare and Medicaid Services (CMS) method of using Scientific Registry of Transplant Recipients (SRTR) risk-adjusted outcomes to identify underperforming centers is controversial. We hypothesized that CMS flagging results in conservative behavior for recipient and organ selection, resulting in fewer patients added to the waitlist and fewer transplantations performed.

METHODS

SRTR reports from July 2012 through July 2017 were included. Center characteristics were compared, stratified by number of flagging events. The impact of flagging for underperformance on risk aversion outcomes was analyzed using a mixed-effects regression model.

RESULTS

A total of 72 centers had reported SRTR data during the study period. Of these, 21 centers (29%) met flagging criteria a median of 2 times (interquartile range, 1 to 4 times) for a total of 53 events. Flagging had no statistically significant impact on waitlist or transplantation volume and patient selection by mixed-effects modeling. Despite similar average expected 1-year survival (86.6% versus 87.7%, p = 0.27), centers that were flagged only once added more patients per year to the waitlist (16.3 patients versus 7.8 patients, p = 0.01) and performed more transplantations per year (28.4 transplantations versus 11.1 transplantations, p = 0.01).

CONCLUSIONS

This analysis defines center-level trends in lung transplantation after CMS flagging. Contrary to our primary hypothesis, flagging did not result in temporal center-level changes. However, programs on prolonged probation demonstrated reduced activity, which likely indicates a shift to higher performing centers.

Expanding clarity or confusion? Volatility of the 5-tier ratings assessing quality of transplant centers in the United States

Outcomes of patients receiving solid organ transplants in the United States are systematically aggregated into bi-annual Program-Specific Reports (PSRs) detailing risk-adjusted survival by transplant center. Recently, the Scientific Registry of Transplant Recipients (SRTR) issued 5-tier ratings evaluating centers based on risk-adjusted 1-year graft survival. Our primary aim was to examine the reliability of 5-tier ratings over time. Using 10 consecutive PSRs for adult kidney transplant centers from June 2012 to December 2016 (n = 208), we applied 5-tier ratings to center outcomes and evaluated ratings over time. From the baseline period (June 2012), 47% of centers had at least a 1-unit tier change within 6 months, 66% by 1 year, and 94% by 3 years. Similarly, 46% of centers had at least a 2-unit tier change by 3 years. In comparison, 15% of centers had a change in the traditional 3-tier rating at 3 years. The 5-tier ratings at 4 years had minimal association with baseline rating (Kappa 0.07, 95% confidence interval [CI] -0.002 to 0.158). Centers had a median of 3 different 5-tier ratings over the period (q1 = 2, q3 = 4). Findings were consistent for center volume, transplant rate, and baseline 5-tier rating. Cumulatively, results suggest that 5-tier ratings are highly volatile, limiting their utility for informing potential stakeholders, particularly transplant candidates given expected waiting times between wait listing and transplantation.

Evaluation of Flagging Criteria of United States Kidney Transplant Center Performance: How to Best Define Outliers?

BACKGROUND

Scientific Registry of Transplant Recipients report cards of US organ transplant center performance are publicly available and used for quality oversight. Low center performance (LP) evaluations are associated with changes in practice including reduced transplant rates and increased waitlist removals. In 2014, Scientific Registry of Transplant Recipients implemented new Bayesian methodology to evaluate performance which was not adopted by Center for Medicare and Medicaid Services (CMS). In May 2016, CMS altered their performance criteria, reducing the likelihood of LP evaluations.

METHODS

Our aims were to evaluate incidence, survival rates, and volume of LP centers with Bayesian, historical (old-CMS) and new-CMS criteria using 6 consecutive program-specific reports (PSR), January 2013 to July 2015 among adult kidney transplant centers.

RESULTS

Bayesian, old-CMS and new-CMS criteria identified 13.4%, 8.3%, and 6.1% LP PSRs, respectively. Over the 3-year period, 31.9% (Bayesian), 23.4% (old-CMS), and 19.8% (new-CMS) of centers had 1 or more LP evaluation. For small centers (<83 transplants/PSR), there were 4-fold additional LP evaluations (52 vs 13 PSRs) for 1-year mortality with Bayesian versus new-CMS criteria. For large centers (>183 transplants/PSR), there were 3-fold additional LP evaluations for 1-year mortality with Bayesian versus new-CMS criteria with median differences in observed and expected patient survival of -1.6% and -2.2%, respectively.

CONCLUSIONS

A significant proportion of kidney transplant centers are identified as low performing with relatively small survival differences compared with expected. Bayesian criteria have significantly higher flagging rates and new-CMS criteria modestly reduce flagging. Critical appraisal of performance criteria is needed to assess whether quality oversight is meeting intended goals and whether further modifications could reduce risk aversion, more efficiently allocate resources, and increase transplant opportunities.

Measuring transplant center performance: The goals are not controversial but the methods and consequences can be

PURPOSE OF REVIEW

Risks of regulatory scrutiny has generated widespread concern about increasingly risk averse transplant center behaviors regarding both donor and candidate acceptance patterns. To address potential unintended consequences threatening access to care, we discuss recent changes in regulatory metrics and potential improvements in quality oversight of transplant centers.

RECENT FINDINGS

Despite many recent changes to one-year patient and graft survival regulatory criteria, the capacity to accurately identify true underperforming centers and avoiding false positive flagging remains an area of great concern. Numerous studies have demonstrated restrictions in transplant volume and access following transplant center flagging.

SUMMARY

Current regulatory criteria are limited in their capacity to accurately identify poorly performing centers and potentially encourage risk-averse behavior by transplant centers. Efforts to address these concerns should focus on (1) improving risk-adjustment models with better data which captures the acuity of candidate and donor risk, (2) reconsidering primary outcomes measured to assess comprehensive transplant center performance, (3) improving education to address rational or perceived disincentives, and (4) using data more effectively to share best practices.

Do Federal Regulations Have an Impact on Kidney Transplant Outcomes?

Transplantation is one of the most highly regulated fields in health care. An important component of transplant oversight is the performance assessment of transplant centers as measured by 1-year patient and graft survival outcomes. The use of the Organ Procurement and Transplantation Network and Scientific Registry of Transplant Recipients flagging mechanism for quality improvement as criteria for Center for Medicare and Medicaid Services certification has resulted in greater importance in transplant program operations. Although supporters of this program of encouraging Quality Assurance and Performance Improvement point to improved survival outcomes for more than the decade, others assert that the oversight is unnecessarily punitive, results in tremendous resource utilization, and discourages innovation. Data exist to support an inhibitory effect on national transplant volume. Although survival outcomes are risk adjusted, limitations on national data collection prevent several important risks from being incorporated into the models. This has led to the consensus that many transplant centers have become increasingly risk averse in this environment, which may indirectly reduce access to transplant for candidates who could still benefit from transplantation. Recently enacted modifications to performance evaluation by Center for Medicare and Medicaid Services and the Organ Procurement and Transplantation Network appear to acknowledge these concerns and have the potential to recalibrate transplant center focus away from first-year outcomes and more toward expanding transplant volume, innovation, and overall improvements in care.

Decade-Long Trends in Liver Transplant Waitlist Removal Due to Illness Severity: The Impact of Centers for Medicare and Medicaid Services Policy

BACKGROUND

The central tenet of liver transplant organ allocation is to prioritize the sickest patients first. However, a 2007 Centers for Medicare and Medicaid Services regulatory policy, Conditions of Participation (COP), which mandates publically reported transplant center performance assessment and outcomes-based auditing, critically altered waitlist management and clinical decision making. We examine the extent to which COP implementation is associated with increased removal of the "sickest" patients from the liver transplant waitlist.

STUDY DESIGN

This study included 90,765 adult (aged 18 years and older) deceased donor liver transplant candidates listed at 102 transplant centers from April 2002 through December 2012 (Scientific Registry of Transplant Recipients). We quantified the effect of COP implementation on trends in waitlist removal due to illness severity and 1-year post-transplant mortality using interrupted time series segmented Poisson regression analysis.

RESULTS

We observed increasing trends in delisting due to illness severity in the setting of comparable demographic and clinical characteristics. Delisting abruptly increased by 16% at the time of COP implementation, and likelihood of being delisted continued to increase by 3% per quarter thereafter, without attenuation (p < 0.001). Results remained consistent after stratifying on key variables (ie, Model for End-Stage Liver Disease and age). The COP did not significantly impact 1-year post-transplant mortality (p = 0.38).

CONCLUSIONS

Although the 2007 Centers for Medicare and Medicaid Services COP policy was a quality initiative designed to improve patient outcomes, in reality, it failed to show beneficial effects in the liver transplant population. Patients who could potentially benefit from transplantation are increasingly being denied this lifesaving procedure while transplant mortality rates remain unaffected. Policy makers and clinicians should strive to balance candidate and recipient needs from a population-benefit perspective when designing performance metrics and during clinical decision making for patients on the waitlist.

Variation in biliary complication rates following liver transplantation: implications for cost and outcome

Although biliary complications (BCs) have a significant impact on the outcome of liver transplantation (LT), variation in BC rates among transplant centers has not been previously analyzed. BC rate, LT outcome and spending were assessed using linked Scientific Registry of Transplant Recipients and Medicare claims (n = 16,286 LTs). Transplant centers were assigned to BC quartiles based upon risk-adjusted observed to expected (O:E) ratio of BC separately for donation after brain death (DBD) and donation after cardiac death (DCD) donors. The median incidence of BC was 300% greater in the highest versus lowest DBD quartiles (19.0% vs. 5.9%) and varied 250% between DCD quartiles (20.3%-8.4%). Donor and recipient characteristics suggest that high BC centers actually used lower donor risk index organs, fewer split livers and fewer imports (p < 0.001 for all). Transplant at a center in the highest O:E quartile was associated with increased posttransplant mortality (adjusted hazard ratio [aHR] 2.53, p = 0.007) in DCD transplant and increased graft loss (aHR 1.21, p = 0.02) in DBD transplant. Medicare spending was $22,895 (p < 0.0001) higher at centers in highest versus lowest BC quartile. In summary, BC rates vary widely among transplant centers and higher rates are a marker for an increased risk of death, graft failure and health-care spending.

Association between kidney transplant center performance and the survival benefit of transplantation versus dialysis

BACKGROUND AND OBJECTIVES

Despite the benefits of kidney transplantation, the total number of transplants performed in the United States has stagnated since 2006. Transplant center quality metrics have been associated with a decline in transplant volume among low-performing centers. There are concerns that regulatory oversight may lead to risk aversion and lack of transplantation growth.

DESIGN, SETTING, PARTICIPANTS, & MEASUREMENTS

A retrospective cohort study of adults (age≥18 years) wait-listed for kidney transplantation in the United States from 2003 to 2010 using the Scientific Registry of Transplant Recipients was conducted. The primary aim was to investigate whether measured center performance modifies the survival benefit of transplantation versus dialysis. Center performance was on the basis of the most recent Scientific Registry of Transplant Recipients evaluation at the time that patients were placed on the waiting list. The primary outcome was the time-dependent adjusted hazard ratio of death compared with remaining on the transplant waiting list.

RESULTS

Among 223,808 waitlisted patients, 59,199 and 32,764 patients received a deceased or living donor transplant, respectively. Median follow-up from listing was 43 months (25th percentile=25 months, 75th percentile=67 months), and there were 43,951 total patient deaths. Deceased donor transplantation was independently associated with lower mortality at each center performance level compared with remaining on the waiting list; adjusted hazard ratio was 0.24 (95% confidence interval, 0.21 to 0.27) among 11,972 patients listed at high-performing centers, adjusted hazard ratio was 0.32 (95% confidence interval, 0.31 to 0.33) among 203,797 patients listed at centers performing as expected, and adjusted hazard ratio was 0.40 (95% confidence interval, 0.35 to 0.45) among 8039 patients listed at low-performing centers. The survival benefit was significantly different by center performance (P value for interaction <0.001).

CONCLUSIONS

Findings indicate that measured center performance modifies the survival benefit of kidney transplantation, but the benefit of transplantation remains highly significant even at centers with low measured quality. Policies that concurrently emphasize improved center performance with access to transplantation should be prioritized to improve ESRD population outcomes.

Association between liver transplant center performance evaluations and transplant volume

There has been increased oversight of transplant centers and stagnation in liver transplantation nationally in recent years. We hypothesized that centers that received low performance (LP) evaluations were more likely to alter protocols, resulting in reduced rates of transplants and patients placed on the waiting list. We evaluated the association of LP evaluations and transplant activity among liver transplant centers in the United States using national Scientific Registry of Transplant Recipients data (January 2007 to July 2012). We compared the average change in recipient and candidate volume and donor and patient characteristics based on whether the centers received LP evaluations. Of 92 eligible centers, 27 (29%) received at least one LP evaluation. Centers without an LP evaluation (n = 65) had an average increase of 9.3 transplants and 14.9 candidates while LP centers had an average decrease of 39.9 transplants (p < 0.01) and 67.3 candidates (p < 0.01). LP centers reduced the use of older donors, donations with longer cold ischemia, and donations after cardiac death (p-values < 0.01). There was no association between the change in transplant volume and measured performance (R(2)  = 0.002, p = 0.91). Findings indicate a strong association between performance evaluations and changes in candidate listings and transplants among liver transplant centers, with no measurable improvement in outcomes associated with reduction in transplant volume.

Big data in organ transplantation: registries and administrative claims

The field of organ transplantation benefits from large, comprehensive, transplant-specific national data sets available to researchers. In addition to the widely used Organ Procurement and Transplantation Network (OPTN)-based registries (the United Network for Organ Sharing and Scientific Registry of Transplant Recipients data sets) and United States Renal Data System (USRDS) data sets, there are other publicly available national data sets, not specific to transplantation, which have historically been underutilized in the field of transplantation. Of particular interest are the Nationwide Inpatient Sample and State Inpatient Databases, produced by the Agency for Healthcare Research and Quality. The USRDS database provides extensive data relevant to studies of kidney transplantation. Linkage of publicly available data sets to external data sources such as private claims or pharmacy data provides further resources for registry-based research. Although these resources can transcend some limitations of OPTN-based registry data, they come with their own limitations, which must be understood to avoid biased inference. This review discusses different registry-based data sources available in the United States, as well as the proper design and conduct of registry-based research.

Quantifying the risk of incompatible kidney transplantation: a multicenter study

Incompatible live donor kidney transplantation (ILDKT) offers a survival advantage over dialysis to patients with anti-HLA donor-specific antibody (DSA). Program-specific reports (PSRs) fail to account for ILDKT, placing this practice at regulatory risk. We collected DSA data, categorized as positive Luminex, negative flow crossmatch (PLNF) (n = 185), positive flow, negative cytotoxic crossmatch (PFNC) (n = 536) or positive cytotoxic crossmatch (PCC) (n = 304), from 22 centers. We tested associations between DSA, graft loss and mortality after adjusting for PSR model factors, using 9669 compatible patients as a comparison. PLNF patients had similar graft loss; however, PFNC (adjusted hazard ratio [aHR] = 1.64, 95% confidence interval [CI]: 1.15-2.23, p = 0.007) and PCC (aHR = 5.01, 95% CI: 3.71-6.77, p < 0.001) were associated with increased graft loss in the first year. PLNF patients had similar mortality; however, PFNC (aHR = 2.04; 95% CI: 1.28-3.26; p = 0.003) and PCC (aHR = 4.59; 95% CI: 2.98-7.07; p < 0.001) were associated with increased mortality. We simulated Centers for Medicare & Medicaid Services flagging to examine ILDKT's effect on the risk of being flagged. Compared to equal-quality centers performing no ILDKT, centers performing 5%, 10% or 20% PFNC had a 1.19-, 1.33- and 1.73-fold higher odds of being flagged. Centers performing 5%, 10% or 20% PCC had a 2.22-, 4.09- and 10.72-fold higher odds. Failure to account for ILDKT's increased risk places centers providing this life-saving treatment in jeopardy of regulatory intervention.