Can donor narratives yield insights? A natural language processing proof of concept to facilitate kidney allocation

Prediction of High-Risk Donors for Kidney Discard and Nonrecovery Using Structured Donor Characteristics and Unstructured Donor Narratives

Importance

Despite the unmet need, many deceased-donor kidneys are discarded or not recovered. Inefficient allocation and prolonged ischemia time are contributing factors, and early detection of high-risk donors may reduce organ loss.

Objective

To evaluate the feasibility of machine learning (ML) and natural language processing (NLP) classification of donors with kidneys that are used vs not used for organ transplant.

Design, Setting, and Participants

This retrospective cohort study used donor information (structured donor characteristics and unstructured donor narratives) from the United Network for Organ Sharing (UNOS). All donor offers to a single transplant center between January 2015 and December 2020 were used to train and validate ML models to predict donors who had at least 1 kidney transplanted (at our center or another center). The donor data from 2021 were used to test each model.

Exposures

Donor information was provided by UNOS to the transplant centers with potential transplant candidates. Each center evaluated the donor and decided within an allotted time whether to accept the kidney for organ transplant.

Main Outcomes and Measures

Outcome metrics of the test cohort included area under the receiver operating characteristic curve (AUROC), F1 score, accuracy, precision, and recall of each ML classifier. Feature importance and Shapley additive explanation (SHAP) summaries were assessed for model explainability.

Results

The training/validation cohort included 9555 donors (median [IQR] age, 50 [36-58] years; 5571 male [58.3%]), and the test cohort included 2481 donors (median [IQR] age, 52 [40-59] years; 1496 male [60.3%]). Only 20% to 30% of potential donors had at least 1 kidney transplanted. The ML model with a single variable (Kidney Donor Profile Index) showed an AUROC of 0.69, F1 score of 0.42, and accuracy of 0.64. Multivariable ML models based on basic a priori structured donor data showed similar metrics (logistic regression: AUROC = 0.70; F1 score = 0.42; accuracy = 0.62; random forest classifier: AUROC = 0.69; F1 score = 0.42; accuracy = 0.64). The classic NLP model (bag-of-words model) showed its best metrics (AUROC = 0.60; F1 score = 0.35; accuracy = 0.59) by the logistic regression classifier. The advanced Bidirectional Encoder Representations From Transformers model showed comparable metrics (AUROC = 0.62; F1 score = 0.39; accuracy = 0.69) only after appending basic donor information. Feature importance and SHAP detected the variables (and words) that affected the models most.

Conclusions and Relevance

Results of this cohort study suggest that models using ML can be applied to predict donors with high-risk kidneys not used for organ transplant, but the models still need further elaboration. The use of unstructured data is likely to expand the possibilities; further exploration of new approaches will be necessary to develop models with better predictive metrics.

The promise of machine learning applications in solid organ transplantation

Solid-organ transplantation is a life-saving treatment for end-stage organ disease in highly selected patients. Alongside the tremendous progress in the last several decades, new challenges have emerged. The growing disparity between organ demand and supply requires optimal patient/donor selection and matching. Improvements in long-term graft and patient survival require data-driven diagnosis and management of post-transplant complications. The growing abundance of clinical, genetic, radiologic, and metabolic data in transplantation has led to increasing interest in applying machine-learning (ML) tools that can uncover hidden patterns in large datasets. ML algorithms have been applied in predictive modeling of waitlist mortality, donor–recipient matching, survival prediction, post-transplant complications diagnosis, and prediction, aiming to optimize immunosuppression and management. In this review, we provide insight into the various applications of ML in transplant medicine, why these were used to evaluate a specific clinical question, and the potential of ML to transform the care of transplant recipients. 36 articles were selected after a comprehensive search of the following databases: Ovid MEDLINE; Ovid MEDLINE Epub Ahead of Print and In-Process & Other Non-Indexed Citations; Ovid Embase; Cochrane Database of Systematic Reviews (Ovid); and Cochrane Central Register of Controlled Trials (Ovid). In summary, these studies showed that ML techniques hold great potential to improve the outcome of transplant recipients. Future work is required to improve the interpretability of these algorithms, ensure generalizability through larger-scale external validation, and establishment of infrastructure to permit clinical integration.

Using Operative Reports to Predict Heart Transplantation Survival

Heart transplantation is a difficult procedure compared with other surgical operations, with a greater outcome uncertainty such as late rejection and death. We can model the success of heart transplants from predicting factors such as the age, sex, diagnosis, etc., of the donor and recipient. Although predictions can mitigate the uncertainty on the transplantation outcome, their accuracy is far from perfect. In this paper, we describe a new method to predict the outcome of a transplantation from textual operative reports instead of traditional tabular data. We carried out an experiment on 300 surgical reports to determine the survival rates at one year and five years. Using a truncated TF-IDF vectorization of the texts and logistic regression, we could reach a macro Fl of 59.1 %, respectively, 54.9% with a five-fold cross validation. While the size of the corpus is relatively small, our experiments show that the operative textual sources can discriminate the transplantation outcomes and could be a valuable additional input to existing prediction systems. Clinical Relevance- Heart transplantation involves a significant number of written reports including in the preoperative examinations and operative documentation. In this paper, we show that these written reports can predict the outcome of the transplantation at one and five years with macro 1s of 59.1 % and 54.9 %, respectively and complement existing prediction methods.

Hardest-to-place kidney transplant outcomes in the United States

The outcomes of hardest-to-place kidney transplants-accepted last in the entire match run after being refused by previous centers-are unclear, potentially translating to risk aversion and unnecessary organ discard. We aimed to determine the outcomes of hardest-to-place kidney transplants and whether the organ acceptance position on the match run sufficiently captures the risk. This is a cohort study of the United Network for Organ Sharing data of all adult kidney-only transplant recipients from deceased donors between 2007 and 2018. Multiple regression models assessed delayed graft function, graft survival, and patient survival stratified by share type: local versus shared kidney acceptance position scaled by tertile. Among 127 028 kidney transplant recipients, 92 855 received local kidneys. The remaining received shared kidneys at sequence number 1-4 (n = 12 322), 5-164 (n = 10 485) and >164 (n = 11 366). Hardest-to-place kidneys, defined as the latest acceptance group in the match-run, were associated with delayed graft function (adjusted odds ratio 1.83, 95% confidence interval [CI] 1.74-1.92) and all-cause allograft failure (adjusted hazard ratio [aHR] 1.11, 95% CI 1.04-1.17). Results of this IRB-approved study were robust to the exclusion of operational allocation bypass and mandatory shares. The hardest-to-place kidneys accepted later in the match run were associated with higher graft failure and delayed graft function.