Toward structuring real-world data: Deep learning for extracting oncology information from clinical text with patient-level supervision

Most detailed patient information in real-world data (RWD) is only consistently available in free-text clinical documents. Manual curation is expensive and time consuming. Developing natural language processing (NLP) methods for structuring RWD is thus essential for scaling real-world evidence generation. We propose leveraging patient-level supervision from medical registries, which are often readily available and capture key patient information, for general RWD applications. We conduct an extensive study on 135,107 patients from the cancer registry of a large integrated delivery network (IDN) comprising healthcare systems in five western US states. Our deep-learning methods attain test area under the receiver operating characteristic curve (AUROC) values of 94%-99% for key tumor attributes and comparable performance on held-out data from separate health systems and states. Ablation results demonstrate the superiority of these advanced deep-learning methods. Error analysis shows that our NLP system sometimes even corrects errors in registrar labels.

Abstract 5618: Multi-institutional validation of a radiomics-based artificial intelligence method for predicting response to PD-1/PD-L1 immune checkpoint inhibitor (ICI) therapy in stage IV NSCLC

There is an urgent clinical need to identify patients likely to benefit from immune checkpoint inhibitor ICI treatment. Approaches available in the clinic today, such as PD-L1 immunohistochemistry (IHC) and tumor mutation burden (TMB), are insufficient for this task, in part as differences in microenvironments expressed by individual tumors may lead to heterogeneous response patterns. Recent efforts exploring the utility of quantitative imaging (radiomic) biomarkers to predict response to ICIs have shown promise to provide a more accurate and scalable method. In contrast to previously published models, our work focuses on generalizable models for predicting individual lesion-level as well as patient-level response at 3-month follow-up per RECIST criteria, using a large multi-institutional “real-world” dataset. The models combine radiomics features with demographic, molecular, and laboratory values routinely available in patients’ electronic medical records. We analyzed radiomic characteristics of 6,295 primary and metastatic lesions from 1,206 metastatic NSCLC patients treated with anti-PD-1/anti-PD-L1 ICIs from 8 institutions across the US and Europe. Patients with unavailable PD-L1 IHC, imaging follow-up, or with oncogenic driver mutations were excluded from analysis, resulting in a total dataset of 766 subjects randomly assigned to training (N=514) and validation sets (N=252). Using gradient-boosted decision tree algorithms, we developed a multi-modal predictive model to identify patients responding to ICI therapy at 3-months and evaluated its performance against an imaging-only CT radiomics model and the clinical standard of care, biopsy-based PD-L1 IHC. The multi-modal model contains CT radiomic features capturing lesion heterogeneity and spicularity, patient demographics, PD-L1 TPS, and tumor burden volume in the lung, lymph nodes, and the liver. Under the two-tailed DeLong test, the multi-modal model demonstrated statistically significant benefit over the current standard of care (PD-L1 IHC) in predicting multi-lesion 3-month response: 0.81 (P=.005) area under the receiver operating characteristic curve (ROC-AUC) in first-line ICI monotherapy patients, 0.72 (P=.044) in all-lines ICI monotherapy, and 0.71 (P=.025) in all-lines ICI-chemotherapy combination. The imaging-only model demonstrated predictive performance comparable to PD-L1 IHC: 0.71 (P=.226), 0.61 (P=.905), 0.62 (P=.674) on the same cohorts respectively. A multi-modal CT radiomics-based approach demonstrated predictive accuracy benefit over the current clinical standard and may provide an opportunity for more personalized patient management, such as risk-based escalation/de-escalation of concurrent chemotherapy in NSCLC patients. We will evaluate this methodology in prospective studies.

Citation Format: Ravi B. Parikh, Petr Jordan, Rita J. Ciaravino, Ryan A. Beasley, Arpan A. Patel, Dwight H. Owen, Arya Amini, Brendan D. Curti, Ray Page, Aurelie Swalduz, Jean-Paul Beregi, Jan Chrusciel, Eric Snyder, Pritam Mukherjee, Heather M. Selby, Soohee Lee, Roshanthi Weerasinghe, Shwetha Pindikuri, Jakob B. Weiss, Andrew L. Wentland, Anish Kirpalani, An Liu, Olivier Gevaert, George Simon, Hugo JWL Aerts. Multi-institutional validation of a radiomics-based artificial intelligence method for predicting response to PD-1/PD-L1 immune checkpoint inhibitor (ICI) therapy in stage IV NSCLC. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 5618.

A Random Forest Genomic Classifier for Tumor Agnostic Prediction of Response to Anti-PD1 Immunotherapy

Tumor mutational burden (TMB), a surrogate for tumor neoepitope burden, is used as a pan-tumor biomarker to identify patients who may benefit from anti-program cell death 1 (PD1) immunotherapy, but it is an imperfect biomarker. Multiple additional genomic characteristics are associated with anti-PD1 responses, but the combined predictive value of these features and the added informativeness of each respective feature remains unknown. We evaluated whether machine learning (ML) approaches using proposed determinants of anti-PD1 response derived from whole exome sequencing (WES) could improve prediction of anti-PD1 responders over TMB alone. Random forest classifiers were trained on publicly available anti-PD1 data (n = 104), and subsequently tested on an independent anti-PD1 cohort (n = 69). Both the training and test datasets included a range of cancer types such as non-small cell lung cancer (NSCLC), head and neck squamous cell carcinoma (HNSCC), melanoma, and smaller numbers of patients from other tumor types. Features used include summaries such as TMB and number of frameshift mutations, as well as more gene-level features such as counts of mutations associated with immune checkpoint response and resistance. Both ML algorithms demonstrated area under the receiver-operator curves (AUC) that exceeded TMB alone (AUC 0.63 "human-guided," 0.64 "cluster," and 0.58 TMB alone). Mutations within oncogenes disproportionately modulate anti-PD1 responses relative to their overall contribution to tumor neoepitope burden. The use of a ML algorithm evaluating multiple proposed genomic determinants of anti-PD1 responses modestly improves performance over TMB alone, highlighting the need to integrate other biomarkers to further improve model performance.

Genomic analysis of early-stage lung cancer reveals a role for TP53 mutations in distant metastasis

Patients with non-small cell lung cancer (NSCLC) who have distant metastases have a poor prognosis. To determine which genomic factors of the primary tumor are associated with metastasis, we analyzed data from 759 patients originally diagnosed with stage I-III NSCLC as part of the AACR Project GENIE Biopharma Collaborative consortium. We found that TP53 mutations were significantly associated with the development of new distant metastases. TP53 mutations were also more prevalent in patients with a history of smoking, suggesting that these patients may be at increased risk for distant metastasis. Our results suggest that additional investigation of the optimal management of patients with early-stage NSCLC harboring TP53 mutations at diagnosis is warranted in light of their higher likelihood of developing new distant metastases.

Biodeterioration of low density polyethylene by mangrove associated endolichenic fungi and their enzymatic regimes

Fungal involvement in biodeterioration of Low Density Polyethylene (LDPE) has received a great attention in recent years. Among diverse groups of fungi, Endolichenic Fungi (ELF) are adapted to thrive in resource limited conditions. Present study was designed to investigate the potential of mangrove associated ELF, in biodeterioration of LDPE and to quantify key-depolymerizing enzymes. A total of 31 ELF species, isolated from 22 lichens of mangrove ecosystems in Negombo lagoon, Sri Lanka were identified using DNA barcoding techniques. ELF were inoculated into mineral salt medium, containing LDPE strips and incubated at 28±2°C, for 21 days, under laboratory conditions. After incubation, biodeterioration was monitored based on percent reductions in weights and tensile properties, increments in degree of water absorption, changes in peaks of Infrared spectra and surface erosions using Scanning Electron Microscopy. Out of 31 species, Chaetomium globosum, Daldinia eschscholtzii, Neofusicoccum occulatum, Phanerochaete chrysosporium, Schizophyllum commune and Xylaria feejeensis showed significant changes. Production of depolymerizing enzymes by these species, were assayed qualitatively using plate-based methods and quantitatively by mass level enzyme production. Among them Phanerochaete chrysosporium showed the highest enzyme activities as (9.69±0.04)x10^-3 , (1.96±0.01)x10^-3 , (5.73±0.03)x10^-3 , (0.88±0.01), (0.64±0.06), (1.43±0.01) U ml^-1 for laccase, lignin peroxidase, manganese peroxidase, amylase, lipase and esterase, respectively.

SAT0219 PATIENT CHARACTERISTICS, TREATMENT PATTERNS, AND RESOURCE UTILIZATION OF SJOGREN’S SYNDROME PATIENTS IN A LARGE US HEALTH NETWORK

Background:

Sjogren’s syndrome (SS) is a chronic progressive autoimmune disorder which occurs as primary (pSS) or secondary SS (sSS). With no approved disease modifying therapy, there is limited information on the treatment patterns and resource utilization among these patients (pts).

Objectives:

To describe pts characteristics, treatment patterns and healthcare resource utilization (HCRU) using electronic health records (EHR) of pts with pSS and sSS treated at the Providence St. Josephs Health system (PSJH).

Methods:

Pts ≥18 years of age with at least one clinical encounter with ICD-9-CM or ICD-10-CM diagnosis of SS, between Jan 2013 and Mar 2019 were included. Date of first encounter with SS diagnosis (index date) was used to assess pts demographics. Pt baseline comorbidities were evaluated during the 24 months pre-index period. Treatment patterns and HCRU were assessed during the 12 months post-index follow-up. Descriptive statistics were used to describe pts’ demographic and clinical characteristics, and medications use in the baseline and follow up.

Results:

Study cohort included 9,108 SS pts of which 76.5% had sSS diagnosis on index date. Majority of SS pts were women, Caucasian, with mean age of 58.3 yrs, and from western states in the US (Table 1). Endocrine conditions including hypo- and hyperthyroidism, and diabetes was the most common (45.5%) comorbidity at baseline, followed by rheumatologic disorders (25.6%) and neurological conditions (22.2%). Among patients with treatment information (4088, 44.88%), 42.95% were using symptomatic treatments for dry eye and mouth at baseline (Table 1). In the follow-up, SS pts had average 5.8 healthcare visits per patient per year (PPPY), including 0.6 inpatient and 3.4 outpatient visit respectively. About 40% of the SS pts (53.8% pSS and 35.8% sSS) were diagnosed by rheumatologists. Majority of the SS pts initiated treatment with cDMARDs (82%) and remained on the same treatment during 1 year follow-up (Fig 2).

Table 1.

Baseline Demographic and Clinical Pts Characteristics

SS Pts (n=9,108)DemographicsAge (years) on index date, mean (SD)58.3 (15.1)Female, n (%)8,338 (91.6)Caucasian, n (%)6.936 (76.2)Western Region, n (%)8,998 (98.8)Married, n (%)5,164 (56.7)Never Smoked, n (%)4,847 (53.2)Primary diagnosis, n (%)2,137 (23.5)Comorbidities, n (%)Cardiovascular1,408 (17.2)Endocrine3,733 (45.5)Oncology800 (9.8)Blood disorders1,221 (14.9)Pulmonary1,802 (22.0)Neurological1,821 (22.2)Liver/Kidney1,782 (21.7)Rheumatologic disorders2,096 (25.6)Autoimmune/ Immune related1,527 (18.6)Baseline Medications, n (%)Symptomatic11,756 (43.0)NSAIDs21,578 (38.6)cDMARDs31,435 (35.1)Corticosteroid41,393 (34.1)bDMARDs5266 (6.5)

1cevimeline, pilocarpine hydrochloride, ophthalmic insert etc;2aspirin, ibuprofen, naproxen;3methotrexate, hydroxychloroquine, sulfasalazine, leflunomide, myophenolate mofetil, azathioprine;4prednisone;5sarilumab, belimumab, ustekinumab, infliximab, adalimumab, certolizumab pegol, golimumab, etanercept, abatacept, tocilizumab, rituximab, tofacitinib, baricitinib

Figure 1.

HCRU for pSS and sSS Pts

Figure 2.

Treatment Sequencing for pSS and sSS Pts. Note: Discontinued: pts who discontinued and didn’t advance to any therapy; same treatment: pts continued on index treatment till we have information.

Conclusion:

Observation of higher comorbidities suggests substantial burden of SS pts on healthcare system, with majority of pts being diagnosed outside of rheumatology offices.

Acknowledgments: :

We acknowledge the contributions of Manasi Suryavanshi towards drafting and reviewing the abstract.

Disclosure of Interests:

Philip J Mease Grant/research support from: Abbott, Amgen, Biogen Idec, BMS, Celgene Corporation, Eli Lilly, Novartis, Pfizer, Sun Pharmaceutical, UCB – grant/research support, Consultant of: Abbott, Amgen, Biogen Idec, BMS, Celgene Corporation, Eli Lilly, Novartis, Pfizer, Sun Pharmaceutical, UCB – consultant, Speakers bureau: Abbott, Amgen, Biogen Idec, BMS, Eli Lilly, Genentech, Janssen, Pfizer, UCB – speakers bureau, Joe Zhuo Shareholder of: Bristol-Myers Squibb, Employee of: Bristol-Myers Squibb, Roshanthi Weerasinghe Grant/research support from:., Qian Xia Shareholder of: I own shares of Bristol-Myers Squibb Company, Employee of: I am a paid employee of Bristol-Myers Squibb Company, Chidananda Samal Consultant of: I work as a consultant for Bristol-Myers Squibb Company, Niyati Sharma Consultant of: I work as a consultant for Bristol-Myers Squibb Company

Can high-volume teams of anesthesiologists and surgeons decrease perioperative costs for pancreatic surgery?

BACKGROUND

Pancreatic surgery outcomes are associated with surgeon and center experience. Anesthesiologists as potential value drivers for pancreatic surgery have not been explored. We sought to evaluate whether anesthesiologists impact perioperative costs for pancreatic surgery.

METHODS

Within an integrated health care system, 796 pancreatic surgeries (526 PDs and 270 DPs) were performed from January 2014 to June 2017. Mean direct operative and anesthesia costs driven by anesthesiologists (operating room (OR) time, anesthesia billing and anesthesia procedures) were determined for each case. The volumes of pancreatic cases per anesthesiologist were calculated, and those above the 75th percentile for volume (4 cases) were considered high-volume. A multivariable analysis of OR/anesthesia costs was performed.

RESULTS

Mean OR and anesthesia costs for PD were $7064 for low-volume anesthesiologists (LVA), higher than $5968 for high-volume anesthesiologists (HVA) (p < 0.001). By multivariable analysis, HVA were associated with decreased costs of $2278 (p < 0.001). Teams of HVA and high-volume surgeons (HVS) were also associated with decreased mean costs of $1790 (p = 0.04).

CONCLUSION

These data suggest that anesthesiologists experienced in the management of complex pancreatic operations such as PDs may contribute to improved efficiencies in care by reducing perioperative costs.

Development of an electronic breast pathology database in a community health system

Background:

Health care systems rely on electronic patient data, yet access to breast tissue pathology results continues to depend on interpreting dictated free-text reports.

Objective:

The objective was to develop a method to electronically search and categorize pathologic diagnoses of patients’ breast tissue specimens from dictated free-text pathology reports in a large health system for multiple users including clinicians.

Design:

A database integrating existing patient-level administrative and clinical information for breast cancer screening and diagnostic services and a web-based application for comprehensive searching of pathology reports were developed by a health system team led by pathologists. The Breast Pathology Assessment Tool and Hierarchy for Diagnosis (BPATH-Dx) provided search terms and guided electronic transcription of diagnoses from text fields on breast pathology clinical reports to standardized categories.

Approach:

Breast pathology encounters in the pathology database were matched with administrative data for 7332 women with breast tissue specimens obtained from an initial procedure in the health system from January 1, 2008 to December 31, 2011. Sequential queries of the pathology text based on BPATH-Dx categorized biopsies according to their worst pathological diagnosis, as is standard practice. Diagnoses ranged from invasive breast cancer (23.3%), carcinoma in situ (7.8%), atypical lesions (6.39%), proliferative lesions without atypia (27.9%), and nonproliferative lesions (34.7%), and were further classified into subcategories. A random sample of 5% of reports that were manually reviewed indicated 97.5% agreement.

Conclusions:

Sequential queries of free-text pathology reports guided by a standardized assessment tool in conjunction with a web-based search application provide an efficient and reproducible approach to accessing nonmalignant breast pathology diagnoses. This method advances the use of pathology data and electronic health records to improve health care quality, patient care, outcomes, and research.

Actualizing Personalized Healthcare for Women through Connected Data Systems: Breast Cancer Screening and Diagnosis

BACKGROUND

Healthcare organizations have invested in electronic patient data systems, yet use of health data to optimize personalized care has been limited.

PRIMARY STUDY OBJECTIVE

To develop and pilot an integrated source of health system data related to breast healthcare.

METHODS/DESIGN

This study is a quality improvement project. Patient-level data from multiple internal sources were identified, mapped to a common data model, linked, and validated to create a breast healthcare-specific data mart. Linkages were based on matching algorithms using patient identifiers to group data from the same patient. Data definitions, a data dictionary, and indicators for quality and benchmarking aligned with standardized measures. Clinical pathways were developed to outline the patient populations, data elements, decision points, and outcomes for specific conditions.

SETTING

Electronic data sources in a community-based health system in the United States.

PARTICIPANTS

Women receiving breast cancer screening, prevention, and diagnosis services.

MAIN OUTCOME MEASURES

Distribution of mammography examinations and pathologic results of breast biopsies.

RESULTS

From 2008 to 2011, 200768 screening and 50200 diagnostic mammograms were obtained; rates varied by age over time. Breast biopsies for 7332 women indicated 23.3% with invasive breast cancer, 6.7% with ductal carcinoma in situ, and 70.0% with nonmalignant diagnoses that would not have been further differentiated by administrative codes alone.

LIMITATIONS

Evaluation of validity and efficiency and additional tracking of clinical outcomes are needed.

CONCLUSIONS

The creation of a patient-centered data system by connecting and integrating disparate data sources within a large health system allows customized analyses of data and improves capacity for clinical decision making and personalized healthcare.

Rates and indications for surgical breast biopsies in a community-based health system

BACKGROUND

High rates of surgical breast biopsies in community hospitals have been reported but may misrepresent actual practice.

METHODS

Patient-level data from 5,757 women who underwent breast biopsies in a large integrated health system were evaluated to determine biopsy types, rates, indications, and diagnoses.

RESULTS

Between 2008 and 2010, 6,047 breast biopsies were performed on 5,757 women. Surgical biopsy was the initial diagnostic procedure in 16% (n = 942) of women overall and in 6% (72 of 1,236) of women with newly diagnosed invasive breast cancer. Invasive breast cancer was diagnosed in 72 women (8%) undergoing surgical biopsy compared with 1,164 (24%) undergoing core needle biopsy (P < .001, age adjusted). Main indications for surgical biopsies included symptomatic abnormalities, technical challenges, and patient choice.

CONCLUSIONS

Surgical biopsy was the initial diagnostic procedure in 16% of women with breast abnormalities, comparable with rates at academic centers. Rates could be improved by more careful consideration of indications.

Trends in research time, fellowship training, and practice patterns among general surgery graduates

SUMMARY

A comparison of research experience, fellowship training, and ultimate practice patterns of general surgery graduates at a university-based surgical residency program. Research experience correlated with pursuing fellowship training and predicted an eventual academic career. More recently, graduates have been able to obtain fellowships without a dedicated research year, perhaps reflecting shifting fellowship training opportunities.

BACKGROUND

We hypothesized that the relationships among dedicated research experience during residency, fellowship training, and career choices is changing as research and fellowship opportunities evolve.

METHODS

Comparison of research experience, fellowship training, and ultimate practice patterns of general surgery graduates for 2 decades (1990-1999, n = 82; 2000-2009, n = 98) at a university-based residency program. Main outcome measures were number of years and area of research, fellowship training, and practice setting.

RESULTS

Compared by decade, graduates became increasingly fellowship-trained (51.2% vs 67.3%; p < 0.05) and pursuit of fellowship training increased for both research and nonresearch participating graduates. The number of residents completing more than 1 year of research doubled (9.8% vs 22.4%, p < 0.05). By decade, the percentage of female graduates increased significantly (22% vs 41%, p = 0.005), with more women participating in dedicated research (17% vs 51%, p < 0.001) and seeking fellowships. The number of graduates going into specialty practice and academic/clinical faculty positions increased over time.

CONCLUSIONS

Surgical residents have completed more dedicated research years and became increasingly fellowship-trained over time. The proportion of female graduates has increased with similar increases in research time and fellowship training in this subgroup. In the earlier decade, dedicated research experiences during surgical residency correlated with pursuing fellowship training, and predicted an eventual academic career. More recently, graduates have obtained fellowships and academic positions without dedicated research time, perhaps reflecting shifting fellowship opportunities.

False-Negative Sentinel Lymph Node Biopsy in Head and Neck Melanoma

Objective. The results of sentinel lymph node biopsy (SLNB) can be useful for staging and deciding on adjuvant treatment for patients with head and neck melanoma. False-negative SLNB can result in treatment delay. This study aimed to evaluate the characteristics and outcome of patients with false-negative SLNB in cutaneous melanoma of the head and neck.

Study Design. Longitudinal cohort study using a prospective institutional tumor registry.

Setting. Academic health center.

Subjects and Methods. Data from 153 patients who underwent SLNB for melanoma of the head and neck were analyzed. False-negative biopsy was defined as recurrence of tumor in a previously identified negative nodal basin. Statistical analysis was performed on registry data.

Results. Positive sentinel lymph nodes were identified in 19 (12.4%) patients. False-negative SLNB was noted in 9 (5.9%) patients, with a false-negative SLNB rate of 32.1%. Using multivariate regression analysis, only examination of a single sentinel lymph node was a significant predictor of false-negative SLNB ( P = .01). The mean treatment delay for the false-negative SLNB group was 470 days compared with 23 days in the positive SLNB group ( P < .001). The 2-year overall survival of patients with false-negative SLNB was 75% compared with 84% and 98% in positive and negative SLNB groups, respectively ( P = .02).

Conclusions. False-negative SLNB is more likely to occur when a single sentinel lymph node is harvested. There is significant treatment delay in patients with false-negative SLNB. False-negative SLNB is associated with poor outcome in patients with melanoma of the head and neck.

Sentinel lymph node staging of cutaneous melanoma: predictors and outcomes

BACKGROUND

The authors updated their experience with sentinel lymph node (SLN) biopsy of clinically node negative (N0) melanoma to clarify indications, predictive factors, and outcomes.

METHODS

A review of patients from the authors' institution's prospective database (n = 397) was performed; survival statistics were obtained from the institutional tumor registry.

RESULTS

The SLN-positive (SLN+) rate was 16% (47 of 282) for lesions >1 mm thick; only 2 of 105 T1 lesions were SLN+. Thickness >2 mm, upper extremity primary, and ulceration predicted SLN+ status. Most SLN+ patients underwent completion node dissection; 12% had additional positive nodes. The false-negative SLN biopsy rate was 4.0%; the majority involved lower extremity and head and neck primaries. The overall complication rate was 26%; all were minor and resolved within 6 months. Overall 5-year survival rates were 73% and 92% for SLN+ and SLN-negative patients, respectively. SLN status was the most significant predictor of survival.

CONCLUSIONS

SLN status, the most important determinant of outcome for clinically N0 melanoma, correlated with T stage, ulceration, and site. Staging of T1 lesions had low yield. A minority of completion node dissections yielded additional positive nodes.