Abstract 1970: Protecting against algorithmic bias of AI-based clinical decision making tools in oncology

MOTIVATION: AI-based decision-making tools hold the potential to reduce patient suffering and burden. However, healthcare data reflects inequity, and datasets often lack representation of underprivileged groups. Without systems and methods to protect all patients, we risk making dangerous biases automatic and invisible. We can only ensure protection against bias if performance is monitored in production, continuously and with meaningful measures. Notably, a lack of representation and bias in the validation of diverse ancestry in genetics data results in genetic misdiagnoses and potential health disparities. Similarly, monitoring calibration in addition to receiver operating characteristic curve area under curve (ROC-AUC) in subgroups could have prevented bias predictions leading to underestimation of the health needs of sicker patients. Here, we describe methods and results implemented in an AI-based clinical decision-making tool to protect against bias.

METHODS: Our model uses proven techniques and metrics to identify and monitor bias and quantify the magnitude of its impact. Our algorithm compares all demographic subgroups to that of a top performer based on overall model performance (measured by both ROC-AUC and Brier score). To determine if the top-performing subgroup is receiving statistically significant preferential treatment, we perform t-tests for each other subgroup against the top performer (p-value > 0.05 indicates that performance is statistically significant) using sample-wise performance metrics (log-loss in this case). We then quantify the magnitude of the difference by computing the ratio of the top-performer to that of the affected subgroup. The bias detection algorithm is continuously run as our model generates risk predictions based on up-to-date patient data. If the algorithm identifies a bias, it immediately notifies data scientists and engineers to begin rapidly restoring balanced model performance for clinical use by investigating algorithms that optimize for overall and balanced performance by subgroup.

RESULTS: We trained and validated our machine learning model to predict 30-day emergency department visits using a data set with over 28,000 oncology patients representing diverse cancer types, races, ethnicities, ages, genders, and socio-economic statuses. Our model performance on a held-out validation data set was exceptional overall (AUC 0.8, Brier score 0.07) and across all cancer type and demographic subgroups (AUC 0.74-0.82, Brier score 0.06-0.1). We continuously monitored our live model in production over three months to ensure consistent and fair performance. Our overall and subgroup performance in production never dropped below AUC 0.75 or above Brier score 0.1.

CONCLUSIONS: We demonstrate an effective framework for continuously and automatically detecting and combating algorithmic bias in oncology AI decision-making tools.

Citation Format: Renee D. George, Benjamin H. Ellis, Chris J. Sidey-Gibbons, Christine Swisher. Protecting against algorithmic bias of AI-based clinical decision making tools in oncology [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 1970.

Combined Spatially Resolved Characterization of Antireflection and Antisoiling Coatings for PV Module Glass

Characterization of photovoltaic (PV) module materials throughout different stages of service life is crucial to understanding and improving the durability of these materials. Currently the large-scale of PV modules (>1 m²) is imbalanced with the small-scale of most materials characterization tools (≤1 cm²). Furthermore, understanding degradation mechanisms often requires a combination of multiple characterization techniques. Here, we present adaptations of three standard materials characterization techniques to enable mapping characterization over moderate sample areas (≥25 cm²). Contact angle, ellipsometry, and UV-vis spectroscopy are each adapted and demonstrated on two representative samples: a commercial multifunctional coating for PV glass and an oxide combinatorial sample library. Best practices are discussed for adapting characterization techniques for large-area mapping and combining mapping information from multiple techniques.

The PlA1/A2 polymorphism of glycoprotein IIIa as a risk factor for stroke: a systematic review and meta-analysis

BACKGROUND

The PlA1/A2 polymorphism of glycoprotein IIIa (GPIIIa) has been reported to be associated with risk of stroke in some studies, although other studies suggest no such association. This meta-analysis and systematic review was conducted to investigate the hypothesis that carriage of the PlA2 allele is a risk factor for stroke.

METHODS

Electronic databases (MEDLINE and EMBASE) were searched for all articles evaluating carriage of the PlA2 allele and the incidence of stroke. Pooled odds ratios (ORs) were calculated using fixed-effect and random-effect models.

FINDINGS

A total of 35 articles were eligible for inclusion, of which 25 studies were suitable for statistical analysis. For carriage of the PlA2 allele, OR 1.12 (n = 11,873; 95% CI = 1.03-1.22; p = 0.011) was observed for the incidence of stroke in adults, with subgroup analyses identifying the association driven by stroke of an ischaemic (n = 10,494; OR = 1.15, 95% CI = 1.05-1.27; p = 0.003) but not haemorrhagic aetiology (n = 2,470; OR = 0.90, 95% CI = 0.71-1.14; p = 0.398). This association with ischaemic stroke was strongest in individuals homozygous for the PlA2 allele compared to those homozygous for wild-type PlA1 (n = 5,906; OR = 1.74, 95% CI = 1.34-2.26; p<0.001). Subgroup analysis of ischaemic stroke subtypes revealed an increased association with stroke of cardioembolic (n = 1,271; OR 1.56, 95% CI 1.14-2.12; p = 0.005) and large vessel (n = 1,394; OR = 1.76, 95% CI 1.34-2.31; p<0.001) aetiology, but not those of small vessel origin (n = 1,356; OR = 0.99, 95% CI 0.74-1.33; p = 0.950). Egger's regression test suggested a low probability of publication bias for all analyses (p>0.05).

CONCLUSIONS

The totality of published data supports the hypothesis that carriage of the PlA2 polymorphism of GPIIIa is a risk factor for ischaemic strokes, and specifically those of cardioembolic and large vessel origin.

A preliminary analysis of memorable support and nonsupport messages received by nurses in acute care settings

This study presents a preliminary analysis of recalled messages for support and nonsupport received by nurses in a midsize, midwestern hospital. In interviews nurses recalled specific incidents of support and nonsupport they had experienced in the workplace. The structural and functional characteristics of these memorable messages of support and nonsupport were then independently coded by trained coders within the following categories: (a) the structure and form of the message, (b) the context and timing of the message, (c) the relationship between the source and receiver, and (d) the content of the message and the message effects. The results of chi-square analyses and McNemar tests are presented. The results suggest that in the specific categorical areas examined, support and nonsupport messages exhibit a relatively high degree of similarity within-participants. That is, participants appeared to use a cognitive template for classifying communication acts as supportive and nonsupportive. Specific between-participant message similarities and differences are also enumerated and explained.