Predicting Patient Demographics From Chest Radiographs With Deep Learning

Demographic bias in misdiagnosis by computational pathology models

Despite increasing numbers of regulatory approvals, deep learning-based computational pathology systems often overlook the impact of demographic factors on performance, potentially leading to biases. This concern is all the more important as computational pathology has leveraged large public datasets that underrepresent certain demographic groups. Using publicly available data from The Cancer Genome Atlas and the EBRAINS brain tumor atlas, as well as internal patient data, we show that whole-slide image classification models display marked performance disparities across different demographic groups when used to subtype breast and lung carcinomas and to predict IDH1 mutations in gliomas. For example, when using common modeling approaches, we observed performance gaps (in area under the receiver operating characteristic curve) between white and Black patients of 3.0% for breast cancer subtyping, 10.9% for lung cancer subtyping and 16.0% for IDH1 mutation prediction in gliomas. We found that richer feature representations obtained from self-supervised vision foundation models reduce performance variations between groups. These representations provide improvements upon weaker models even when those weaker models are combined with state-of-the-art bias mitigation strategies and modeling choices. Nevertheless, self-supervised vision foundation models do not fully eliminate these discrepancies, highlighting the continuing need for bias mitigation efforts in computational pathology. Finally, we demonstrate that our results extend to other demographic factors beyond patient race. Given these findings, we encourage regulatory and policy agencies to integrate demographic-stratified evaluation into their assessment guidelines.

Deep learning-based age estimation from chest CT scans

PURPOSE

Medical imaging can be used to estimate a patient's biological age, which may provide complementary information to clinicians compared to chronological age. In this study, we aimed to develop a method to estimate a patient's age based on their chest CT scan. Additionally, we investigated whether chest CT estimated age is a more accurate predictor of lung cancer risk compared to chronological age.

METHODS

To develop our age prediction model, we utilized composite CT images and Inception-ResNet-v2. The model was trained, validated, and tested on 13,824 chest CT scans from the National Lung Screening Trial, with 91% for training, 5% for validation, and 4% for testing. Additionally, we independently tested the model on 1849 CT scans collected locally. To assess chest CT estimated age as a risk factor for lung cancer, we computed the relative lung cancer risk between two groups. Group 1 consisted of individuals assigned a CT age older than their chronological age, while Group 2 comprised those assigned a CT age younger than their chronological age.

RESULTS

Our analysis revealed a mean absolute error of 1.84 years and a Pearson's correlation coefficient of 0.97 for our local data when comparing chronological age with the estimated CT age. The model showed the most activation in the area associated with the lungs during age estimation. The relative risk for lung cancer was 1.82 (95% confidence interval, 1.65-2.02) for individuals assigned a CT age older than their chronological age compared to those assigned a CT age younger than their chronological age.

CONCLUSION

Findings suggest that chest CT age captures some aspects of biological aging and may be a more accurate predictor of lung cancer risk than chronological age. Future studies with larger and more diverse patients are required for the generalization of the interpretations.

Risk of Bias in Chest Radiography Deep Learning Foundation Models

PURPOSE

To analyze a recently published chest radiography foundation model for the presence of biases that could lead to subgroup performance disparities across biologic sex and race.

MATERIALS AND METHODS

This Health Insurance Portability and Accountability Act-compliant retrospective study used 127 118 chest radiographs from 42 884 patients (mean age, 63 years ± 17 [SD]; 23 623 male, 19 261 female) from the CheXpert dataset that were collected between October 2002 and July 2017. To determine the presence of bias in features generated by a chest radiography foundation model and baseline deep learning model, dimensionality reduction methods together with two-sample Kolmogorov-Smirnov tests were used to detect distribution shifts across sex and race. A comprehensive disease detection performance analysis was then performed to associate any biases in the features to specific disparities in classification performance across patient subgroups.

RESULTS

Ten of 12 pairwise comparisons across biologic sex and race showed statistically significant differences in the studied foundation model, compared with four significant tests in the baseline model. Significant differences were found between male and female (P < .001) and Asian and Black (P < .001) patients in the feature projections that primarily capture disease. Compared with average model performance across all subgroups, classification performance on the "no finding" label decreased between 6.8% and 7.8% for female patients, and performance in detecting "pleural effusion" decreased between 10.7% and 11.6% for Black patients.

CONCLUSION

The studied chest radiography foundation model demonstrated racial and sex-related bias, which led to disparate performance across patient subgroups; thus, this model may be unsafe for clinical applications.Keywords: Conventional Radiography, Computer Application-Detection/Diagnosis, Chest Radiography, Bias, Foundation Models Supplemental material is available for this article. Published under a CC BY 4.0 license.See also commentary by Czum and Parr in this issue.

Sequestration of imaging studies in MIDRC: stratified sampling to balance demographic characteristics of patients in a multi-institutional data commons

Purpose

The Medical Imaging and Data Resource Center (MIDRC) is a multi-institutional effort to accelerate medical imaging machine intelligence research and create a publicly available image repository/commons as well as a sequestered commons for performance evaluation and benchmarking of algorithms. After de-identification, approximately 80% of the medical images and associated metadata become part of the open commons and 20% are sequestered from the open commons. To ensure that both commons are representative of the population available, we introduced a stratified sampling method to balance the demographic characteristics across the two datasets.

Approach

Our method uses multi-dimensional stratified sampling where several demographic variables of interest are sequentially used to separate the data into individual strata, each representing a unique combination of variables. Within each resulting stratum, patients are assigned to the open or sequestered commons. This algorithm was used on an example dataset containing 5000 patients using the variables of race, age, sex at birth, ethnicity, COVID-19 status, and image modality and compared resulting demographic distributions to naïve random sampling of the dataset over 2000 independent trials.

Results

Resulting prevalence of each demographic variable matched the prevalence from the input dataset within one standard deviation. Mann-Whitney U test results supported the hypothesis that sequestration by stratified sampling provided more balanced subsets than naïve randomization, except for demographic subcategories with very low prevalence.

Conclusions

The developed multi-dimensional stratified sampling algorithm can partition a large dataset while maintaining balance across several variables, superior to the balance achieved from naïve randomization.

AI pitfalls and what not to do: mitigating bias in AI

Various forms of artificial intelligence (AI) applications are being deployed and used in many healthcare systems. As the use of these applications increases, we are learning the failures of these models and how they can perpetuate bias. With these new lessons, we need to prioritize bias evaluation and mitigation for radiology applications; all the while not ignoring the impact of changes in the larger enterprise AI deployment which may have downstream impact on performance of AI models. In this paper, we provide an updated review of known pitfalls causing AI bias and discuss strategies for mitigating these biases within the context of AI deployment in the larger healthcare enterprise. We describe these pitfalls by framing them in the larger AI lifecycle from problem definition, data set selection and curation, model training and deployment emphasizing that bias exists across a spectrum and is a sequela of a combination of both human and machine factors.

Patient Perspectives on Artificial Intelligence in Radiology

There are two major areas for patient engagement in radiology artificial intelligence (AI). One is in the sharing of data for AI development; the second is the use of AI in patient care. In general, individuals support sharing deidentified data if used for the common good, to help others with similar health conditions, or for research. However, there is concern with risk to privacy including reidentification and use for other than intended purposes. Lack of trust is mentioned as a barrier for data sharing. Individuals want to be involved in the data-sharing process. In the use of AI in medical care, patients generally support AI as an assist to the radiologist but lack trust in unsupervised AI. Patients worry about liability in case of bad outcomes. Patients are concerned about loss of the human connection and the loss of empathy during a vulnerable time in their lives. Patients expressed concern about risk of discrimination due to bias in AI algorithms. Building trust in AI requires transparency, explainability, security, and privacy protection. Radiologists can take action to prepare their patients to become more trusting of AI. Developing and implementing data-sharing agreements allows patients to voluntarily help in the algorithm development process. Developing AI disclosure guidelines and having AI use disclosure discussions with patients will help them understand the use of AI in their care. As the use of AI increases, there is an opportunity for radiologists to develop and maintain close relationships with their patients and to become more involved in their care.

Building Diversity, Equity, and Inclusion Within Radiology Artificial Intelligence: Representation Matters, From Data to the Workforce

Diversity, equity, and inclusion (DEI) is both a critical ingredient and moral imperative in shaping the future of radiology artificial intelligence (AI) for improved patient care, from design to deployment. At the design level: Potential biases and discrimination within data sets results in inaccurate radiology AI models, and there is an urgent need to purposefully embed DEI principles throughout the AI development and implementation process. At the deployment level: Diverse representation in radiology AI leadership, research, and career development is necessary to avoid worsening structural and historical health inequities. To create an inclusive and equitable AI-enabled future in healthcare, a DEI radiology AI leadership training program may be needed to cultivate a diverse and sustainable pipeline of leaders in the field.

Implications of predicting race variables from medical images

AI-predicted race variables pose risks and opportunities for studying health disparities.

Opportunistic detection of type 2 diabetes using deep learning from frontal chest radiographs

Deep learning (DL) models can harness electronic health records (EHRs) to predict diseases and extract radiologic findings for diagnosis. With ambulatory chest radiographs (CXRs) frequently ordered, we investigated detecting type 2 diabetes (T2D) by combining radiographic and EHR data using a DL model. Our model, developed from 271,065 CXRs and 160,244 patients, was tested on a prospective dataset of 9,943 CXRs. Here we show the model effectively detected T2D with a ROC AUC of 0.84 and a 16% prevalence. The algorithm flagged 1,381 cases (14%) as suspicious for T2D. External validation at a distinct institution yielded a ROC AUC of 0.77, with 5% of patients subsequently diagnosed with T2D. Explainable AI techniques revealed correlations between specific adiposity measures and high predictivity, suggesting CXRs' potential for enhanced T2D screening.