On Algorithmic Fairness in Medical Practice

Predicting 5-year-olds mental health at birth: development and internal validation of a multivariable model using the prospective ELFE birth cohort

We developed and internally validated a multivariable model to be used in the perinatal period, to predict 5-year-olds mental health, using the ELFE prospective French multicentre birth cohort (n=9768). Twenty-six candidate predictors were used, spanning pre-pregnancy maternal health, pregnancy-specific-experiences, birth factors and sociodemographic risk (maternal age, education, relationship, migrancy and family income). The Strengths and Difficulties Questionnaire total score at 5-years, dichotomised at the recommended cut-off (16), was the outcome. Least Absolute Shrinkage and Selector Operator followed by bootstrapping was used. High and low-risk was classified by ≥8% risk-threshold score. Stability of the model at population- and individual-level and model performance across groups of interest (sex, sociodemographic risk and neonatal intensive care admissions) was also examined. 10 variables (total number pregnancy-specific experiences, sociodemographic risk, maternal pre-existing hypertension and psychological difficulties, gravidity, maternal mental health problems in a previous pregnancy, smoking and alcohol use in current pregnancy, how labour started and infant sex) with a C-statistic of 0.67; 95%CI (0.64-0.69) predicted mental health. The positive and negative predictive value were 12% & 95.4% respectively, leading to 78.8% of children correctly classified. Model performance was similar across groups of interest but increased for children (born ≥33-weeks-gestation) with neonatal admissions (AUC 0.78; 95%CI (0.69-0.87)). This model is most useful for identifying low-risk children. Applying this model in a tiered preventative intervention framework could be beneficial with those predicted to be high-risk receiving further screening to determine the level of intervention required. External validation and implementation research are required before considering its use in practice.

Bridging gaps in artificial intelligence adoption for maternal-fetal and obstetric care: Unveiling transformative capabilities and challenges

PURPOSE

This review aims to comprehensively explore the application of Artificial Intelligence (AI) to an area that has not been traditionally explored in depth: the continuum of maternal-fetal health. In doing so, the intent was to examine this physiologically continuous spectrum of mother and child health, as well as to highlight potential pitfalls, and suggest solutions for the same.

METHOD

A systematic search identified studies employing AI techniques for prediction, diagnosis, and decision support employing various modalities like imaging, electrophysiological signals and electronic health records in the domain of obstetrics and fetal health. In the selected articles then, AI applications in fetal morphology, gestational age assessment, congenital defect detection, fetal monitoring, placental analysis, and maternal physiological monitoring were critically examined both from the perspective of the domain and artificial intelligence.

RESULT

AI-driven solutions demonstrate promising capabilities in medical diagnostics and risk prediction, offering automation, improved accuracy, and the potential for personalized medicine. However, challenges regarding data availability, algorithmic transparency, and ethical considerations must be overcome to ensure responsible and effective clinical implementation. These challenges must be urgently addressed to ensure a domain as critical to public health as obstetrics and fetal health, is able to fully benefit from the gigantic strides made in the field of artificial intelligence.

CONCLUSION

Open access to relevant datasets is crucial for equitable progress in this critical public health domain. Integrating responsible and explainable AI, while addressing ethical considerations, is essential to maximize the public health benefits of AI-driven solutions in maternal-fetal care.

Examining inclusivity: the use of AI and diverse populations in health and social care: a systematic review

BACKGROUND

Artificial intelligence (AI)-based systems are being rapidly integrated into the fields of health and social care. Although such systems can substantially improve the provision of care, diverse and marginalized populations are often incorrectly or insufficiently represented within these systems. This review aims to assess the influence of AI on health and social care among these populations, particularly with regard to issues related to inclusivity and regulatory concerns.

METHODS

We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Six leading databases were searched, and 129 articles were selected for this review in line with predefined eligibility criteria.

RESULTS

This research revealed disparities in AI outcomes, accessibility, and representation among diverse groups due to biased data sources and a lack of representation in training datasets, which can potentially exacerbate inequalities in care delivery for marginalized communities.

CONCLUSION

AI development practices, legal frameworks, and policies must be reformulated to ensure that AI is applied in an equitable manner. A holistic approach must be used to address disparities, enforce effective regulations, safeguard privacy, promote inclusion and equity, and emphasize rigorous validation.

Can the potential benefit of individualizing treatment be assessed using trial summary statistics alone?

Individualizing treatment assignment can improve outcomes for diseases with patient-to-patient variability in comparative treatment effects. When a clinical trial demonstrates that some patients improve on treatment while others do not, it is tempting to assume that treatment effect heterogeneity exists. However, if outcome variability is mainly driven by factors other than variability in the treatment effect, investigating the extent to which covariate data can predict differential treatment response is a potential waste of resources. Motivated by recent meta-analyses assessing the potential of individualizing treatment for major depressive disorder using only summary statistics, we provide a method that uses summary statistics widely available in published clinical trial results to bound the benefit of optimally assigning treatment to each patient. We also offer alternate bounds for settings in which trial results are stratified by another covariate. Our upper bounds can be especially informative when they are small, as there is then little benefit to collecting additional covariate data. We demonstrate our approach using summary statistics from a depression treatment trial. Our methods are implemented in the rct2otrbounds R package.

A survey of recent methods for addressing AI fairness and bias in biomedicine

OBJECTIVES

Artificial intelligence (AI) systems have the potential to revolutionize clinical practices, including improving diagnostic accuracy and surgical decision-making, while also reducing costs and manpower. However, it is important to recognize that these systems may perpetuate social inequities or demonstrate biases, such as those based on race or gender. Such biases can occur before, during, or after the development of AI models, making it critical to understand and address potential biases to enable the accurate and reliable application of AI models in clinical settings. To mitigate bias concerns during model development, we surveyed recent publications on different debiasing methods in the fields of biomedical natural language processing (NLP) or computer vision (CV). Then we discussed the methods, such as data perturbation and adversarial learning, that have been applied in the biomedical domain to address bias.

METHODS

We performed our literature search on PubMed, ACM digital library, and IEEE Xplore of relevant articles published between January 2018 and December 2023 using multiple combinations of keywords. We then filtered the result of 10,041 articles automatically with loose constraints, and manually inspected the abstracts of the remaining 890 articles to identify the 55 articles included in this review. Additional articles in the references are also included in this review. We discuss each method and compare its strengths and weaknesses. Finally, we review other potential methods from the general domain that could be applied to biomedicine to address bias and improve fairness.

RESULTS

The bias of AIs in biomedicine can originate from multiple sources such as insufficient data, sampling bias and the use of health-irrelevant features or race-adjusted algorithms. Existing debiasing methods that focus on algorithms can be categorized into distributional or algorithmic. Distributional methods include data augmentation, data perturbation, data reweighting methods, and federated learning. Algorithmic approaches include unsupervised representation learning, adversarial learning, disentangled representation learning, loss-based methods and causality-based methods.

Health equity assessment of machine learning performance (HEAL): a framework and dermatology AI model case study

Background

Artificial intelligence (AI) has repeatedly been shown to encode historical inequities in healthcare. We aimed to develop a framework to quantitatively assess the performance equity of health AI technologies and to illustrate its utility via a case study.

Methods

Here, we propose a methodology to assess whether health AI technologies prioritise performance for patient populations experiencing worse outcomes, that is complementary to existing fairness metrics. We developed the Health Equity Assessment of machine Learning performance (HEAL) framework designed to quantitatively assess the performance equity of health AI technologies via a four-step interdisciplinary process to understand and quantify domain-specific criteria, and the resulting HEAL metric. As an illustrative case study (analysis conducted between October 2022 and January 2023), we applied the HEAL framework to a dermatology AI model. A set of 5420 teledermatology cases (store-and-forward cases from patients of 20 years or older, submitted from primary care providers in the USA and skin cancer clinics in Australia), enriched for diversity in age, sex and race/ethnicity, was used to retrospectively evaluate the AI model's HEAL metric, defined as the likelihood that the AI model performs better for subpopulations with worse average health outcomes as compared to others. The likelihood that AI performance was anticorrelated to pre-existing health outcomes was estimated using bootstrap methods as the probability that the negated Spearman's rank correlation coefficient (i.e., "R") was greater than zero. Positive values of R suggest that subpopulations with poorer health outcomes have better AI model performance. Thus, the HEAL metric, defined as p (R >0), measures how likely the AI technology is to prioritise performance for subpopulations with worse average health outcomes as compared to others (presented as a percentage below). Health outcomes were quantified as disability-adjusted life years (DALYs) when grouping by sex and age, and years of life lost (YLLs) when grouping by race/ethnicity. AI performance was measured as top-3 agreement with the reference diagnosis from a panel of 3 dermatologists per case.

Findings

Across all dermatologic conditions, the HEAL metric was 80.5% for prioritizing AI performance of racial/ethnic subpopulations based on YLLs, and 92.1% and 0.0% respectively for prioritizing AI performance of sex and age subpopulations based on DALYs. Certain dermatologic conditions were significantly associated with greater AI model performance compared to a reference category of less common conditions. For skin cancer conditions, the HEAL metric was 73.8% for prioritizing AI performance of age subpopulations based on DALYs.

Interpretation

Analysis using the proposed HEAL framework showed that the dermatology AI model prioritised performance for race/ethnicity, sex (all conditions) and age (cancer conditions) subpopulations with respect to pre-existing health disparities. More work is needed to investigate ways of promoting equitable AI performance across age for non-cancer conditions and to better understand how AI models can contribute towards improving equity in health outcomes.

Funding

Google LLC.

Stability of clinical prediction models developed using statistical or machine learning methods

Clinical prediction models estimate an individual's risk of a particular health outcome. A developed model is a consequence of the development dataset and model-building strategy, including the sample size, number of predictors, and analysis method (e.g., regression or machine learning). We raise the concern that many models are developed using small datasets that lead to instability in the model and its predictions (estimated risks). We define four levels of model stability in estimated risks moving from the overall mean to the individual level. Through simulation and case studies of statistical and machine learning approaches, we show instability in a model's estimated risks is often considerable, and ultimately manifests itself as miscalibration of predictions in new data. Therefore, we recommend researchers always examine instability at the model development stage and propose instability plots and measures to do so. This entails repeating the model-building steps (those used to develop the original prediction model) in each of multiple (e.g., 1000) bootstrap samples, to produce multiple bootstrap models, and deriving (i) a prediction instability plot of bootstrap model versus original model predictions; (ii) the mean absolute prediction error (mean absolute difference between individuals' original and bootstrap model predictions), and (iii) calibration, classification, and decision curve instability plots of bootstrap models applied in the original sample. A case study illustrates how these instability assessments help reassure (or not) whether model predictions are likely to be reliable (or not), while informing a model's critical appraisal (risk of bias rating), fairness, and further validation requirements.

The ABC of algorithmic aversion: not agent, but benefits and control determine the acceptance of automated decision-making

While algorithmic decision-making (ADM) is projected to increase exponentially in the coming decades, the academic debate on whether people are ready to accept, trust, and use ADM as opposed to human decision-making is ongoing. The current research aims at reconciling conflicting findings on ‘algorithmic aversion’ in the literature. It does so by investigating algorithmic aversion while controlling for two important characteristics that are often associated with ADM: increased benefits (monetary and accuracy) and decreased user control. Across three high-powered (Ntotal = 1192), preregistered 2 (agent: algorithm/human) × 2 (benefits: high/low) × 2 (control: user control/no control) between-subjects experiments, and two domains (finance and dating), the results were quite consistent: there is little evidence for a default aversion against algorithms and in favor of human decision makers. Instead, users accept or reject decisions and decisional agents based on their predicted benefits and the ability to exercise control over the decision.

Enabling Fairness in Healthcare Through Machine Learning

The use of machine learning systems for decision-support in healthcare may exacerbate health inequalities. However, recent work suggests that algorithms trained on sufficiently diverse datasets could in principle combat health inequalities. One concern about these algorithms is that their performance for patients in traditionally disadvantaged groups exceeds their performance for patients in traditionally advantaged groups. This renders the algorithmic decisions unfair relative to the standard fairness metrics in machine learning. In this paper, we defend the permissible use of affirmative algorithms; that is, algorithms trained on diverse datasets that perform better for traditionally disadvantaged groups. Whilst such algorithmic decisions may be unfair, the fairness of algorithmic decisions is not the appropriate locus of moral evaluation. What matters is the fairness of final decisions, such as diagnoses, resulting from collaboration between clinicians and algorithms. We argue that affirmative algorithms can permissibly be deployed provided the resultant final decisions are fair.