Guiding Principles to Address the Impact of Algorithm Bias on Racial and Ethnic Disparities in Health and Health Care

Predicting cancer survival at different stages: Insights from fair and explainable machine learning approaches

OBJECTIVES

While prior machine learning (ML) models for cancer survivability prediction often treated all cancer stages uniformly, cancer survivability prediction should involve understanding how different stages impact the outcomes. Additionally, the success of ML-powered cancer survival prediction models depends a lot on being fair and easy to understand, especially for different stages of cancer. This study addresses cancer survivability prediction using fair and explainable ML methods.

METHODS

Focusing on bladder, breast, and prostate cancers using SEER Program data, we developed and validated fair and explainable ML strategies to train separate models for each stage. These computational strategies also advance the fairness and explainability of the ML models.

RESULTS

The current work highlights the important role of ML fairness and explainability in stage-specific cancer survivability prediction, capturing and interpreting the associated factors influencing cancer survivability.

CONCLUSIONS

This contribution advocates for integrating fairness and explainability in these ML models to ensure equitable, fair, interpretable, and transparent predictions, ultimately enhancing patient care and shared decision-making in cancer treatment.

Explanation and elaboration of MedinAI: guidelines for reporting artificial intelligence studies in medicines, pharmacotherapy, and pharmaceutical services

The increasing adoption of artificial intelligence (AI) in medicines, pharmacotherapy, and pharmaceutical services necessitates clear guidance on reporting standards. While the MedinAI Statement (Bottacin in Int J Clin Pharm, https://doi.org/10.1007/s11096-025-01905-3, 2025) provides core guidelines for reporting AI studies in these fields, detailed explanations and practical examples are crucial for optimal implementation. This companion document was developed to offer comprehensive guidance and real-world examples for each guideline item. The document elaborates on all 14 items and 78 sub-items across four domains: core, ethical considerations in medication and pharmacotherapy, medicines as products, and services related to medicines and pharmacotherapy. Through clear, actionable guidance and diverse examples, this document enhances MedinAI's utility, enabling researchers and stakeholders to improve the quality and transparency of AI research reporting across various contexts, study designs, and development stages.

Monitoring oral health remotely: ethical considerations when using AI among vulnerable populations

Technological innovations in dentistry are revolutionizing the monitoring and management of oral health. This perspective article critically examines the rapid expansion of remote monitoring technologies-including artificial intelligence (AI)-driven diagnostics, electronic health records (EHR), wearable devices, mobile health applications, and chatbots-and discusses their ethical, legal, and social implications. The accelerated adoption of these digital tools, particularly in the wake of the COVID-19 pandemic, has enhanced accessibility to care while simultaneously raising significant concerns regarding patient consent, data privacy, and algorithmic biases. We review current applications ranging from AI-assisted detection of dental pathologies to blockchain-enabled data transfer within EHR systems, highlighting the potential for improved diagnostic accuracy and the risks associated with over-reliance on remote assessments. Furthermore, we underscore the challenges posed by the digital divide, where disparities in digital literacy and access may inadvertently exacerbate existing socio-economic and health inequalities. This article calls for the development and rigorous implementation of ethical frameworks and regulatory guidelines that ensure the reliability, transparency, and accountability of digital health innovations. By integrating multidisciplinary insights, our discussion aims to foster a balanced approach that maximizes the clinical benefits of emerging technologies while safeguarding patient autonomy and promoting equitable healthcare delivery.

Leveraging Preexisting Cardiovascular Data to Improve the Detection and Treatment of Hypertension: The NOTIFY-LVH Randomized Clinical Trial

Importance

Hypertension is often underrecognized, leading to preventable morbidity and mortality. Tailored data systems combined with care augmented by trained nonphysicians have the potential to improve cardiovascular care.

Objective

To determine whether previously collected cardiovascular imaging data could be harnessed to improve the detection and treatment of hypertension through a system-level intervention.

Design, Setting, and Participants

The NOTIFY-LVH trial was a 2-arm, pragmatic randomized clinical trial conducted from March 2023 through June 2024 within the Mass General Brigham health care system, a multi-institutional network serving the greater Boston, Massachusetts, area. The study included individuals with a Mass General Brigham primary care affiliation who had left ventricular hypertrophy (LVH) on a prior echocardiogram, had no established cardiomyopathy diagnosis, and were not being treated with antihypertensive medications. Patients were followed for 12 months postintervention.

Intervention

Population health coordinators contacted clinicians of patients randomized to the intervention, notifying them of LVH and offering assistance with follow-up care. A clinical support pathway-including 24-hour ambulatory blood pressure monitoring or cardiology referrals-was provided to aid LVH evaluation.

Main Outcomes and Measures

The primary outcome was the initiation of an antihypertensive medication. Secondary outcomes included new hypertension and cardiomyopathy diagnoses.

Results

A total of 648 patients were randomized-326 to the intervention and 322 to the control. Mean (SD) patient age was 59.4 (10.8) years and 248 patients (38.3%) were female. A total of 102 patients (15.7%) had a baseline diagnosis of hypertension and 109 patients (20.1%) had a mean outpatient blood pressure of 130/80 mm Hg or higher. Over 12 months, 53 patients (16.3%) in the intervention arm were prescribed an antihypertensive medication vs 16 patients (5.0%) in the control arm (adjusted odds ratio [OR], 3.76; 95% CI, 2.09-6.75; P < .001). Individuals in the intervention group were also more likely to be diagnosed with hypertension (adjusted OR, 4.43; 95% CI, 2.36-8.33; P < .001). Cardiomyopathy diagnoses did not significantly differ between groups.

Conclusions and Relevance

In the NOTIFY-LVH randomized clinical trial, a centralized population health coordinator-led notification and clinical support pathway for individuals with LVH on prior echocardiograms increased the initial treatment of hypertension. This work highlights the potential benefit of leveraging preexisting but potentially underutilized cardiovascular data to improve health care delivery through mechanisms augmenting the traditional ambulatory care system.

Trial Registration

ClinicalTrials.gov Identifier: NCT05713916.

Online database of clinical algorithms with race and ethnicity

Some clinical algorithms incorporate an individual's race, ethnicity, or both as an input variable or predictor in determining diagnoses, prognoses, treatment plans, or risk assessments. Inappropriate use of race and ethnicity in clinical algorithms at the point of care may exacerbate health disparities and promote harmful practices of race-based medicine. Using database analysis primarily, we identified 42 risk calculators that use race and ethnicity as predictors, five laboratory test results with reference ranges that differed based on race and ethnicity, one therapy recommendation based on race and ethnicity, 15 medications with race- and ethnicity-based initiation and monitoring guidelines, and five medical devices with differential racial and ethnic performances. Information on these clinical algorithms is freely available at https://www.clinical-algorithms-with-race-and-ethnicity.org/ . This resource aims to raise awareness about the use of race and ethnicity in clinical algorithms and track progress toward eliminating their inappropriate use. The database is actively updated to include clinical algorithms that were missed and additional characteristics of these algorithms.

Let's get in sync: current standing and future of AI-based detection of patient-ventilator asynchrony

BACKGROUND

Patient-ventilator asynchrony (PVA) is a mismatch between the patient's respiratory drive/effort and the ventilator breath delivery. It occurs frequently in mechanically ventilated patients and has been associated with adverse events and increased duration of ventilation. Identifying PVA through visual inspection of ventilator waveforms is highly challenging and time-consuming. Automated PVA detection using Artificial Intelligence (AI) has been increasingly studied, potentially offering real-time monitoring at the bedside. In this review, we discuss advances in automatic detection of PVA, focusing on developments of the last 15 years.

RESULTS

Nineteen studies were identified. Multiple forms of AI have been used for the automated detection of PVA, including rule-based algorithms, machine learning and deep learning. Three licensed algorithms are currently reported. Results of algorithms are generally promising (average reported sensitivity, specificity and accuracy of 0.80, 0.93 and 0.92, respectively), but most algorithms are only available offline, can detect a small subset of PVAs (focusing mostly on ineffective effort and double trigger asynchronies), or remain in the development or validation stage (84% (16/19 of the reviewed studies)). Moreover, only in 58% (11/19) of the studies a reference method for monitoring patient's breathing effort was available. To move from bench to bedside implementation, data quality should be improved and algorithms that can detect multiple PVAs should be externally validated, incorporating measures for breathing effort as ground truth. Last, prospective integration and model testing/finetuning in different ICU settings is key.

CONCLUSIONS

AI-based techniques for automated PVA detection are increasingly studied and show potential. For widespread implementation to succeed, several steps, including external validation and (near) real-time employment, should be considered. Then, automated PVA detection could aid in monitoring and mitigating PVAs, to eventually optimize personalized mechanical ventilation, improve clinical outcomes and reduce clinician's workload.

Developing and Applying the BE-FAIR Equity Framework to a Population Health Predictive Model: A Retrospective Observational Cohort Study

BACKGROUND

Population health programs rely on healthcare predictive models to allocate resources, yet models can perpetuate biases that exacerbate health disparities among marginalized communities.

OBJECTIVE

We developed the Bias-reduction and Equity Framework for Assessing, Implementing, and Redesigning (BE-FAIR) healthcare predictive models, an applied framework tested within a large health system using a population health predictive model, aiming to minimize bias and enhance equity.

DESIGN

Retrospective cohort study conducted at an academic medical center. Data collected from September 30, 2020, to October 1, 2022, were analyzed to assess bias resulting from model use.

PARTICIPANTS

Primary care or payer-attributed patients at the medical center identified through electronic health records and claims data. Participants were stratified by race-ethnicity, gender, and social vulnerability defined by the Healthy Places Index (HPI).

INTERVENTION

BE-FAIR implementation involved steps such as an anti-racism lens application, de-siloed team structure, historical intervention review, disaggregated data analysis, and calibration evaluation.

MAIN MEASURES

The primary outcome was the calibration and discrimination of the model across different demographic groups, measured by logistic regression and area under the receiver operating characteristic curve (AUROC).

RESULTS

The study population consisted of 114,311 individuals with a mean age of 43.4 years (SD 24.0 years), 55.4% female, and 59.5% white/Caucasian. Calibration differed by race-ethnicity and HPI with significantly lower predicted probabilities of hospitalization for African Americans (0.129±0.051, p=0.016), Hispanics (0.133±0.047, p=0.004), AAPI (0.120±0.051, p=0.018), and multi-race (0.245±0.087, p=0.005) relative to white/Caucasians and for individuals in low HPI areas (0 - 25%, 0.178±0.042, p<0.001; 25 - 50%, 0.129±0.044, p=0.003). AUROC values varied among demographic groups.

CONCLUSIONS

The BE-FAIR framework offers a practical approach to address bias in healthcare predictive models, guiding model development, and implementation. By identifying and mitigating biases, BE-FAIR enhances the fairness and equity of healthcare delivery, particularly for minoritized groups, paving the way for more inclusive and effective population health strategies.

Bias recognition and mitigation strategies in artificial intelligence healthcare applications

Artificial intelligence (AI) is delivering value across all aspects of clinical practice. However, bias may exacerbate healthcare disparities. This review examines the origins of bias in healthcare AI, strategies for mitigation, and responsibilities of relevant stakeholders towards achieving fair and equitable use. We highlight the importance of systematically identifying bias and engaging relevant mitigation activities throughout the AI model lifecycle, from model conception through to deployment and longitudinal surveillance.

Implementing Clinical Practice Guidelines: Considerations for Epileptologists

Over 5000 epilepsy-related articles are indexed annually, posing a challenge for clinicians to stay updated on all relevant research. Clinical Practice Guidelines (CPGs) are vital tools for translating evidence into practice and promoting equitable, high-quality care while addressing practice variations. This review examines CPG applicability for epileptologists, emphasizing the nuances between primary and specialty care, addressing disparities, and comparing guideline usage in the United States and internationally. CPGs are utilized differently across specialties. General practitioners often manage initial epilepsy cases guided by first-seizure and new-onset epilepsy guidelines. Specialists, dealing with complex cases like treatment-resistant epilepsy, face challenges as guidelines may lag behind emerging therapies. Yet, evidence shows specialists heavily rely on CPGs to ensure optimal care. The use of race in medical algorithms highlights disparities, with examples like race-based adjustments in glomerular filtration rate calculations raising equity concerns. While frameworks exist to reduce biases, ongoing monitoring and inclusive approaches are critical. Globally, CPG implementation varies. The UK's centralized system integrates cost-effectiveness analyses, while the United States adopts a decentralized approach prioritizing clinical efficacy. Emerging technologies, such as electronic medical records and clinical decision support systems, improve CPG adoption and patient outcomes. Success stories like the "Get with the Guidelines" stroke program illustrate the potential of structured CPG frameworks. However, challenges persist, such as inconsistencies in epilepsy guidelines for acute seizure management. Ultimately, bridging the gap between evidence and practice requires rigorous, inclusive guideline development, effective communication, and proactive implementation strategies tailored to diverse healthcare systems.

Gerotherapeutics: aging mechanism-based pharmaceutical and behavioral interventions to reduce cancer racial and ethnic disparities

The central premise of this article is that a portion of the established relationships between social determinants of health and racial and ethnic disparities in cancer morbidity and mortality is mediated through differences in rates of biological aging processes. We further posit that using knowledge about aging could enable discovery and testing of new mechanism-based pharmaceutical and behavioral interventions ("gerotherapeutics") to differentially improve the health of cancer survivors from minority populations and reduce cancer disparities. These hypotheses are based on evidence that lifelong differences in adverse social determinants of health contribute to disparities in rates of biological aging ("social determinants of aging"), with individuals from minoritized groups experiencing accelerated aging (ie, a steeper slope or trajectory of biological aging over time relative to chronological age) more often than individuals from nonminoritized groups. Acceleration of biological aging can increase the risk, age of onset, aggressiveness, and stage of many adult cancers. There are also documented negative feedback loops whereby the cellular damage caused by cancer and its therapies act as drivers of additional biological aging. Together, these dynamic intersectional forces can contribute to differences in cancer outcomes between survivors from minoritized vs nonminoritized populations. We highlight key targetable biological aging mechanisms with potential applications to reducing cancer disparities and discuss methodological considerations for preclinical and clinical testing of the impact of gerotherapeutics on cancer outcomes in minoritized populations. Ultimately, the promise of reducing cancer disparities will require broad societal policy changes that address the structural causes of accelerated biological aging and ensure equitable access to all new cancer control paradigms.

The urgency of centering safety-net organizations in AI governance

Although robust AI governance requires the engagement of diverse stakeholders across the artificial intelligence (AI) ecosystem, the US safety net has largely been excluded from this kind of collaboration. Without a reorientation of the AI governance agenda, marginalized patients will disproportionately bear the risks of AI in the US healthcare system. To prevent this replication of digital inequity and an organizational digital divide, we suggest specific next steps for diverse stakeholders to progress toward more equitable policy and practice.

Towards machine learning fairness in classifying multicategory causes of deaths in colorectal or lung cancer patients

Classification of patient multicategory survival outcomes is important for personalized cancer treatments. Machine Learning (ML) algorithms have increasingly been used to inform healthcare decisions, but these models are vulnerable to biases in data collection and algorithm creation. ML models have previously been shown to exhibit racial bias, but their fairness towards patients from different age and sex groups have yet to be studied. Therefore, we compared the multimetric performances of 5 ML models (random forests, multinomial logistic regression, linear support vector classifier, linear discriminant analysis, and multilayer perceptron) when classifying colorectal cancer patients ( n =515) of various age, sex, and racial groups using the TCGA data. All five models exhibited biases for these sociodemographic groups. We then repeated the same process on lung adenocarcinoma ( n =589) to validate our findings. Surprisingly, most models tended to perform more poorly overall for the largest sociodemographic groups. Methods to optimize model performance, including testing the model on merged age, sex, or racial groups, and creating a model trained on and used for an individual or merged sociodemographic group, show potential to reduce disparities in model performance for different groups. Notably, these methods may be used to improve ML fairness while avoiding penalizing the model for exhibiting bias and thus sacrificing overall performance.

Clinical trials informed framework for real world clinical implementation and deployment of artificial intelligence applications

With rapidly evolving artificial intelligence solutions, healthcare organizations need an implementation roadmap. A "clinical trials" informed approach can promote safe and impactful implementation of artificial intelligence. This framework includes four phases: (1) Safety; (2) Efficacy; (3) Effectiveness and comparison to an existing standard; and (4) Monitoring. Combined with inter-institutional collaboration and national funding support, this approach will advance safe, usable, effective, and equitable deployments of artificial intelligence in healthcare.

American Association of Clinical Endocrinology Clinical Practice Guideline on Pharmacologic Management of Adults With Dyslipidemia

OBJECTIVE

To review the evidence and provide updated and new recommendations for the pharmacologic management of adults with dyslipidemia to prevent adverse cardiovascular outcomes. These recommendations are intended for use by clinicians, health care team members, patients, caregivers, and other stakeholders.

METHODS

This guideline was developed by a multidisciplinary task force of content experts and guideline methodologists based on systematic reviews of randomized controlled trials or cohort studies from database inception to November 7, 2023. An updated literature search was completed for any additional articles published by May 31, 2024. Clinical questions addressing nonstatin medications and patient-important outcomes were prioritized. The task force assessed the certainty of the evidence and developed recommendations using the Grading of Recommendations Assessment, Development, and Evaluation framework. All recommendations were based on the consideration of the certainty of the evidence across patient-important outcomes, in addition to issues of feasibility, acceptability, equity, and patient preferences and values.

RESULTS

This guideline update includes 13 evidence-based recommendations for the pharmacologic management of adults with dyslipidemia focused on patient-important outcomes of atherosclerotic cardiovascular disease (ASCVD) risk reduction. The task force issued a good practice statement to assess the risk of ASCVD events for primary prevention in adults with dyslipidemia. The task force suggested the use of alirocumab, evolocumab, or bempedoic acid for adults who have ASCVD or who are at increased risk for ASCVD in addition to standard care. The task force suggested against the use of these medications in adults without ASCVD. There was insufficient evidence to recommend for or against the addition of inclisiran. For adults with hypertriglyceridemia and ASCVD or increased risk of ASCVD, the task force suggested the use of eicosapentaenoic acid but not eicosapentaenoic acid plus docosahexaenoic acid and strongly recommended against the use of niacin. There was insufficient evidence for recommendations regarding pharmacologic management in adults with severe hypertriglyceridemia (≥500 mg/dL). The task force suggested a low-density lipoprotein cholesterol treatment goal of <70 mg/dL in adults with dyslipidemia and ASCVD or at increased risk of ASCVD.

CONCLUSIONS

Pharmacotherapy is recommended in adults with dyslipidemia to reduce the risk of ASCVD events. There are several effective and safe treatment options for adults with dyslipidemia who have ASCVD or at increased risk of ASCVD who need additional lipid-lowering medications. Shared decision-making discussions are essential to determine the best option for each individual.

Sociodemographic bias in clinical machine learning models: a scoping review of algorithmic bias instances and mechanisms

BACKGROUND AND OBJECTIVES

Clinical machine learning (ML) technologies can sometimes be biased and their use could exacerbate health disparities. The extent to which bias is present, the groups who most frequently experience bias, and the mechanism through which bias is introduced in clinical ML applications is not well described. The objective of this study was to examine instances of bias in clinical ML models. We identified the sociodemographic subgroups PROGRESS that experienced bias and the reported mechanisms of bias introduction.

METHODS

We searched MEDLINE, EMBASE, PsycINFO, and Web of Science for all studies that evaluated bias on sociodemographic factors within ML algorithms created for the purpose of facilitating clinical care. The scoping review was conducted according to the Joanna Briggs Institute guide and reported using the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) extension for scoping reviews.

RESULTS

We identified 6448 articles, of which 760 reported on a clinical ML model and 91 (12.0%) completed a bias evaluation and met all inclusion criteria. Most studies evaluated a single sociodemographic factor (n = 56, 61.5%). The most frequently evaluated sociodemographic factor was race (n = 59, 64.8%), followed by sex/gender (n = 41, 45.1%), and age (n = 24, 26.4%), with one study (1.1%) evaluating intersectional factors. Of all studies, 74.7% (n = 68) reported that bias was present, 18.7% (n = 17) reported bias was not present, and 6.6% (n = 6) did not state whether bias was present. When present, 87% of studies reported bias against groups with socioeconomic disadvantage.

CONCLUSION

Most ML algorithms that were evaluated for bias demonstrated bias on sociodemographic factors. Furthermore, most bias evaluations concentrated on race, sex/gender, and age, while other sociodemographic factors and their intersection were infrequently assessed. Given potential health equity implications, bias assessments should be completed for all clinical ML models.

A scoping review and expert consensus on digital determinants of health

Objective

To map how social, commercial, political and digital determinants of health have changed or emerged during the recent digital transformation of society and to identify priority areas for policy action.

Methods

We systematically searched MEDLINE, Embase and Web of Science on 24 September 2023, to identify eligible reviews published in 2018 and later. To ensure we included the most recent literature, we supplemented our review with non-systematic searches in PubMed® and Google Scholar, along with records identified by subject matter experts. Using thematic analysis, we clustered the extracted data into five societal domains affected by digitalization. The clustering also informed a novel framework, which the authors and contributors reviewed for comprehensiveness and accuracy. Using a two-round consensus process, we rated the identified determinants into high, moderate and low urgency for policy actions.

Findings

We identified 13 804 records, of which 204 met the inclusion criteria. A total of 127 health determinants were found to have emerged or changed during the digital transformation of society (37 digital, 33 social, 33 commercial and economic and 24 political determinants). Of these, 30 determinants (23.6%) were considered particularly urgent for policy action.

Conclusion

This review offers a comprehensive overview of health determinants across digital, social, commercial and economic, and political domains, highlighting how policy decisions, individual behaviours and broader factors influence health by digitalization. The findings deepen our understanding of how health outcomes manifest within a digital ecosystem and inform strategies for addressing the complex and evolving networks of health determinants.

The recent history and near future of digital health in the field of behavioral medicine: an update on progress from 2019 to 2024

The field of behavioral medicine has a long and successful history of leveraging digital health tools to promote health behavior change. Our 2019 summary of the history and future of digital health in behavioral medicine (Arigo in J Behav Med 8: 67-83, 2019) was one of the most highly cited articles in the Journal of Behavioral Medicine from 2010 to 2020; here, we provide an update on the opportunities and challenges we identified in 2019. We address the impact of the COVID-19 pandemic on behavioral medicine research and practice and highlight some of the digital health advances it prompted. We also describe emerging challenges and opportunities in the evolving ecosystem of digital health in the field of behavioral medicine, including the emergence of new evidence, research methods, and tools to promote health and health behaviors. Specifically, we offer updates on advanced research methods, the science of digital engagement, dissemination and implementation science, and artificial intelligence technologies, including examples of uses in healthcare and behavioral medicine. We also provide recommendations for next steps in these areas with attention to ethics, training, and accessibility considerations. The field of behavioral medicine has made meaningful advances since 2019 and continues to evolve with impressive pace and innovation.

The bias algorithm: how AI in healthcare exacerbates ethnic and racial disparities - a scoping review

This scoping review examined racial and ethnic bias in artificial intelligence health algorithms (AIHA), the role of stakeholders in oversight, and the consequences of AIHA for health equity. Using the PRISMA-ScR guidelines, databases were searched between 2020 and 2024 using the terms racial and ethnic bias in health algorithms resulting in a final sample of 23 sources. Suggestions for how to mitigate algorithmic bias were compiled and evaluated, roles played by stakeholders were identified, and governance and stewardship plans for AIHA were examined. While AIHA represent a significant breakthrough in predictive analytics and treatment optimization, regularly outperforming humans in diagnostic precision and accuracy, they also present serious challenges to patient privacy, data security, institutional transparency, and health equity. Evidence from extant sources including those in this review showed that AIHA carry the potential to perpetuate health inequities. While the current study considered AIHA in the US, the use of AIHA carries implications for global health equity.

Fairness in Low Birthweight Predictive Models: Implications of Excluding Race/Ethnicity

CONTEXT

To evaluate algorithmic fairness in low birthweight predictive models.

STUDY DESIGN

This study analyzed insurance claims (n = 9,990,990; 2013-2021) linked with birth certificates (n = 173,035; 2014-2021) from the Arkansas All Payers Claims Database (APCD).

METHODS

Low birthweight (< 2500 g) predictive models included four approaches (logistic, elastic net, linear discriminate analysis, and gradient boosting machines [GMB]) with and without racial/ethnic information. Model performance was assessed overall, among Hispanic individuals, and among non-Hispanic White, Black, Native Hawaiian/Other Pacific Islander, and Asian individuals using multiple measures of predictive performance (i.e., AUC [area under the receiver operating characteristic curve] scores, calibration, sensitivity, and specificity).

RESULTS

AUC scores were lower (underperformed) for Black and Asian individuals relative to White individuals. In the strongest performing model (i.e., GMB), the AUC scores for Black (0.718 [95% CI: 0.705-0.732]) and Asian (0.655 [95% CI: 0.582-0.728]) populations were lower than the AUC for White individuals (0.764 [95% CI: 0.754-0.775 ]). Model performance measured using AUC was comparable in models that included and excluded race/ethnicity; however, sensitivity (i.e., the percent of records correctly predicted as "low birthweight" among those who actually had low birthweight) was lower and calibration was weaker, suggesting underprediction for Black individuals when race/ethnicity were excluded.

CONCLUSIONS

This study found that racially blind models resulted in underprediction and reduced algorithmic performance, measured using sensitivity and calibration, for Black populations. Such under prediction could unfairly decrease resource allocation needed to reduce perinatal health inequities. Population health management programs should carefully consider algorithmic fairness in predictive models and associated resource allocation decisions.

Era of Generalist Conversational Artificial Intelligence to Support Public Health Communications

The integration of artificial intelligence (AI) into health communication systems has introduced a transformative approach to public health management, particularly during public health emergencies, capable of reaching billions through familiar digital channels. This paper explores the utility and implications of generalist conversational artificial intelligence (CAI) advanced AI systems trained on extensive datasets to handle a wide range of conversational tasks across various domains with human-like responsiveness. The specific focus is on the application of generalist CAI within messaging services, emphasizing its potential to enhance public health communication. We highlight the evolution and current applications of AI-driven messaging services, including their ability to provide personalized, scalable, and accessible health interventions. Specifically, we discuss the integration of large language models and generative AI in mainstream messaging platforms, which potentially outperform traditional information retrieval systems in public health contexts. We report a critical examination of the advantages of generalist CAI in delivering health information, with a case of its operationalization during the COVID-19 pandemic and propose the strategic deployment of these technologies in collaboration with public health agencies. In addition, we address significant challenges and ethical considerations, such as AI biases, misinformation, privacy concerns, and the required regulatory oversight. We envision a future with leverages generalist CAI in messaging apps, proposing a multiagent approach to enhance the reliability and specificity of health communications. We hope this commentary initiates the necessary conversations and research toward building evaluation approaches, adaptive strategies, and robust legal and technical frameworks to fully realize the benefits of AI-enhanced communications in public health, aiming to ensure equitable and effective health outcomes across diverse populations.

Harnessing artificial intelligence in sepsis care: advances in early detection, personalized treatment, and real-time monitoring

Sepsis remains a leading cause of morbidity and mortality worldwide due to its rapid progression and heterogeneous nature. This review explores the potential of Artificial Intelligence (AI) to transform sepsis management, from early detection to personalized treatment and real-time monitoring. AI, particularly through machine learning (ML) techniques such as random forest models and deep learning algorithms, has shown promise in analyzing electronic health record (EHR) data to identify patterns that enable early sepsis detection. For instance, random forest models have demonstrated high accuracy in predicting sepsis onset in intensive care unit (ICU) patients, while deep learning approaches have been applied to recognize complications such as sepsis-associated acute respiratory distress syndrome (ARDS). Personalized treatment plans developed through AI algorithms predict patient-specific responses to therapies, optimizing therapeutic efficacy and minimizing adverse effects. AI-driven continuous monitoring systems, including wearable devices, provide real-time predictions of sepsis-related complications, enabling timely interventions. Beyond these advancements, AI enhances diagnostic accuracy, predicts long-term outcomes, and supports dynamic risk assessment in clinical settings. However, ethical challenges, including data privacy concerns and algorithmic biases, must be addressed to ensure fair and effective implementation. The significance of this review lies in addressing the current limitations in sepsis management and highlighting how AI can overcome these hurdles. By leveraging AI, healthcare providers can significantly enhance diagnostic accuracy, optimize treatment protocols, and improve overall patient outcomes. Future research should focus on refining AI algorithms with diverse datasets, integrating emerging technologies, and fostering interdisciplinary collaboration to address these challenges and realize AI's transformative potential in sepsis care.

The Impact of Race, Ethnicity, and Sex on Fairness in Artificial Intelligence for Glaucoma Prediction Models

Objective

Despite advances in artificial intelligence (AI) in glaucoma prediction, most works lack multicenter focus and do not consider fairness concerning sex, race, or ethnicity. This study aims to examine the impact of these sensitive attributes on developing fair AI models that predict glaucoma progression to necessitating incisional glaucoma surgery.

Design

Database study.

Participants

Thirty-nine thousand ninety patients with glaucoma, as identified by International Classification of Disease codes from 7 academic eye centers participating in the Sight OUtcomes Research Collaborative.

Methods

We developed XGBoost models using 3 approaches: (1) excluding sensitive attributes as input features, (2) including them explicitly as input features, and (3) training separate models for each group. Model input features included demographic details, diagnosis codes, medications, and clinical information (intraocular pressure, visual acuity, etc.), from electronic health records. The models were trained on patients from 5 sites (N = 27 999) and evaluated on a held-out internal test set (N = 3499) and 2 external test sets consisting of N = 1550 and N = 2542 patients.

Main Outcomes and Measures

Area under the receiver operating characteristic curve (AUROC) and equalized odds on the test set and external sites.

Results

Six thousand six hundred eighty-two (17.1%) of 39 090 patients underwent glaucoma surgery with a mean age of 70.1 (standard deviation 14.6) years, 54.5% female, 62.3% White, 22.1% Black, and 4.7% Latinx/Hispanic. We found that not including the sensitive attributes led to better classification performance (AUROC: 0.77-0.82) but worsened fairness when evaluated on the internal test set. However, on external test sites, the opposite was true: including sensitive attributes resulted in better classification performance (AUROC: external #1 - [0.73-0.81], external #2 - [0.67-0.70]), but varying degrees of fairness for sex and race as measured by equalized odds.

Conclusions

Artificial intelligence models predicting whether patients with glaucoma progress to surgery demonstrated bias with respect to sex, race, and ethnicity. The effect of sensitive attribute inclusion and exclusion on fairness and performance varied based on internal versus external test sets. Prior to deployment, AI models should be evaluated for fairness on the target population.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Literacy and Language Barriers to Overcome in Laboratory Medicine

In the context of laboratory medicine, the authors describe 3 barriers to health communication: access, health communication, and language responsiveness. These barriers are interconnected and present in millions of people in need of equitable access to health communication. Equitable access entails health communication written in plain language and languages other than English to address language and literacy barriers and increase trust by avoiding language discordance and the spread of infodemics. This review includes several options to implement multidisciplinary efforts that lead to measurable improvements in health literacy.

Testing process for artificial intelligence applications in radiology practice

Artificial intelligence (AI) applications are becoming increasingly common in radiology. However, ensuring reliable operation and expected clinical benefits remains a challenge. A systematic testing process aims to facilitate clinical deployment by confirming software applicability to local patient populations, practises, adherence to regulatory and safety requirements, and compatibility with existing systems. In this work, we present our testing process developed based on practical experience. First, a survey and pre-evaluation is conducted, where information requests are sent for potential products, and the specifications are evaluated against predetermined requirements. In the second phase, data collection, testing, and analysis are conducted. In the retrospective stage, the application undergoes testing with a pre selected dataset and is evaluated against specified key performance indicators (KPIs). In the prospective stage, the application is integrated into the clinical workflow and evaluated with additional process-specific KPIs. In the final phase, the results are evaluated in terms of safety, effectiveness, productivity, and integration. The final report summarises the results and includes a procurement/deployment or rejection recommendation. The process allows termination at any phase if the application fails to meet essential criteria. In addition, we present practical remarks from our experiences in AI testing and provide forms to guide and document the testing process. The established AI testing process facilitates a systematic evaluation and documentation of new technologies ensuring that each application undergoes equal and sufficient validation. Testing with local data is crucial for identifying biases and pitfalls of AI algorithms to improve the quality and safety, ultimately benefiting patient care.

Health Risk Assessment Using Machine Learning: Systematic Review

According to the World Health Organization, chronic illnesses account for over 70% of deaths globally, underscoring the need for effective health risk assessment (HRA). While machine learning (ML) has shown potential in enhancing HRA, no systematic review has explored its application in general health risk assessments. Existing reviews typically focus on specific conditions. This paper reviews published articles that utilize ML for HRA, and it aims to identify the model development methods. A systematic review following Tranfield et al.’s three-stage approach was conducted, and it adhered to the PRISMA protocol. The literature was sourced from five databases, including PubMed. Of the included articles, 42% (11/26) addressed general health risks. Secondary data sources were most common (14/26, 53.85%), while primary data were used in eleven studies, with nine (81.81%) using data from a specific population. Random forest was the most popular algorithm, which was used in nine studies (34.62%). Notably, twelve studies implemented multiple algorithms, while seven studies incorporated model interpretability techniques. Although these studies have shown promise in addressing digital health inequities, more research is needed to include diverse sample populations, particularly from underserved communities, to enhance the generalizability of existing models. Furthermore, model interpretability should be prioritized to ensure transparent, trustworthy, and broadly applicable healthcare solutions.

Simulated misuse of large language models and clinical credit systems

In the future, large language models (LLMs) may enhance the delivery of healthcare, but there are risks of misuse. These methods may be trained to allocate resources via unjust criteria involving multimodal data - financial transactions, internet activity, social behaviors, and healthcare information. This study shows that LLMs may be biased in favor of collective/systemic benefit over the protection of individual rights and could facilitate AI-driven social credit systems.

ChatGPT-4 Omni Performance in USMLE Disciplines and Clinical Skills: Comparative Analysis

Background

Recent studies, including those by the National Board of Medical Examiners, have highlighted the remarkable capabilities of recent large language models (LLMs) such as ChatGPT in passing the United States Medical Licensing Examination (USMLE). However, there is a gap in detailed analysis of LLM performance in specific medical content areas, thus limiting an assessment of their potential utility in medical education.

Objective

This study aimed to assess and compare the accuracy of successive ChatGPT versions (GPT-3.5, GPT-4, and GPT-4 Omni) in USMLE disciplines, clinical clerkships, and the clinical skills of diagnostics and management.

Methods

This study used 750 clinical vignette-based multiple-choice questions to characterize the performance of successive ChatGPT versions (ChatGPT 3.5 [GPT-3.5], ChatGPT 4 [GPT-4], and ChatGPT 4 Omni [GPT-4o]) across USMLE disciplines, clinical clerkships, and in clinical skills (diagnostics and management). Accuracy was assessed using a standardized protocol, with statistical analyses conducted to compare the models' performances.

Results

GPT-4o achieved the highest accuracy across 750 multiple-choice questions at 90.4%, outperforming GPT-4 and GPT-3.5, which scored 81.1% and 60.0%, respectively. GPT-4o's highest performances were in social sciences (95.5%), behavioral and neuroscience (94.2%), and pharmacology (93.2%). In clinical skills, GPT-4o's diagnostic accuracy was 92.7% and management accuracy was 88.8%, significantly higher than its predecessors. Notably, both GPT-4o and GPT-4 significantly outperformed the medical student average accuracy of 59.3% (95% CI 58.3-60.3).

Conclusions

GPT-4o's performance in USMLE disciplines, clinical clerkships, and clinical skills indicates substantial improvements over its predecessors, suggesting significant potential for the use of this technology as an educational aid for medical students. These findings underscore the need for careful consideration when integrating LLMs into medical education, emphasizing the importance of structured curricula to guide their appropriate use and the need for ongoing critical analyses to ensure their reliability and effectiveness.

Raising awareness of potential biases in medical machine learning: Experience from a Datathon

Objective

To challenge clinicians and informaticians to learn about potential sources of bias in medical machine learning models through investigation of data and predictions from an open-source severity of illness score.

Methods

Over a two-day period (total elapsed time approximately 28 hours), we conducted a datathon that challenged interdisciplinary teams to investigate potential sources of bias in the Global Open Source Severity of Illness Score. Teams were invited to develop hypotheses, to use tools of their choosing to identify potential sources of bias, and to provide a final report.

Results

Five teams participated, three of which included both informaticians and clinicians. Most (4/5) used Python for analyses, the remaining team used R. Common analysis themes included relationship of the GOSSIS-1 prediction score with demographics and care related variables; relationships between demographics and outcomes; calibration and factors related to the context of care; and the impact of missingness. Representativeness of the population, differences in calibration and model performance among groups, and differences in performance across hospital settings were identified as possible sources of bias.

Discussion

Datathons are a promising approach for challenging developers and users to explore questions relating to unrecognized biases in medical machine learning algorithms.

Equity in Using Artificial Intelligence Mortality Predictions to Target Goals of Care Documentation

BACKGROUND

Artificial intelligence (AI) algorithms are increasingly used to target patients with elevated mortality risk scores for goals-of-care (GOC) conversations.

OBJECTIVE

To evaluate the association between the presence or absence of AI-generated mortality risk scores with GOC documentation.

DESIGN

Retrospective cross-sectional study at one large academic medical center between July 2021 and December 2022.

PARTICIPANTS

Hospitalized adult patients with AI-defined Serious Illness Risk Indicator (SIRI) scores indicating > 30% 90-day mortality risk (defined as "elevated" SIRI) or no SIRI scores due to insufficient data.

INTERVENTION

A targeted intervention to increase GOC documentation for patients with AI-generated scores predicting elevated risk of mortality.

MAIN MEASURES

Odds ratios comparing GOC documentation for patients with elevated or no SIRI scores with similar severity of illness using propensity score matching and risk-adjusted mixed-effects logistic regression.

KEY RESULTS

Among 13,710 patients with elevated (n = 3643, 27%) or no (n = 10,067, 73%) SIRI scores, the median age was 64 years (SD 18). Twenty-five percent were non-White, 18% had Medicaid, 43% were admitted to an intensive care unit, and 11% died during admission. Patients lacking SIRI scores were more likely to be younger (median 60 vs. 72 years, p < 0.0001), be non-White (29% vs. 13%, p < 0.0001), and have Medicaid (22% vs. 9%, p < 0.0001). Patients with elevated versus no SIRI scores were more likely to have GOC documentation in the unmatched (aOR 2.5, p < 0.0001) and propensity-matched cohorts (aOR 2.1, p < 0.0001).

CONCLUSIONS

Using AI predictions of mortality to target GOC documentation may create differences in documentation prevalence between patients with and without AI mortality prediction scores with similar severity of illness. These finding suggest using AI to target GOC documentation may have the unintended consequence of disadvantaging severely ill patients lacking AI-generated scores from receiving targeted GOC documentation, including patients who are more likely to be non-White and have Medicaid insurance.

Core Concepts in Pharmacoepidemiology: Principled Use of Artificial Intelligence and Machine Learning in Pharmacoepidemiology and Healthcare Research

Artificial intelligence (AI) and machine learning (ML) are important tools across many fields of health and medical research. Pharmacoepidemiologists can bring essential methodological rigor and study design expertise to the design and use of these technologies within healthcare settings. AI/ML-based tools also play a role in pharmacoepidemiology research, as we may apply them to answer our own research questions, take responsibility for evaluating medical devices with AI/ML components, or participate in interdisciplinary research to create new AI/ML algorithms. While epidemiologic expertise is essential to deploying AI/ML responsibly and ethically, the rapid advancement of these technologies in the past decade has resulted in a knowledge gap for many in the field. This article provides a brief overview of core AI/ML concepts, followed by a discussion of potential applications of AI/ML in pharmacoepidemiology research, and closes with a review of important concepts across application areas, including interpretability and fairness. This review is intended to provide an accessible, practical overview of AI/ML for pharmacoepidemiology research, with references to further, more detailed resources on fundamental topics.

Machine learning methods for propensity and disease risk score estimation in high-dimensional data: a plasmode simulation and real-world data cohort analysis

Introduction

Machine learning (ML) methods are promising and scalable alternatives for propensity score (PS) estimation, but their comparative performance in disease risk score (DRS) estimation remains unexplored.

Methods

We used real-world data comparing antihypertensive users to non-users with 69 negative control outcomes, and plasmode simulations to study the performance of ML methods in PS and DRS estimation. We conducted a cohort study using UK primary care records. Further, we conducted a plasmode simulation with synthetic treatment and outcome mimicking empirical data distributions. We compared four PS and DRS estimation methods: 1. Reference: Logistic regression including clinically chosen confounders. 2. Logistic regression with L1 regularisation (LASSO). 3. Multi-layer perceptron (MLP). 4. Extreme Gradient Boosting (XgBoost). Covariate balance, coverage of the null effect of negative control outcomes (real-world data) and bias based on the absolute difference between observed and true effects (for plasmode) were estimated. 632,201 antihypertensive users and nonusers were included.

Results

ML methods outperformed the reference method for PS estimation in some scenarios, both in terms of covariate balance and coverage/bias. Specifically, XgBoost achieved the best performance. DRS-based methods performed worse than PS in all tested scenarios.

Discussion

We found that ML methods could be reliable alternatives for PS estimation. ML-based DRS methods performed worse than PS ones, likely given the rarity of outcomes.

Machine learning applications in studying mental health among immigrants and racial and ethnic minorities: an exploratory scoping review

BACKGROUND

The use of machine learning (ML) in mental health (MH) research is increasing, especially as new, more complex data types become available to analyze. By examining the published literature, this review aims to explore the current applications of ML in MH research, with a particular focus on its use in studying diverse and vulnerable populations, including immigrants, refugees, migrants, and racial and ethnic minorities.

METHODS

From October 2022 to March 2024, Google Scholar, EMBASE, and PubMed were queried. ML-related, MH-related, and population-of-focus search terms were strung together with Boolean operators. Backward reference searching was also conducted. Included peer-reviewed studies reported using a method or application of ML in an MH context and focused on the populations of interest. We did not have date cutoffs. Publications were excluded if they were narrative or did not exclusively focus on a minority population from the respective country. Data including study context, the focus of mental healthcare, sample, data type, type of ML algorithm used, and algorithm performance were extracted from each.

RESULTS

Ultimately, 13 peer-reviewed publications were included. All the articles were published within the last 6 years, and over half of them studied populations within the US. Most reviewed studies used supervised learning to explain or predict MH outcomes. Some publications used up to 16 models to determine the best predictive power. Almost half of the included publications did not discuss their cross-validation method.

CONCLUSIONS

The included studies provide proof-of-concept for the potential use of ML algorithms to address MH concerns in these special populations, few as they may be. Our review finds that the clinical application of these models for classifying and predicting MH disorders is still under development.

Language disparities in pandemic information: Autocomplete analysis of COVID-19 searches in New York

Objective: To audit and compare search autocomplete results in Spanish and English during the early COVID-19 pandemic in the New York metropolitan area. The pandemic led to significant online search activity about the disease, its spread, and remedies. As gatekeepers, search engines like Google can influence public opinion. Autocomplete predictions help users complete searches faster but may also shape their views. Understanding these differences is crucial to identify biases and ensure equitable information dissemination. Methods: The study tracked autocomplete results daily for five COVID-19 related search terms in English and Spanish over 100+ days in 2020, yielding a total of 9164 autocomplete predictions. Results: Queries in Spanish yielded fewer autocomplete options and often included more negative content than English autocompletes. The topical coverage differed, with Spanish autocompletes including themes related to religion and spirituality that were absent in the English search autocompletes. Conclusion: The contrast in search autocomplete results could lead to divergent impressions about the pandemic and remedial actions among different sections of society. Continuous auditing of autocompletes by public health stakeholders and search engine organizations is recommended to reduce potential bias and misinformation.

Predictive Tools in Charity Care-Revenue vs Access

This Viewpoint discusses possible outcomes of predictive analytic tool use in charity care determinations: hospital revenue and patient debt.

Dynamic and Transdiagnostic Risk Calculator Based on Natural Language Processing for the Prediction of Psychosis in Secondary Mental Health Care: Development and Internal-External Validation Cohort Study

BACKGROUND

Automatic transdiagnostic risk calculators can improve the detection of individuals at risk of psychosis. However, they rely on assessment at a single point in time and can be refined with dynamic modeling techniques that account for changes in risk over time.

METHODS

We included 158,139 patients (5007 events) who received a first index diagnosis of a nonorganic and nonpsychotic mental disorder within electronic health records from the South London and Maudsley National Health Service Foundation Trust between January 1, 2008, and October 8, 2021. A dynamic Cox landmark model was developed to estimate the 2-year risk of developing psychosis according to the TRIPOD (Transparent Reporting of a multivariate prediction model for Individual Prognosis or Diagnosis) statement. The dynamic model included 24 predictors extracted at 9 landmark points (baseline, 0, 6, 12, 24, 30, 36, 42, and 48 months): 3 demographic, 1 clinical, and 20 natural language processing-based symptom and substance use predictors. Performance was compared with a static Cox regression model with all predictors assessed at baseline only and indexed via discrimination (C-index), calibration (calibration plots), and potential clinical utility (decision curves) in internal-external validation.

RESULTS

The dynamic model improved discrimination performance from baseline compared with the static model (dynamic: C-index = 0.9; static: C-index = 0.87) and the final landmark point (dynamic: C-index = 0.79; static: C-index = 0.76). The dynamic model was also significantly better calibrated (calibration slope = 0.97-1.1) than the static model at later landmark points (≥24 months). Net benefit was higher for the dynamic than for the static model at later landmark points (≥24 months).

CONCLUSIONS

These findings suggest that dynamic prediction models can improve the detection of individuals at risk for psychosis in secondary mental health care settings.

Artificial Intelligence in Otolaryngology: Topics in Epistemology & Ethics

To fuel artificial intelligence (AI) potential in clinical practice in otolaryngology, researchers must understand its epistemic limitations, which are tightly linked to ethical dilemmas requiring careful consideration. AI tools are fundamentally opaque systems, though there are methods to increase explainability and transparency. Reproducibility and replicability limitations can be overcomed by sharing computing code, raw data, and data processing methodology. The risk of bias can be mitigated via algorithmic auditing, careful consideration of the training data, and advocating for a diverse AI workforce to promote algorithmic pluralism, reflecting our population's diverse values and preferences.

Simulated Misuse of Large Language Models and Clinical Credit Systems

Large language models (LLMs) have been proposed to support many healthcare tasks, including disease diagnostics and treatment personalization. While AI may be applied to assist or enhance the delivery of healthcare, there is also a risk of misuse. LLMs could be used to allocate resources via unfair, unjust, or inaccurate criteria. For example, a social credit system uses big data to assess "trustworthiness" in society, penalizing those who score poorly based on evaluation metrics defined only by a power structure (e.g., a corporate entity or governing body). Such a system may be amplified by powerful LLMs which can evaluate individuals based on multimodal data - financial transactions, internet activity, and other behavioral inputs. Healthcare data is perhaps the most sensitive information which can be collected and could potentially be used to violate civil liberty or other rights via a "clinical credit system", which may include limiting access to care. The results of this study show that LLMs may be biased in favor of collective or systemic benefit over protecting individual rights, potentially enabling this type of future misuse. Moreover, experiments in this report simulate how clinical datasets might be exploited with current LLMs, demonstrating the urgency of addressing these ethical dangers. Finally, strategies are proposed to mitigate the risk of developing large AI models for healthcare.

The potential role for artificial intelligence in fracture risk prediction

Osteoporotic fractures are a major health challenge in older adults. Despite the availability of safe and effective therapies for osteoporosis, these therapies are underused in individuals at high risk for fracture, calling for better case-finding and fracture risk assessment strategies. Artificial intelligence (AI) and machine learning (ML) hold promise for enhancing identification of individuals at high risk for fracture by distilling useful features from high-dimensional data derived from medical records, imaging, and wearable devices. AI-ML could enable automated opportunistic screening for vertebral fractures and osteoporosis, home-based monitoring and intervention targeting lifestyle factors, and integration of multimodal features to leverage fracture prediction, ultimately aiding improved fracture risk assessment and individualised treatment. Optimism must be balanced with consideration for the explainability of AI-ML models, biases (including information inequity in numerically under-represented populations), model limitations, and net clinical benefit and workload impact. Clinical integration of AI-ML algorithms has the potential to transform osteoporosis management, offering a more personalised approach to reduce the burden of osteoporotic fractures.

Fairness in Predicting Cancer Mortality Across Racial Subgroups

Importance

Machine learning has potential to transform cancer care by helping clinicians prioritize patients for serious illness conversations. However, models need to be evaluated for unequal performance across racial groups (ie, racial bias) so that existing racial disparities are not exacerbated.

Objective

To evaluate whether racial bias exists in a predictive machine learning model that identifies 180-day cancer mortality risk among patients with solid malignant tumors.

Design, Setting, and Participants

In this cohort study, a machine learning model to predict cancer mortality for patients aged 21 years or older diagnosed with cancer between January 2016 and December 2021 was developed with a random forest algorithm using retrospective data from the Mount Sinai Health System cancer registry, Social Security Death Index, and electronic health records up to the date when databases were accessed for cohort extraction (February 2022).

Exposure

Race category.

Main Outcomes and Measures

The primary outcomes were model discriminatory performance (area under the receiver operating characteristic curve [AUROC], F1 score) among each race category (Asian, Black, Native American, White, and other or unknown) and fairness metrics (equal opportunity, equalized odds, and disparate impact) among each pairwise comparison of race categories. True-positive rate ratios represented equal opportunity; both true-positive and false-positive rate ratios, equalized odds; and the percentage of predictive positive rate ratios, disparate impact. All metrics were estimated as a proportion or ratio, with variability captured through 95% CIs. The prespecified criterion for the model's clinical use was a threshold of at least 80% for fairness metrics across different racial groups to ensure the model's prediction would not be biased against any specific race.

Results

The test validation dataset included 43 274 patients with balanced demographics. Mean (SD) age was 64.09 (14.26) years, with 49.6% older than 65 years. A total of 53.3% were female; 9.5%, Asian; 18.9%, Black; 0.1%, Native American; 52.2%, White; and 19.2%, other or unknown race; 0.1% had missing race data. A total of 88.9% of patients were alive, and 11.1% were dead. The AUROCs, F1 scores, and fairness metrics maintained reasonable concordance among the racial subgroups: the AUROCs ranged from 0.75 (95% CI, 0.72-0.78) for Asian patients and 0.75 (95% CI, 0.73-0.77) for Black patients to 0.77 (95% CI, 0.75-0.79) for patients with other or unknown race; F1 scores, from 0.32 (95% CI, 0.32-0.33) for White patients to 0.40 (95% CI, 0.39-0.42) for Black patients; equal opportunity ratios, from 0.96 (95% CI, 0.95-0.98) for Black patients compared with White patients to 1.02 (95% CI, 1.00-1.04) for Black patients compared with patients with other or unknown race; equalized odds ratios, from 0.87 (95% CI, 0.85-0.92) for Black patients compared with White patients to 1.16 (1.10-1.21) for Black patients compared with patients with other or unknown race; and disparate impact ratios, from 0.86 (95% CI, 0.82-0.89) for Black patients compared with White patients to 1.17 (95% CI, 1.12-1.22) for Black patients compared with patients with other or unknown race.

Conclusions and Relevance

In this cohort study, the lack of significant variation in performance or fairness metrics indicated an absence of racial bias, suggesting that the model fairly identified cancer mortality risk across racial groups. It remains essential to consistently review the model's application in clinical settings to ensure equitable patient care.

Recognition of Patient Gender: A Machine Learning Preliminary Analysis Using Heart Sounds from Children and Adolescents

Research has shown that X-rays and fundus images can classify gender, age group, and race, raising concerns about bias and fairness in medical AI applications. However, the potential for physiological sounds to classify sociodemographic traits has not been investigated. Exploring this gap is crucial for understanding the implications and ensuring fairness in the field of medical sound analysis. We aimed to develop classifiers to determine gender (men/women) based on heart sound recordings and using machine learning (ML). Data-driven ML analysis. We utilized the open-access CirCor DigiScope Phonocardiogram Dataset obtained from cardiac screening programs in Brazil. Volunteers < 21 years of age. Each participant completed a questionnaire and underwent a clinical examination, including electronic auscultation at four cardiac points: aortic (AV), mitral (MV), pulmonary (PV), and tricuspid (TV). We used Mel-frequency cepstral coefficients (MFCCs) to develop the ML classifiers. From each patient and from each auscultation sound recording, we extracted 10 MFCCs. In sensitivity analysis, we additionally extracted 20, 30, 40, and 50 MFCCs. The most effective gender classifier was developed using PV recordings (AUC ROC = 70.3%). The second best came from MV recordings (AUC ROC = 58.8%). AV and TV recordings produced classifiers with an AUC ROC of 56.4% and 56.1%, respectively. Using more MFCCs did not substantially improve the classifiers. It is possible to classify between males and females using phonocardiogram data. As health-related audio recordings become more prominent in ML applications, research is required to explore if these recordings contain signals that could distinguish sociodemographic features.

The potential of artificial intelligence to revolutionize health care delivery, research, and education in cardiac electrophysiology

The field of electrophysiology (EP) has benefited from numerous seminal innovations and discoveries that have enabled clinicians to deliver therapies and interventions that save lives and promote quality of life. The rapid pace of innovation in EP may be hindered by several challenges including the aging population with increasing morbidity, the availability of multiple costly therapies that, in many instances, confer minor incremental benefit, the limitations of healthcare reimbursement, the lack of response to therapies by some patients, and the complications of the invasive procedures performed. To overcome these challenges and continue on a steadfast path of transformative innovation, the EP community must comprehensively explore how artificial intelligence (AI) can be applied to healthcare delivery, research, and education and consider all opportunities in which AI can catalyze innovation; create workflow, research, and education efficiencies; and improve patient outcomes at a lower cost. In this white paper, we define AI and discuss the potential of AI to revolutionize the EP field. We also address the requirements for implementing, maintaining, and enhancing quality when using AI and consider ethical, operational, and regulatory aspects of AI implementation. This manuscript will be followed by several perspective papers that will expand on some of these topics.

Preparing for the bedside-optimizing a postpartum depression risk prediction model for clinical implementation in a health system

OBJECTIVE

We developed and externally validated a machine-learning model to predict postpartum depression (PPD) using data from electronic health records (EHRs). Effort is under way to implement the PPD prediction model within the EHR system for clinical decision support. We describe the pre-implementation evaluation process that considered model performance, fairness, and clinical appropriateness.

MATERIALS AND METHODS

We used EHR data from an academic medical center (AMC) and a clinical research network database from 2014 to 2020 to evaluate the predictive performance and net benefit of the PPD risk model. We used area under the curve and sensitivity as predictive performance and conducted a decision curve analysis. In assessing model fairness, we employed metrics such as disparate impact, equal opportunity, and predictive parity with the White race being the privileged value. The model was also reviewed by multidisciplinary experts for clinical appropriateness. Lastly, we debiased the model by comparing 5 different debiasing approaches of fairness through blindness and reweighing.

RESULTS

We determined the classification threshold through a performance evaluation that prioritized sensitivity and decision curve analysis. The baseline PPD model exhibited some unfairness in the AMC data but had a fair performance in the clinical research network data. We revised the model by fairness through blindness, a debiasing approach that yielded the best overall performance and fairness, while considering clinical appropriateness suggested by the expert reviewers.

DISCUSSION AND CONCLUSION

The findings emphasize the need for a thorough evaluation of intervention-specific models, considering predictive performance, fairness, and appropriateness before clinical implementation.