1
|
Mohottige D, Farouk S, Poteat T, Radix A, Witchel SF. Considerations of sex as a binary variable in clinical algorithms. Nat Rev Nephrol 2024; 20:347-348. [PMID: 38654119 DOI: 10.1038/s41581-024-00840-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/25/2024]
Affiliation(s)
- Dinushika Mohottige
- Barbara T. Murphy Division of Nephrology, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
- Institute for Health Equity Research, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
| | - Samira Farouk
- Barbara T. Murphy Division of Nephrology, Department of Medicine, Icahn School of Medicine at Mount Sinai, New York, NY, USA
- Recanati Miller Transplant Institute, Mount Sinai Hospital, New York, NY, USA
| | - Tonia Poteat
- Division of Health Care in Adult Populations, Duke University School of Nursing, Durham, NC, USA
| | - Asa Radix
- Callen-Lorde Community Health Center, New York, NY, USA
- Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, NY, USA
| | - Selma Feldman Witchel
- Division of Pediatric Endocrinology, Department of Pediatrics, UPMC Children's Hospital of Pittsburgh, University of Pittsburgh, Pittsburgh, PA, USA
| |
Collapse
|
2
|
Greenhalgh T, Engebretsen E. Pragmatism and crisis: A response to three commentaries. Soc Sci Med 2024; 348:116782. [PMID: 38538379 DOI: 10.1016/j.socscimed.2024.116782] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Accepted: 03/11/2024] [Indexed: 04/29/2024]
Affiliation(s)
- Trisha Greenhalgh
- Nuffield Department of Primary Care Health Sciences, University of Oxford, UK.
| | - Eivind Engebretsen
- Centre for Sustainable Healthcare Education, Faculty of Medicine, University of Oslo, Norway
| |
Collapse
|
3
|
Cary MP, De Gagne JC, Kauschinger ED, Carter BM. Advancing Health Equity Through Artificial Intelligence: An Educational Framework for Preparing Nurses in Clinical Practice and Research. Creat Nurs 2024; 30:154-164. [PMID: 38689433 DOI: 10.1177/10784535241249193] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/02/2024]
Abstract
The integration of artificial intelligence (AI) into health care offers the potential to enhance patient care, improve diagnostic precision, and broaden access to health-care services. Nurses, positioned at the forefront of patient care, play a pivotal role in utilizing AI to foster a more efficient and equitable health-care system. However, to fulfil this role, nurses will require education that prepares them with the necessary skills and knowledge for the effective and ethical application of AI. This article proposes a framework for nurses which includes AI principles, skills, competencies, and curriculum development focused on the practical use of AI, with an emphasis on care that aims to achieve health equity. By adopting this educational framework, nurses will be prepared to make substantial contributions to reducing health disparities and fostering a health-care system that is more efficient and equitable.
Collapse
Affiliation(s)
- Michael P Cary
- Duke University School of Nursing, Durham, NC, USA
- Duke University School of Medicine, Durham, NC, USA
- Duke AI Health, Durham, NC, USA
- American Association of Colleges of Nursing, Durham, NC, USA
| | - Jennie C De Gagne
- Duke University School of Nursing, Durham, NC, USA
- Duke University School of Medicine, Durham, NC, USA
- Duke AI Health, Durham, NC, USA
- American Association of Colleges of Nursing, Durham, NC, USA
| | - Elaine D Kauschinger
- Duke University School of Nursing, Durham, NC, USA
- Duke University School of Medicine, Durham, NC, USA
- Duke AI Health, Durham, NC, USA
- American Association of Colleges of Nursing, Durham, NC, USA
| | - Brigit M Carter
- Duke University School of Nursing, Durham, NC, USA
- Duke University School of Medicine, Durham, NC, USA
- Duke AI Health, Durham, NC, USA
- American Association of Colleges of Nursing, Durham, NC, USA
| |
Collapse
|
4
|
Economou-Zavlanos NJ, Bessias S, Cary MP, Bedoya AD, Goldstein BA, Jelovsek JE, O’Brien CL, Walden N, Elmore M, Parrish AB, Elengold S, Lytle KS, Balu S, Lipkin ME, Shariff AI, Gao M, Leverenz D, Henao R, Ming DY, Gallagher DM, Pencina MJ, Poon EG. Translating ethical and quality principles for the effective, safe and fair development, deployment and use of artificial intelligence technologies in healthcare. J Am Med Inform Assoc 2024; 31:705-713. [PMID: 38031481 PMCID: PMC10873841 DOI: 10.1093/jamia/ocad221] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 10/06/2023] [Accepted: 11/03/2023] [Indexed: 12/01/2023] Open
Abstract
OBJECTIVE The complexity and rapid pace of development of algorithmic technologies pose challenges for their regulation and oversight in healthcare settings. We sought to improve our institution's approach to evaluation and governance of algorithmic technologies used in clinical care and operations by creating an Implementation Guide that standardizes evaluation criteria so that local oversight is performed in an objective fashion. MATERIALS AND METHODS Building on a framework that applies key ethical and quality principles (clinical value and safety, fairness and equity, usability and adoption, transparency and accountability, and regulatory compliance), we created concrete guidelines for evaluating algorithmic technologies at our institution. RESULTS An Implementation Guide articulates evaluation criteria used during review of algorithmic technologies and details what evidence supports the implementation of ethical and quality principles for trustworthy health AI. Application of the processes described in the Implementation Guide can lead to algorithms that are safer as well as more effective, fair, and equitable upon implementation, as illustrated through 4 examples of technologies at different phases of the algorithmic lifecycle that underwent evaluation at our academic medical center. DISCUSSION By providing clear descriptions/definitions of evaluation criteria and embedding them within standardized processes, we streamlined oversight processes and educated communities using and developing algorithmic technologies within our institution. CONCLUSIONS We developed a scalable, adaptable framework for translating principles into evaluation criteria and specific requirements that support trustworthy implementation of algorithmic technologies in patient care and healthcare operations.
Collapse
Affiliation(s)
| | - Sophia Bessias
- Duke AI Health, Duke University School of Medicine, Durham, NC 27705, United States
| | - Michael P Cary
- Duke AI Health, Duke University School of Medicine, Durham, NC 27705, United States
- Duke University School of Nursing, Durham, NC 27710, United States
| | - Armando D Bedoya
- Duke Health Technology Solutions, Duke University Health System, Durham, NC 27705, United States
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
| | - Benjamin A Goldstein
- Duke AI Health, Duke University School of Medicine, Durham, NC 27705, United States
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC 27705, United States
| | - John E Jelovsek
- Department of Obstetrics and Gynecology, Duke University School of Medicine, Durham, NC 27710, United States
| | - Cara L O’Brien
- Duke Health Technology Solutions, Duke University Health System, Durham, NC 27705, United States
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
| | - Nancy Walden
- Duke AI Health, Duke University School of Medicine, Durham, NC 27705, United States
| | - Matthew Elmore
- Duke AI Health, Duke University School of Medicine, Durham, NC 27705, United States
| | - Amanda B Parrish
- Office of Regulatory Affairs and Quality, Duke University School of Medicine, Durham, NC 27705, United States
| | - Scott Elengold
- Office of Counsel, Duke University, Durham, NC 27701, United States
| | - Kay S Lytle
- Duke University School of Nursing, Durham, NC 27710, United States
- Duke Health Technology Solutions, Duke University Health System, Durham, NC 27705, United States
| | - Suresh Balu
- Duke Institute for Health Innovation, Duke University, Durham, NC 27701, United States
| | - Michael E Lipkin
- Department of Urology, Duke University School of Medicine, Durham, NC 27710, United States
| | - Afreen Idris Shariff
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
- Duke Endocrine-Oncology Program, Duke University Health System, Durham, NC 27710, United States
| | - Michael Gao
- Duke Institute for Health Innovation, Duke University, Durham, NC 27701, United States
| | - David Leverenz
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
| | - Ricardo Henao
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC 27705, United States
- Department of Bioengineering, King Abdullah University of Science and Technology, Thuwal 23955, Saudi Arabia
| | - David Y Ming
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
- Duke Department of Pediatrics, Duke University Health System, Durham, NC 27705, United States
- Department of Population Health Sciences, Duke University School of Medicine, Durham, NC 27701, United States
| | - David M Gallagher
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
| | - Michael J Pencina
- Duke AI Health, Duke University School of Medicine, Durham, NC 27705, United States
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC 27705, United States
| | - Eric G Poon
- Duke Health Technology Solutions, Duke University Health System, Durham, NC 27705, United States
- Department of Medicine, Duke University School of Medicine, Durham, NC 27710, United States
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Durham, NC 27705, United States
| |
Collapse
|
5
|
Shah NH, Halamka JD, Saria S, Pencina M, Tazbaz T, Tripathi M, Callahan A, Hildahl H, Anderson B. A Nationwide Network of Health AI Assurance Laboratories. JAMA 2024; 331:245-249. [PMID: 38117493 DOI: 10.1001/jama.2023.26930] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/21/2023]
Abstract
Importance Given the importance of rigorous development and evaluation standards needed of artificial intelligence (AI) models used in health care, nationwide accepted procedures to provide assurance that the use of AI is fair, appropriate, valid, effective, and safe are urgently needed. Observations While there are several efforts to develop standards and best practices to evaluate AI, there is a gap between having such guidance and the application of such guidance to both existing and new AI models being developed. As of now, there is no publicly available, nationwide mechanism that enables objective evaluation and ongoing assessment of the consequences of using health AI models in clinical care settings. Conclusion and Relevance The need to create a public-private partnership to support a nationwide health AI assurance labs network is outlined here. In this network, community best practices could be applied for testing health AI models to produce reports on their performance that can be widely shared for managing the lifecycle of AI models over time and across populations and sites where these models are deployed.
Collapse
Affiliation(s)
- Nigam H Shah
- Stanford Medicine, Palo Alto, California
- Coalition for Health AI, Dover, Delaware
| | - John D Halamka
- Coalition for Health AI, Dover, Delaware
- Mayo Clinic Platform, Mayo Clinic, Rochester, Minnesota
| | - Suchi Saria
- Coalition for Health AI, Dover, Delaware
- Bayesian Health, New York, New York
- Johns Hopkins University, Baltimore, Maryland
- Johns Hopkins Medicine, Baltimore, Maryland
| | - Michael Pencina
- Coalition for Health AI, Dover, Delaware
- Duke AI Health, Duke University School of Medicine, Durham, North Carolina
| | - Troy Tazbaz
- US Food and Drug Administration, Silver Spring, Maryland
| | - Micky Tripathi
- US Office of the National Coordinator for Health IT, Washington, DC
| | | | | | - Brian Anderson
- Coalition for Health AI, Dover, Delaware
- MITRE Corporation, Bedford, Massachusetts
| |
Collapse
|
6
|
Pendo E, Oliva J. Challenging Disability Discrimination in the Clinical Use of PDMP Algorithms. Hastings Cent Rep 2024; 54:3-7. [PMID: 38390676 DOI: 10.1002/hast.1561] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/24/2024]
Abstract
State prescription drug monitoring programs (PDMPs) use proprietary, predictive software platforms that deploy algorithms to determine whether a patient is at risk for drug misuse, drug diversion, doctor shopping, or substance use disorder (SUD). Clinical overreliance on PDMP algorithm-generated information and risk scores motivates clinicians to refuse to treat-or to inappropriately treat-vulnerable people based on actual, perceived, or past SUDs, chronic pain conditions, or other disabilities. This essay provides a framework for challenging PDMP algorithmic discrimination as disability discrimination under federal antidiscrimination laws, including a new proposed rule interpreting section 1557 of the Affordable Care Act.
Collapse
|
7
|
Chin MH, Afsar-Manesh N, Bierman AS, Chang C, Colón-Rodríguez CJ, Dullabh P, Duran DG, Fair M, Hernandez-Boussard T, Hightower M, Jain A, Jordan WB, Konya S, Moore RH, Moore TT, Rodriguez R, Shaheen G, Snyder LP, Srinivasan M, Umscheid CA, Ohno-Machado L. Guiding Principles to Address the Impact of Algorithm Bias on Racial and Ethnic Disparities in Health and Health Care. JAMA Netw Open 2023; 6:e2345050. [PMID: 38100101 DOI: 10.1001/jamanetworkopen.2023.45050] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/18/2023] Open
Abstract
Importance Health care algorithms are used for diagnosis, treatment, prognosis, risk stratification, and allocation of resources. Bias in the development and use of algorithms can lead to worse outcomes for racial and ethnic minoritized groups and other historically marginalized populations such as individuals with lower income. Objective To provide a conceptual framework and guiding principles for mitigating and preventing bias in health care algorithms to promote health and health care equity. Evidence Review The Agency for Healthcare Research and Quality and the National Institute for Minority Health and Health Disparities convened a diverse panel of experts to review evidence, hear from stakeholders, and receive community feedback. Findings The panel developed a conceptual framework to apply guiding principles across an algorithm's life cycle, centering health and health care equity for patients and communities as the goal, within the wider context of structural racism and discrimination. Multiple stakeholders can mitigate and prevent bias at each phase of the algorithm life cycle, including problem formulation (phase 1); data selection, assessment, and management (phase 2); algorithm development, training, and validation (phase 3); deployment and integration of algorithms in intended settings (phase 4); and algorithm monitoring, maintenance, updating, or deimplementation (phase 5). Five principles should guide these efforts: (1) promote health and health care equity during all phases of the health care algorithm life cycle; (2) ensure health care algorithms and their use are transparent and explainable; (3) authentically engage patients and communities during all phases of the health care algorithm life cycle and earn trustworthiness; (4) explicitly identify health care algorithmic fairness issues and trade-offs; and (5) establish accountability for equity and fairness in outcomes from health care algorithms. Conclusions and Relevance Multiple stakeholders must partner to create systems, processes, regulations, incentives, standards, and policies to mitigate and prevent algorithmic bias. Reforms should implement guiding principles that support promotion of health and health care equity in all phases of the algorithm life cycle as well as transparency and explainability, authentic community engagement and ethical partnerships, explicit identification of fairness issues and trade-offs, and accountability for equity and fairness.
Collapse
Affiliation(s)
| | | | | | - Christine Chang
- Agency for Healthcare Research and Quality, Rockville, Maryland
| | | | | | | | - Malika Fair
- Association of American Medical Colleges, Washington, DC
| | | | | | - Anjali Jain
- Agency for Healthcare Research and Quality, Rockville, Maryland
| | | | - Stephen Konya
- Office of the National Coordinator for Health Information Technology, Washington, DC
| | - Roslyn Holliday Moore
- US Department of Health and Human Services Office of Minority Health, Rockville, Maryland
| | | | | | | | | | | | | | | |
Collapse
|