Discrepancies in Stroke Distribution and Dataset Origin in Machine Learning for Stroke

Estimating individualized effectiveness of receiving successful recanalization for ischemic stroke cases using machine learning techniques

OBJECTIVES

Directly measuring the causal effect of mechanical thrombectomy (MT) for each ischemic stroke patient remains challenging, as it is impossible to observe the outcomes for both with and without successful recanalization in the same individual. In this study, we aimed to use machine learning to identify characteristics influencing the likelihood of not benefiting from successful recanalization.

MATERIALS & METHODS

A total of 1718 non-reperfused patients (Thrombolysis in Cerebral Infarction [TICI] ≤ 2a) and 10339 reperfused patients (TICI ≥ 2b) were included in the study as nonreperfusion and reperfusion groups, respectively. The primary target variable was probability of poor functional outcome after three months, defined by the modified Rankin Scale score of 3 to 6. Two random forest (RF) models trained on pre-treatment covariates of nonreperfusion and reperfusion groups, were used to predict the probability of poor outcome under unsuccessful and successful recanalization scenarios, respectively. The individual effect of successful recanalization was defined as the difference in predicted probabilities returned by the two models.

RESULTS

Strong calibration was achieved by the RF models trained on nonreperfusion group (intercept:0.027, slope: 1.030) and reperfused group (intercept:0.010, slope: 1.017). The average risk reduction under successful recanalization scenario was 22.0 % (95 % CI [21.7 % - 22.3 %]) for the reperfused group and 19.8 % (95 % CI [19.1 % - 20.5 %]) for the nonreperfusion group. Key factors associated with not benefiting from successful recanalization included older age, higher pre-stroke mRS scores and higher National Institutes of Health Stroke Scale score at admission.

CONCLUSIONS

This study highlights the potential of predictive ML techniques to estimate the individual effect of successful recanalization on ischemic stroke patients undergoing MT.

An intelligent learning system based on electronic health records for unbiased stroke prediction

Stroke has a negative impact on people's lives and is one of the leading causes of death and disability worldwide. Early detection of symptoms can significantly help predict stroke and promote a healthy lifestyle. Researchers have developed several methods to predict strokes using machine learning (ML) techniques. However, the proposed systems have suffered from the following two main problems. The first problem is that the machine learning models are biased due to the uneven distribution of classes in the dataset. Recent research has not adequately addressed this problem, and no preventive measures have been taken. Synthetic Minority Oversampling (SMOTE) has been used to remove bias and balance the training of the proposed ML model. The second problem is to solve the problem of lower classification accuracy of machine learning models. We proposed a learning system that combines an autoencoder with a linear discriminant analysis (LDA) model to increase the accuracy of the proposed ML model for stroke prediction. Relevant features are extracted from the feature space using the autoencoder, and the extracted subset is then fed into the LDA model for stroke classification. The hyperparameters of the LDA model are found using a grid search strategy. However, the conventional accuracy metric does not truly reflect the performance of ML models. Therefore, we employed several evaluation metrics to validate the efficiency of the proposed model. Consequently, we evaluated the proposed model's accuracy, sensitivity, specificity, area under the curve (AUC), and receiver operator characteristic (ROC). The experimental results show that the proposed model achieves a sensitivity and specificity of 98.51% and 97.56%, respectively, with an accuracy of 99.24% and a balanced accuracy of 98.00%.

Challenges in Reducing Bias Using Post-Processing Fairness for Breast Cancer Stage Classification with Deep Learning

Breast cancer is the most common cancer affecting women globally. Despite the significant impact of deep learning models on breast cancer diagnosis and treatment, achieving fairness or equitable outcomes across diverse populations remains a challenge when some demographic groups are underrepresented in the training data. We quantified the bias of models trained to predict breast cancer stage from a dataset consisting of 1000 biopsies from 842 patients provided by AIM-Ahead (Artificial Intelligence/Machine Learning Consortium to Advance Health Equity and Researcher Diversity). Notably, the majority of data (over 70%) were from White patients. We found that prior to post-processing adjustments, all deep learning models we trained consistently performed better for White patients than for non-White patients. After model calibration, we observed mixed results, with only some models demonstrating improved performance. This work provides a case study of bias in breast cancer medical imaging models and highlights the challenges in using post-processing to attempt to achieve fairness.

Centering and collaborating with community knowledge systems: piloting a novel participatory modeling approach

BACKGROUND

Systems science approaches like simulation modeling can offer an opportunity for community voice to shape policies. In the episteme of many communities there are elders, leaders, and researchers who are seen as bearers of historic knowledge and can contextualize and interpret contemporary research using knowledge systems of the community. There is a need for a systematic methodology to collaborate with community Knowledge Bearers and Knowledge Interpreters. In this paper we report the results of piloting a systematic methodology for collaborating with a community Knowledge-Bearer and Knowledge-Interpreter to develop a conceptual model revealing the local-level influences and architecture of systems shaping community realities. The use case for this pilot is 'persistent poverty' in the United States, specifically within the inner-city African American community in Baltimore City.

METHODS

This pilot of a participatory modeling approach was conducted over a span of 7 sessions and included the following steps, each with an associated script: Step 1: Knowledge-Bearer and Knowledge-Interpreter recruitment Step 2: Relationship building Step 3: Session introduction, Vignette development & enrichment Step 4: Vignette analysis & constructing architecture of systems map Step 5: Augmenting architecture of systems map RESULTS: Each step of the participatory modeling approach resulted in artifacts that were valuable for both the communities and the research effort. Vignette construction resulted in narratives representing a spectrum of lived experiences, trajectories, and outcomes within a community. The collaborative analysis of vignettes yielded the Architecture of Systemic Factors map, that revealed how factors inter-relate to form a system in which lived experience of poverty occurs. A literature search provided an opportunity for the community to contextualize existing research about them using realities of lived experience.

CONCLUSION

This methodology showed that a community Knowledge Bearer can function as communicators and interpreters of their community's knowledge base, can develop coherent narratives of lived experiences within which research and knowledge is contextualized, and can collaboratively construct conceptual mappings necessary for simulation modeling. This participatory modeling approach showed that even if there already exists a vast body of research about a community, collaborating with community gives context to that research and brings together disparate findings within narratives of lived experience.

Sources of bias in artificial intelligence that perpetuate healthcare disparities-A global review

BACKGROUND

While artificial intelligence (AI) offers possibilities of advanced clinical prediction and decision-making in healthcare, models trained on relatively homogeneous datasets, and populations poorly-representative of underlying diversity, limits generalisability and risks biased AI-based decisions. Here, we describe the landscape of AI in clinical medicine to delineate population and data-source disparities.

METHODS

We performed a scoping review of clinical papers published in PubMed in 2019 using AI techniques. We assessed differences in dataset country source, clinical specialty, and author nationality, sex, and expertise. A manually tagged subsample of PubMed articles was used to train a model, leveraging transfer-learning techniques (building upon an existing BioBERT model) to predict eligibility for inclusion (original, human, clinical AI literature). Of all eligible articles, database country source and clinical specialty were manually labelled. A BioBERT-based model predicted first/last author expertise. Author nationality was determined using corresponding affiliated institution information using Entrez Direct. And first/last author sex was evaluated using the Gendarize.io API.

RESULTS

Our search yielded 30,576 articles, of which 7,314 (23.9%) were eligible for further analysis. Most databases came from the US (40.8%) and China (13.7%). Radiology was the most represented clinical specialty (40.4%), followed by pathology (9.1%). Authors were primarily from either China (24.0%) or the US (18.4%). First and last authors were predominately data experts (i.e., statisticians) (59.6% and 53.9% respectively) rather than clinicians. And the majority of first/last authors were male (74.1%).

INTERPRETATION

U.S. and Chinese datasets and authors were disproportionately overrepresented in clinical AI, and almost all of the top 10 databases and author nationalities were from high income countries (HICs). AI techniques were most commonly employed for image-rich specialties, and authors were predominantly male, with non-clinical backgrounds. Development of technological infrastructure in data-poor regions, and diligence in external validation and model re-calibration prior to clinical implementation in the short-term, are crucial in ensuring clinical AI is meaningful for broader populations, and to avoid perpetuating global health inequity.