Deconstructing the diagnostic reasoning of human versus artificial intelligence

Artificial intelligence in diagnosing upper limb musculoskeletal disorders: a systematic review and meta-analysis of diagnostic tests

Purpose

The integration of artificial intelligence (AI) in radiology has revolutionized diagnostics, optimizing precision and decision-making. Specifically in musculoskeletal imaging, AI tools can improve accuracy for upper extremity pathologies. This study aimed to assess the diagnostic performance of AI models in detecting musculoskeletal pathologies of the upper extremity using different imaging modalities.

Methods

A meta-analysis was conducted, involving searches on MEDLINE/PubMed, SCOPUS, Cochrane Library, Lilacs, and SciELO. The quality of the studies was assessed using the QUADAS-2 tool. Diagnostic accuracy measures including sensitivity, specificity, diagnostic odds ratio (DOR), positive and negative likelihood ratios (PLR, NLR), area under the curve (AUC), and summary receiver operating characteristic were pooled using a random-effects model. Heterogeneity and subgroup analyses were also included. All statistical analyses and plots were performed using the R software package.

Results

Thirteen models from ten articles were analyzed. The sensitivity and specificity of the AI models to detect musculoskeletal conditions in the upper extremity were 0.926 (95% CI: 0.900; 0.945) and 0.908 (95% CI: 0.810; 0.958). The PLR, NLR, lnDOR, and the AUC estimates were found to be 19.18 (95% CI: 8.90; 29.34), 0.11 (95% CI: 0.18; 0.46), 4.62 (95% CI: 4.02; 5.22) with a (P < 0.001), and 95%, respectively.

Conclusion

The AI models exhibited strong univariate and bivariate performance in detecting both positive and negative cases within the analyzed dataset of musculoskeletal pathologies in the upper extremity.

Transformation and Articulation of Clinical Data to Understand Students' and Health Professionals' Clinical Reasoning: Protocol for a Scoping Review

BACKGROUND

There are still unanswered questions regarding effective educational strategies to promote the transformation and articulation of clinical data while teaching and learning clinical reasoning. Additionally, understanding how this process can be analyzed and assessed is crucial, particularly considering the rapid growth of natural language processing in artificial intelligence.

OBJECTIVE

The aim of this study is to map educational strategies to promote the transformation and articulation of clinical data among students and health care professionals and to explore the methods used to assess these individuals' transformation and articulation of clinical data.

METHODS

This scoping review follows the Joanna Briggs Institute framework for scoping reviews and the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) checklist for the analysis. A literature search was performed in November 2022 using 5 databases: CINAHL (EBSCOhost), MEDLINE (Ovid), Embase (Ovid), PsycINFO (Ovid), and Web of Science (Clarivate). The protocol was registered on the Open Science Framework in November 2023. The scoping review will follow the 9-step framework proposed by Peters and colleagues of the Joanna Briggs Institute. A data extraction form has been developed using key themes from the research questions.

RESULTS

After removing duplicates, the initial search yielded 6656 results, and study selection is underway. The extracted data will be qualitatively analyzed and presented in a diagrammatic or tabular form alongside a narrative summary. The review will be completed by February 2024.

CONCLUSIONS

By synthesizing the evidence on semantic transformation and articulation of clinical data during clinical reasoning education, this review aims to contribute to the refinement of educational strategies and assessment methods used in academic and continuing education programs. The insights gained from this review will help educators develop more effective semantic approaches for teaching or learning clinical reasoning, as opposed to fragmented, purely symptom-based or probabilistic approaches. Besides, the results may suggest some ways to address challenges related to the assessment of clinical reasoning and ensure that the assessment tasks accurately reflect learners' developing competencies and educational progress.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/50797.

Diagnostic accuracy of an artificial intelligence online engine in migraine: A multi‐center study

Objective

This study assesses the concordance in migraine diagnosis between an online, self‐administered, Computer‐based, Diagnostic Engine (CDE) and semi‐structured interview (SSI) by a headache specialist, both using International Classification of Headache Disorders, 3rd edition (ICHD‐3) criteria.

Background

Delay in accurate diagnosis is a major barrier to headache care. Accurate computer‐based algorithms may help reduce the need for SSI‐based encounters to arrive at correct ICHD‐3 diagnosis.

Methods

Between March 2018 and August 2019, adult participants were recruited from three academic headache centers and the community via advertising to our cross‐sectional study. Participants completed two evaluations: phone interview conducted by headache specialists using the SSI and a web‐based expert questionnaire and analytics, CDE. Participants were randomly assigned to either the SSI followed by the web‐based questionnaire or the web‐based questionnaire followed by the SSI. Participants completed protocols a few minutes apart. The concordance in migraine/probable migraine (M/PM) diagnosis between SSI and CDE was measured using Cohen’s kappa statistics. The diagnostic accuracy of CDE was assessed using the SSI as reference standard.

Results

Of the 276 participants consented, 212 completed both SSI and CDE (study completion rate = 77%; median age = 32 years [interquartile range: 28–40], female:male ratio = 3:1). Concordance in M/PM diagnosis between SSI and CDE was: κ = 0.83 (95% confidence interval [CI]: 0.75–0.91). CDE diagnostic accuracy: sensitivity = 90.1% (118/131), 95% CI: 83.6%–94.6%; specificity = 95.8% (68/71), 95% CI: 88.1%–99.1%. Positive and negative predictive values = 97.0% (95% CI: 91.3%–99.0%) and 86.6% (95% CI: 79.3%–91.5%), respectively, using identified migraine prevalence of 60%. Assuming a general migraine population prevalence of 10%, positive and negative predictive values were 70.3% (95% CI: 43.9%–87.8%) and 98.9% (95% CI: 98.1%–99.3%), respectively.

Conclusion

The SSI and CDE have excellent concordance in diagnosing M/PM. Positive CDE helps rule in M/PM, through high specificity and positive likelihood ratio. A negative CDE helps rule out M/PM through high sensitivity and low negative likelihood ratio. CDE that mimics SSI logic is a valid tool for migraine diagnosis.

The Validity of Machine Learning Procedures in Orthodontics: What Is Still Missing?

Artificial intelligence (AI) models and procedures hold remarkable predictive efficiency in the medical domain through their ability to discover hidden, non-obvious clinical patterns in data. However, due to the sparsity, noise, and time-dependency of medical data, AI procedures are raising unprecedented issues related to the mismatch between doctors’ mentalreasoning and the statistical answers provided by algorithms. Electronic systems can reproduce or even amplify noise hidden in the data, especially when the diagnosis of the subjects in the training data set is inaccurate or incomplete. In this paper we describe the conditions that need to be met for AI instruments to be truly useful in the orthodontic domain. We report some examples of computational procedures that are capable of extracting orthodontic knowledge through ever deeper patient representation. To have confidence in these procedures, orthodontic practitioners should recognize the benefits, shortcomings, and unintended consequences of AI models, as algorithms that learn from human decisions likewise learn mistakes and biases.

“I’m afraid I can’t let you do that, Doctor”: meaningful disagreements with AI in medical contexts

This paper explores the role and resolution of disagreements between physicians and their diagnostic AI-based decision support systems (DSS). With an ever-growing number of applications for these independently operating diagnostic tools, it becomes less and less clear what a physician ought to do in case their diagnosis is in faultless conflict with the results of the DSS. The consequences of such uncertainty can ultimately lead to effects detrimental to the intended purpose of such machines, e.g. by shifting the burden of proof towards a physician. Thus, we require normative clarity for integrating these machines without affecting established, trusted, and relied upon workflows. In reconstructing different causes of conflicts between physicians and their AI-based tools—inspired by the approach of “meaningful human control” over autonomous systems and the challenges to resolve them—we will delineate normative conditions for “meaningful disagreements”. These incorporate the potential of DSS to take on more tasks and outline how the moral responsibility of a physician can be preserved in an increasingly automated clinical work environment.

Role of Digital Health and Artificial Intelligence in Inflammatory Bowel Disease: A Scoping Review

Inflammatory bowel diseases (IBD), subdivided into Crohn's disease (CD) and ulcerative colitis (UC), are chronic diseases that are characterized by relapsing and remitting periods of inflammation in the gastrointestinal tract. In recent years, the amount of research surrounding digital health (DH) and artificial intelligence (AI) has increased. The purpose of this scoping review is to explore this growing field of research to summarize the role of DH and AI in the diagnosis, treatment, monitoring and prognosis of IBD. A review of 21 articles revealed the impact of both AI algorithms and DH technologies; AI algorithms can improve diagnostic accuracy, assess disease activity, and predict treatment response based on data modalities such as endoscopic imaging and genetic data. In terms of DH, patients utilizing DH platforms experienced improvements in quality of life, disease literacy, treatment adherence, and medication management. In addition, DH methods can reduce the need for in-person appointments, decreasing the use of healthcare resources without compromising the standard of care. These articles demonstrate preliminary evidence of the potential of DH and AI for improving the management of IBD. However, the majority of these studies were performed in a regulated clinical environment. Therefore, further validation of these results in a real-world environment is required to assess the efficacy of these methods in the general IBD population.

Lupus or not? SLE Risk Probability Index (SLERPI): a simple, clinician-friendly machine learning-based model to assist the diagnosis of systemic lupus erythematosus

Objectives

Diagnostic reasoning in systemic lupus erythematosus (SLE) is a complex process reflecting the probability of disease at a given timepoint against competing diagnoses. We applied machine learning in well-characterised patient data sets to develop an algorithm that can aid SLE diagnosis.

Methods

From a discovery cohort of randomly selected 802 adults with SLE or control rheumatologic diseases, clinically selected panels of deconvoluted classification criteria and non-criteria features were analysed. Feature selection and model construction were done with Random Forests and Least Absolute Shrinkage and Selection Operator-logistic regression (LASSO-LR). The best model in 10-fold cross-validation was tested in a validation cohort (512 SLE, 143 disease controls).

Results

A novel LASSO-LR model had the best performance and included 14 variably weighed features with thrombocytopenia/haemolytic anaemia, malar/maculopapular rash, proteinuria, low C3 and C4, antinuclear antibodies (ANA) and immunologic disorder being the strongest SLE predictors. Our model produced SLE risk probabilities (depending on the combination of features) correlating positively with disease severity and organ damage, and allowing the unbiased classification of a validation cohort into diagnostic certainty levels (unlikely, possible, likely, definitive SLE) based on the likelihood of SLE against other diagnoses. Operating the model as binary (lupus/not-lupus), we noted excellent accuracy (94.8%) for identifying SLE, and high sensitivity for early disease (93.8%), nephritis (97.9%), neuropsychiatric (91.8%) and severe lupus requiring immunosuppressives/biologics (96.4%). This was converted into a scoring system, whereby a score >7 has 94.2% accuracy.

Conclusions

We have developed and validated an accurate, clinician-friendly algorithm based on classical disease features for early SLE diagnosis and treatment to improve patient outcomes.

Artificial Intelligence/Machine Learning in Respiratory Medicine and Potential Role in Asthma and COPD Diagnosis

Artificial intelligence (AI) and machine learning, a subset of AI, are increasingly used in medicine. AI excels at performing well-defined tasks, such as image recognition; for example, classifying skin biopsy lesions, determining diabetic retinopathy severity, and detecting brain tumors. This article provides an overview of the use of AI in medicine and particularly in respiratory medicine, where it is used to evaluate lung cancer images, diagnose fibrotic lung disease, and more recently is being developed to aid the interpretation of pulmonary function tests and the diagnosis of a range of obstructive and restrictive lung diseases. The development and validation of AI algorithms requires large volumes of well-structured data, and the algorithms must work with variable levels of data quality. It is important that clinicians understand how AI can function in the context of heterogeneous conditions such as asthma and chronic obstructive pulmonary disease where diagnostic criteria overlap, how AI use fits into everyday clinical practice, and how issues of patient safety should be addressed. AI has a clear role in providing support for doctors in the clinical workplace, but its relatively recent introduction means that confidence in its use still has to be fully established. Overall, AI is expected to play a key role in aiding clinicians in the diagnosis and management of respiratory diseases in the future, and it will be exciting to see the benefits that arise for patients and doctors from its use in everyday clinical practice.

Lupus or not? SLE Risk Probability Index (SLERPI): a simple, clinician-friendly machine learning-based model to assist the diagnosis of systemic lupus erythematosus

OBJECTIVES

Diagnostic reasoning in systemic lupus erythematosus (SLE) is a complex process reflecting the probability of disease at a given timepoint against competing diagnoses. We applied machine learning in well-characterised patient data sets to develop an algorithm that can aid SLE diagnosis.

METHODS

From a discovery cohort of randomly selected 802 adults with SLE or control rheumatologic diseases, clinically selected panels of deconvoluted classification criteria and non-criteria features were analysed. Feature selection and model construction were done with Random Forests and Least Absolute Shrinkage and Selection Operator-logistic regression (LASSO-LR). The best model in 10-fold cross-validation was tested in a validation cohort (512 SLE, 143 disease controls).

RESULTS

A novel LASSO-LR model had the best performance and included 14 variably weighed features with thrombocytopenia/haemolytic anaemia, malar/maculopapular rash, proteinuria, low C3 and C4, antinuclear antibodies (ANA) and immunologic disorder being the strongest SLE predictors. Our model produced SLE risk probabilities (depending on the combination of features) correlating positively with disease severity and organ damage, and allowing the unbiased classification of a validation cohort into diagnostic certainty levels (unlikely, possible, likely, definitive SLE) based on the likelihood of SLE against other diagnoses. Operating the model as binary (lupus/not-lupus), we noted excellent accuracy (94.8%) for identifying SLE, and high sensitivity for early disease (93.8%), nephritis (97.9%), neuropsychiatric (91.8%) and severe lupus requiring immunosuppressives/biologics (96.4%). This was converted into a scoring system, whereby a score >7 has 94.2% accuracy.

CONCLUSIONS

We have developed and validated an accurate, clinician-friendly algorithm based on classical disease features for early SLE diagnosis and treatment to improve patient outcomes.

Natural Language Processing-Based Virtual Cofacilitator for Online Cancer Support Groups: Protocol for an Algorithm Development and Validation Study

Background

Cancer and its treatment can significantly impact the short- and long-term psychological well-being of patients and families. Emotional distress and depressive symptomatology are often associated with poor treatment adherence, reduced quality of life, and higher mortality. Cancer support groups, especially those led by health care professionals, provide a safe place for participants to discuss fear, normalize stress reactions, share solidarity, and learn about effective strategies to build resilience and enhance coping. However, in-person support groups may not always be accessible to individuals; geographic distance is one of the barriers for access, and compromised physical condition (eg, fatigue, pain) is another. Emerging evidence supports the effectiveness of online support groups in reducing access barriers. Text-based and professional-led online support groups have been offered by Cancer Chat Canada. Participants join the group discussion using text in real time. However, therapist leaders report some challenges leading text-based online support groups in the absence of visual cues, particularly in tracking participant distress. With multiple participants typing at the same time, the nuances of the text messages or red flags for distress can sometimes be missed. Recent advances in artificial intelligence such as deep learning–based natural language processing offer potential solutions. This technology can be used to analyze online support group text data to track participants’ expressed emotional distress, including fear, sadness, and hopelessness. Artificial intelligence allows session activities to be monitored in real time and alerts the therapist to participant disengagement.

Objective

We aim to develop and evaluate an artificial intelligence–based cofacilitator prototype to track and monitor online support group participants’ distress through real-time analysis of text-based messages posted during synchronous sessions.

Methods

An artificial intelligence–based cofacilitator will be developed to identify participants who are at-risk for increased emotional distress and track participant engagement and in-session group cohesion levels, providing real-time alerts for therapist to follow-up; generate postsession participant profiles that contain discussion content keywords and emotion profiles for each session; and automatically suggest tailored resources to participants according to their needs. The study is designed to be conducted in 4 phases consisting of (1) development based on a subset of data and an existing natural language processing framework, (2) performance evaluation using human scoring, (3) beta testing, and (4) user experience evaluation.

Results

This study received ethics approval in August 2019. Phase 1, development of an artificial intelligence–based cofacilitator, was completed in January 2020. As of December 2020, phase 2 is underway. The study is expected to be completed by September 2021.

Conclusions

An artificial intelligence–based cofacilitator offers a promising new mode of delivery of person-centered online support groups tailored to individual needs.

International Registered Report Identifier (IRRID)

DERR1-10.2196/21453

Relationships Between Context, Process, and Outcome Indicators to Assess Quality of Physiotherapy Care in Patients with Whiplash-Associated Disorders: Applying Donabedian's Model of Care

BACKGROUND

Quality indicators (QIs) are measurable elements of practice performance and may relate to context, process, outcome and structure. A valid set of QIs have been developed, reflecting the clinical reasoning used in primary care physiotherapy for patients with whiplash-associated disorders (WAD). Donabedian's model postulates relationships between the constructs of quality of care, acting in a virtuous circle.

AIM

To explore the relative strengths of the relationships between context, process, and outcome indicators in the assessment of primary care physiotherapy in patients with WAD.

MATERIALS AND METHODS

Data on WAD patients (N=810) were collected over a period of 16 years in primary care physiotherapy practices by means of patients records. This routinely collected dataset (RCD-WAD) was classified in context, process, and outcome variables and analyzed retrospectively. Clinically relevant variables were selected based on expert consensus. Associations were expressed, using zero-order, as Spearman rank correlation coefficients (criterion: rs >0.25 [minimum: fair]; α-value = 0.05).

RESULTS

In round 1, 62 of 85 (72.9%) variables were selected by an expert panel as relevant for clinical reasoning; in round 2, 34 of 62 (54.8%) (context variables 9 of 18 [50.0%]; process variables 18 of 34 [52.9]; outcome variables 8 of 10 [90.0%]) as highly relevant. Associations between the selected context and process variables ranged from 0.27 to 0.53 (p≤0.00), between selected context and outcome variables from 0.26 to 0.55 (p≤0.00), and between selected process and outcome variables from 0.29 to 0.59 (p≤0.00). Moderate associations (rs >0.50; p≤0.00) were found between "pain coping" and "fear avoidance" as process variables, and "pain intensity" and "functioning" as outcome variables.

CONCLUSION

The identified associations between selected context, process, and outcome variables were fair to moderate. Ongoing work may clarify some of these associations and provide guidance to physiotherapists on how best to improve the quality of clinical reasoning in terms of relationships between context, process, and outcome in the management of patients with WAD.

Mechanizing medicine - Tomorrows history started yesterday

History is by nature a retrospective subject, there usually being an interval between any event, a review or impact of the subject being considered. This NPSA Historian's paper, takes a long and quick historical view of influences that fostered changes resulting in the current state of affairs in the field of medicine and medical care. The fields of medicine and surgery, are undergoing rapid changes as a result of technological and other advances that are making tomorrow's medical history seemingly happening yesterday. Prospectively, the impact of current change and its rapidity has the potential to radically change the future practice of the art and craft of our profession.