Towards nationally curated data archives for clinical radiology image analysis at scale: Learnings from national data collection in response to a pandemic

Learning From International Comparators of National Medical Imaging Initiatives for AI Development: Multiphase Qualitative Study

BACKGROUND

The COVID-19 pandemic drove investment and research into medical imaging platforms to provide data to create artificial intelligence (AI) algorithms for the management of patients with COVID-19. Building on the success of England's National COVID-19 Chest Imaging Database, the national digital policy body (NHSX) sought to create a generalized national medical imaging platform for the development, validation, and deployment of algorithms.

OBJECTIVE

This study aims to understand international use cases of medical imaging platforms for the development and implementation of algorithms to inform the creation of England's national imaging platform.

METHODS

The National Health Service (NHS) AI Lab Policy and Strategy Team adopted a multiphased approach: (1) identification and prioritization of national AI imaging platforms; (2) Political, Economic, Social, Technological, Legal, and Environmental (PESTLE) factor analysis deep dive into national AI imaging platforms; (3) semistructured interviews with key stakeholders; (4) workshop on emerging themes and insights with the internal NHSX team; and (5) formulation of policy recommendations.

RESULTS

International use cases of national AI imaging platforms (n=7) were prioritized for PESTLE factor analysis. Stakeholders (n=13) from the international use cases were interviewed. Themes (n=8) from the semistructured interviews, including interview quotes, were analyzed with workshop participants (n=5). The outputs of the deep dives, interviews, and workshop were synthesized thematically into 8 categories with 17 subcategories. On the basis of the insights from the international use cases, policy recommendations (n=12) were developed to support the NHS AI Lab in the design and development of the English national medical imaging platform.

CONCLUSIONS

The creation of AI algorithms supporting technology and infrastructure such as platforms often occurs in isolation within countries, let alone between countries. This novel policy research project sought to bridge the gap by learning from the challenges, successes, and experience of England's international counterparts. Policy recommendations based on international learnings focused on the demonstrable benefits of the platform to secure sustainable funding, validation of algorithms and infrastructure to support in situ deployment, and creating wraparound tools for nontechnical participants such as clinicians to engage with algorithm creation. As health care organizations increasingly adopt technological solutions, policy makers have a responsibility to ensure that initiatives are informed by learnings from both national and international initiatives as well as disseminating the outcomes of their work.

The impact of imputation quality on machine learning classifiers for datasets with missing values

BACKGROUND

Classifying samples in incomplete datasets is a common aim for machine learning practitioners, but is non-trivial. Missing data is found in most real-world datasets and these missing values are typically imputed using established methods, followed by classification of the now complete samples. The focus of the machine learning researcher is to optimise the classifier's performance.

METHODS

We utilise three simulated and three real-world clinical datasets with different feature types and missingness patterns. Initially, we evaluate how the downstream classifier performance depends on the choice of classifier and imputation methods. We employ ANOVA to quantitatively evaluate how the choice of missingness rate, imputation method, and classifier method influences the performance. Additionally, we compare commonly used methods for assessing imputation quality and introduce a class of discrepancy scores based on the sliced Wasserstein distance. We also assess the stability of the imputations and the interpretability of model built on the imputed data.

RESULTS

The performance of the classifier is most affected by the percentage of missingness in the test data, with a considerable performance decline observed as the test missingness rate increases. We also show that the commonly used measures for assessing imputation quality tend to lead to imputed data which poorly matches the underlying data distribution, whereas our new class of discrepancy scores performs much better on this measure. Furthermore, we show that the interpretability of classifier models trained using poorly imputed data is compromised.

CONCLUSIONS

It is imperative to consider the quality of the imputation when performing downstream classification as the effects on the classifier can be considerable.

Delineating COVID-19 subgroups using routine clinical data identifies distinct in-hospital outcomes

The COVID-19 pandemic has been a great challenge to healthcare systems worldwide. It highlighted the need for robust predictive models which can be readily deployed to uncover heterogeneities in disease course, aid decision-making and prioritise treatment. We adapted an unsupervised data-driven model-SuStaIn, to be utilised for short-term infectious disease like COVID-19, based on 11 commonly recorded clinical measures. We used 1344 patients from the National COVID-19 Chest Imaging Database (NCCID), hospitalised for RT-PCR confirmed COVID-19 disease, splitting them equally into a training and an independent validation cohort. We discovered three COVID-19 subtypes (General Haemodynamic, Renal and Immunological) and introduced disease severity stages, both of which were predictive of distinct risks of in-hospital mortality or escalation of treatment, when analysed using Cox Proportional Hazards models. A low-risk Normal-appearing subtype was also discovered. The model and our full pipeline are available online and can be adapted for future outbreaks of COVID-19 or other infectious disease.

The Artificial Intelligence in Digital Radiology: Part 1: The Challenges, Acceptance and Consensus

Artificial intelligence is having important developments in the world of digital radiology also thanks to the boost given to the research sector by the COVID-19 pandemic. In the last two years, there was an important development of studies focused on both challenges and acceptance and consensus in the field of Artificial Intelligence. The challenges and acceptance and consensus are two strategic aspects in the development and integration of technologies in the health domain. The study conducted two narrative reviews by means of two parallel points of view to take stock both on the ongoing challenges and on initiatives conducted to face the acceptance and consensus in this area. The methodology of the review was based on: (I) search of PubMed and Scopus and (II) an eligibility assessment, using parameters with 5 levels of score. The results have: (a) highlighted and categorized the important challenges in place. (b) Illustrated the different types of studies conducted through original questionnaires. The study suggests for future research based on questionnaires a better calibration and inclusion of the challenges in place together with validation and administration paths at an international level.

Artificial intelligence and the future of radiographic scoring in rheumatoid arthritis: a viewpoint

Rheumatoid arthritis is an autoimmune condition that predominantly affects the synovial joints, causing joint destruction, pain, and disability. Historically, the standard for measuring the long-term efficacy of disease-modifying antirheumatic drugs has been the assessment of plain radiographs with scoring techniques that quantify joint damage. However, with significant improvements in therapy, current radiographic scoring systems may no longer be fit for purpose for the milder spectrum of disease seen today. We argue that artificial intelligence is an apt solution to further improve upon radiographic scoring, as it can readily learn to recognize subtle patterns in imaging data to not only improve efficiency, but can also increase the sensitivity to variation in mild disease. Current work in the area demonstrates the feasibility of automating scoring but is yet to take full advantage of the strengths of artificial intelligence. By fully leveraging the power of artificial intelligence, faster and more sensitive scoring could enable the ongoing development of effective treatments for patients with rheumatoid arthritis.