Regulatory Aspects of the Use of Artificial Intelligence Medical Software

Applying and validating a quality management system for in-house developed medical software

Introduction

The legislation regarding in-house development of medical devices has changed substantially with the introduction of the Medical Device Regulation (MDR) in 2021. Practical guidelines regarding the implementation of a quality management system for in-house developed medical software are scarce. In this article, we describe our experience with fulfilling the requirements of the MDR for an in-house developed prediction model, qualified as medical software.

Methods and materials

Our quality management system (QMS) is based on the ISO13485:2016. It is a workflow consisting of elements subdivided in subelements, which consist of procedures, work instructions and/or formats. Within the data science team procedures regarding the process and documentation of software development were already in place. The existing procedures and documentation were compared with the procedures of the QMS and where possible, integrated into the workflow. The gap between the existing procedures regarding software development and the procedures of the QMS was defined. Existing documentation and procedures were used as much as possible. If there was a gap, additional documentation was written.

Results

The majority of the (sub)elements was considered to be applicable for our software development project beforehand. Only in 6 out of 32 cases (19%), the (sub)element was deemed not applicable. For 32% of the applicable elements the documentation of the data scientists team was sufficient and additional information was not needed. For 23% the documentation was incomplete and we decided to add relevant information to fulfil the requirements of the MDR and for 45% the documentation was completely lacking and the standard formats were used.

Conclusion

We showed in this article that it is possible to use a QMS developed with physical medical products in mind for medical software and thus comply with applicable legislation and regulations. This can be done without too much effort when there is already some structured form of software development methodology in place.

The translation of in-house imaging AI research into a medical device ensuring ethical and regulatory integrity

This manuscript delineates the pathway from in-house research on Artificial Intelligence (AI) to the development of a medical device, addressing critical phases including conceptualization, development, validation, and regulatory compliance. Key stages in the transformation process involve identifying clinical needs, data management, model training, and rigorous validation to ensure AI models are both robust and clinically relevant. Continuous post-deployment surveillance is essential to maintain performance and adapt to changes in clinical practice. The regulatory landscape is complex, encompassing stringent certification processes under the EU Medical Device Regulation (MDR) and the upcoming EU AI Act, which imposes additional compliance requirements aimed at mitigating AI-specific risks. Ethical considerations such as, emphasizing transparency, patient privacy, and equitable access to AI technologies, are paramount. The manuscript underscores the importance of interdisciplinary collaboration, between healthcare institutions and industry partners, and navigation of commercialization and market entry of AI devices. This overview provides a strategic framework for radiologists and healthcare leaders to effectively integrate AI into clinical practice, while adhering to regulatory and ethical standards, ultimately enhancing patient care and operational efficiency.

Models for the marrow: A comprehensive review of AI-based cell classification methods and malignancy detection in bone marrow aspirate smears

Given the high prevalence of artificial intelligence (AI) research in medicine, the development of deep learning (DL) algorithms based on image recognition, such as the analysis of bone marrow aspirate (BMA) smears, is rapidly increasing in the field of hematology and oncology. The models are trained to identify the optimal regions of the BMA smear for differential cell count and subsequently detect and classify a number of cell types, which can ultimately be utilized for diagnostic purposes. Moreover, AI is capable of identifying genetic mutations phenotypically. This pipeline has the potential to offer an accurate and rapid preliminary analysis of the bone marrow in the clinical routine. However, the intrinsic complexity of hematological diseases presents several challenges for the automatic morphological assessment. To ensure general applicability across multiple medical centers and to deliver high accuracy on prospective clinical data, AI models would require highly heterogeneous training datasets. This review presents a systematic analysis of models for cell classification and detection of hematological malignancies published in the last 5 years (2019-2024). It provides insight into the challenges and opportunities of these DL-assisted tasks.

The Evolution of Artificial Intelligence in Medical Imaging: From Computer Science to Machine and Deep Learning

Artificial intelligence (AI), the wide spectrum of technologies aiming to give machines or computers the ability to perform human-like cognitive functions, began in the 1940s with the first abstract models of intelligent machines. Soon after, in the 1950s and 1960s, machine learning algorithms such as neural networks and decision trees ignited significant enthusiasm. More recent advancements include the refinement of learning algorithms, the development of convolutional neural networks to efficiently analyze images, and methods to synthesize new images. This renewed enthusiasm was also due to the increase in computational power with graphical processing units and the availability of large digital databases to be mined by neural networks. AI soon began to be applied in medicine, first through expert systems designed to support the clinician's decision and later with neural networks for the detection, classification, or segmentation of malignant lesions in medical images. A recent prospective clinical trial demonstrated the non-inferiority of AI alone compared with a double reading by two radiologists on screening mammography. Natural language processing, recurrent neural networks, transformers, and generative models have both improved the capabilities of making an automated reading of medical images and moved AI to new domains, including the text analysis of electronic health records, image self-labeling, and self-reporting. The availability of open-source and free libraries, as well as powerful computing resources, has greatly facilitated the adoption of deep learning by researchers and clinicians. Key concerns surrounding AI in healthcare include the need for clinical trials to demonstrate efficacy, the perception of AI tools as 'black boxes' that require greater interpretability and explainability, and ethical issues related to ensuring fairness and trustworthiness in AI systems. Thanks to its versatility and impressive results, AI is one of the most promising resources for frontier research and applications in medicine, in particular for oncological applications.

ChatGPT's contributions to the evolution of neurosurgical practice and education: a systematic review of benefits, concerns and limitations

Aim This study provides a comprehensive review of the current literature on the use of ChatGPT, a generative Artificial Intelligence (AI) tool, in neurosurgery. The study examines potential benefits and limitations of ChatGPT in neurosurgical practice and education. Methods The study involved a systematic review of the current literature on the use of AI in neurosurgery, with a focus on ChatGPT. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed to ensure a comprehensive and transparent review process. Thirteen studies met the inclusion criteria and were included in the final analysis. The data extracted from the included studies were analysed and synthesized to provide an overview of the current state of research on the use of ChatGPT in neurosurgery. Results The ChatGPT showed a potential to complement and enhance neurosurgical practice. However, there are risks and limitations associated with its use, including question format limitations, validation challenges, and algorithmic bias. The study highlights the importance of validating machine-generated content for accuracy and addressing ethical concerns associated with AI technologies. The study also identifies potential benefits of ChatGPT, such as providing personalized treatment plans, supporting surgical planning and navigation, and enhancing large data processing efficiency and accuracy. Conclusion The integration of AI technologies into neurosurgery should be approached with caution and careful consideration of ethical and validation issues. Continued research and development of AI tools in neurosurgery can help us further understand their potential benefits and limitations.

Impact of artificial intelligence on prognosis, shared decision-making, and precision medicine for patients with inflammatory bowel disease: a perspective and expert opinion

INTRODUCTION

Artificial intelligence (AI) is expected to impact all facets of inflammatory bowel disease (IBD) management, including disease assessment, treatment decisions, discovery and development of new biomarkers and therapeutics, as well as clinician-patient communication.

AREAS COVERED

This perspective paper provides an overview of the application of AI in the clinical management of IBD through a review of the currently available AI models that could be potential tools for prognosis, shared decision-making, and precision medicine. This overview covers models that measure treatment response based on statistical or machine-learning methods, or a combination of the two. We briefly discuss a computational model that allows integration of immune/biological system knowledge with mathematical modeling and also involves a 'digital twin', which allows measurement of temporal trends in mucosal inflammatory activity for predicting treatment response. A viewpoint on AI-enabled wearables and nearables and their use to improve IBD management is also included.

EXPERT OPINION

Although challenges regarding data quality, privacy, and security; ethical concerns; technical limitations; and regulatory barriers remain to be fully addressed, a growing body of evidence suggests a tremendous potential for integration of AI into daily clinical practice to enable precision medicine and shared decision-making.

Clinical Use of a Commercial Artificial Intelligence-Based Software for Autocontouring in Radiation Therapy: Geometric Performance and Dosimetric Impact

PURPOSE

When autocontouring based on artificial intelligence (AI) is used in the radiotherapy (RT) workflow, the contours are reviewed and eventually adjusted by a radiation oncologist before an RT treatment plan is generated, with the purpose of improving dosimetry and reducing both interobserver variability and time for contouring. The purpose of this study was to evaluate the results of application of a commercial AI-based autocontouring for RT, assessing both geometric accuracies and the influence on optimized dose from automatically generated contours after review by human operator.

MATERIALS AND METHODS

A commercial autocontouring system was applied to a retrospective database of 40 patients, of which 20 were treated with radiotherapy for prostate cancer (PCa) and 20 for head and neck cancer (HNC). Contours resulting from AI were compared against AI contours reviewed by human operator and human-only contours using Dice similarity coefficient (DSC), Hausdorff distance (HD), and relative volume difference (RVD). Dosimetric indices such as Dmean, D0.03cc, and normalized plan quality metrics were used to compare dose distributions from RT plans generated from structure sets contoured by humans assisted by AI against plans from manual contours. The reduction in contouring time obtained by using automated tools was also assessed. A Wilcoxon rank sum test was computed to assess the significance of differences. Interobserver variability of the comparison of manual vs. AI-assisted contours was also assessed among two radiation oncologists for PCa.

RESULTS

For PCa, AI-assisted segmentation showed good agreement with expert radiation oncologist structures with average DSC among patients ≥ 0.7 for all structures, and minimal radiation oncology adjustment of structures (DSC of adjusted versus AI structures ≥ 0.91). For HNC, results of comparison between manual and AI contouring varied considerably e.g., 0.77 for oral cavity and 0.11-0.13 for brachial plexus, but again, adjustment was generally minimal (DSC of adjusted against AI contours 0.97 for oral cavity, 0.92-0.93 for brachial plexus). The difference in dose for the target and organs at risk were not statistically significant between human and AI-assisted, with the only exceptions of D0.03cc to the anal canal and Dmean to the brachial plexus. The observed average differences in plan quality for PCa and HNC cases were 8% and 6.7%, respectively. The dose parameter changes due to interobserver variability in PCa were small, with the exception of the anal canal, where large dose variations were observed. The reduction in time required for contouring was 72% for PCa and 84% for HNC.

CONCLUSIONS

When an autocontouring system is used in combination with human review, the time of the RT workflow is significantly reduced without affecting dose distribution and plan quality.

Challenges and Ethical Considerations to Successfully Implement Artificial Intelligence in Clinical Medicine and Neuroscience: a Narrative Review

This narrative review discusses how the safe and effective use of clinical artificial intelligence (AI) prediction tools requires recognition of the importance of human intelligence. Human intelligence, creativity, situational awareness, and professional knowledge, are required for successful implementation. The implementation of clinical AI prediction tools may change the workflow in medical practice resulting in new challenges and safety implications. Human understanding of how a clinical AI prediction tool performs in routine and exceptional situations is fundamental to successful implementation. Physicians must be involved in all aspects of the selection, implementation, and ongoing product monitoring of clinical AI prediction tools.

Ethical Considerations of Using ChatGPT in Health Care

ChatGPT has promising applications in health care, but potential ethical issues need to be addressed proactively to prevent harm. ChatGPT presents potential ethical challenges from legal, humanistic, algorithmic, and informational perspectives. Legal ethics concerns arise from the unclear allocation of responsibility when patient harm occurs and from potential breaches of patient privacy due to data collection. Clear rules and legal boundaries are needed to properly allocate liability and protect users. Humanistic ethics concerns arise from the potential disruption of the physician-patient relationship, humanistic care, and issues of integrity. Overreliance on artificial intelligence (AI) can undermine compassion and erode trust. Transparency and disclosure of AI-generated content are critical to maintaining integrity. Algorithmic ethics raise concerns about algorithmic bias, responsibility, transparency and explainability, as well as validation and evaluation. Information ethics include data bias, validity, and effectiveness. Biased training data can lead to biased output, and overreliance on ChatGPT can reduce patient adherence and encourage self-diagnosis. Ensuring the accuracy, reliability, and validity of ChatGPT-generated content requires rigorous validation and ongoing updates based on clinical practice. To navigate the evolving ethical landscape of AI, AI in health care must adhere to the strictest ethical standards. Through comprehensive ethical guidelines, health care professionals can ensure the responsible use of ChatGPT, promote accurate and reliable information exchange, protect patient privacy, and empower patients to make informed decisions about their health care.

Artificial intelligence for clinical decision support for monitoring patients in cardiovascular ICUs: A systematic review

Background

Artificial intelligence (AI) and machine learning (ML) models continue to evolve the clinical decision support systems (CDSS). However, challenges arise when it comes to the integration of AI/ML into clinical scenarios. In this systematic review, we followed the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA), the population, intervention, comparator, outcome, and study design (PICOS), and the medical AI life cycle guidelines to investigate studies and tools which address AI/ML-based approaches towards clinical decision support (CDS) for monitoring cardiovascular patients in intensive care units (ICUs). We further discuss recent advances, pitfalls, and future perspectives towards effective integration of AI into routine practices as were identified and elaborated over an extensive selection process for state-of-the-art manuscripts.

Methods

Studies with available English full text from PubMed and Google Scholar in the period from January 2018 to August 2022 were considered. The manuscripts were fetched through a combination of the search keywords including AI, ML, reinforcement learning (RL), deep learning, clinical decision support, and cardiovascular critical care and patients monitoring. The manuscripts were analyzed and filtered based on qualitative and quantitative criteria such as target population, proper study design, cross-validation, and risk of bias.

Results

More than 100 queries over two medical search engines and subjective literature research were developed which identified 89 studies. After extensive assessments of the studies both technically and medically, 21 studies were selected for the final qualitative assessment.

Discussion

Clinical time series and electronic health records (EHR) data were the most common input modalities, while methods such as gradient boosting, recurrent neural networks (RNNs) and RL were mostly used for the analysis. Seventy-five percent of the selected papers lacked validation against external datasets highlighting the generalizability issue. Also, interpretability of the AI decisions was identified as a central issue towards effective integration of AI in healthcare.

How Radiomics Can Improve Breast Cancer Diagnosis and Treatment

Recent technological advances in the field of artificial intelligence hold promise in addressing medical challenges in breast cancer care, such as early diagnosis, cancer subtype determination and molecular profiling, prediction of lymph node metastases, and prognostication of treatment response and probability of recurrence. Radiomics is a quantitative approach to medical imaging, which aims to enhance the existing data available to clinicians by means of advanced mathematical analysis using artificial intelligence. Various published studies from different fields in imaging have highlighted the potential of radiomics to enhance clinical decision making. In this review, we describe the evolution of AI in breast imaging and its frontiers, focusing on handcrafted and deep learning radiomics. We present a typical workflow of a radiomics analysis and a practical "how-to" guide. Finally, we summarize the methodology and implementation of radiomics in breast cancer, based on the most recent scientific literature to help researchers and clinicians gain fundamental knowledge of this emerging technology. Alongside this, we discuss the current limitations of radiomics and challenges of integration into clinical practice with conceptual consistency, data curation, technical reproducibility, adequate accuracy, and clinical translation. The incorporation of radiomics with clinical, histopathological, and genomic information will enable physicians to move forward to a higher level of personalized management of patients with breast cancer.

The Use of Artificial Intelligence (AI) in the Radiology Field: What Is the State of Doctor-Patient Communication in Cancer Diagnosis?

BACKGROUND

In the past decade, interest in applying Artificial Intelligence (AI) in radiology to improve diagnostic procedures increased. AI has potential benefits spanning all steps of the imaging chain, from the prescription of diagnostic tests to the communication of test reports. The use of AI in the field of radiology also poses challenges in doctor-patient communication at the time of the diagnosis. This systematic review focuses on the patient role and the interpersonal skills between patients and physicians when AI is implemented in cancer diagnosis communication.

METHODS

A systematic search was conducted on PubMed, Embase, Medline, Scopus, and PsycNet from 1990 to 2021. The search terms were: ("artificial intelligence" or "intelligence machine") and "communication" "radiology" and "oncology diagnosis". The PRISMA guidelines were followed.

RESULTS

517 records were identified, and 5 papers met the inclusion criteria and were analyzed. Most of the articles emphasized the success of the technological support of AI in radiology at the expense of patient trust in AI and patient-centered communication in cancer disease. Practical implications and future guidelines were discussed according to the results.

CONCLUSIONS

AI has proven to be beneficial in helping clinicians with diagnosis. Future research may improve patients' trust through adequate information about the advantageous use of AI and an increase in medical compliance with adequate training on doctor-patient diagnosis communication.

Digital Twins in Radiology

A digital twin is a virtual model developed to accurately reflect a physical thing or a system. In radiology, a digital twin of a radiological device enables developers to test its characteristics, make alterations to the design or materials, and test the success or failure of the modifications in a virtual environment. Innovative technologies, such as AI and -omics sciences, may build virtual models for patients that are continuously adjustable based on live-tracked health/lifestyle parameters. Accordingly, healthcare could use digital twins to improve personalized medicine. Furthermore, the accumulation of digital twin models from real-world deployments will enable large cohorts of digital patients that may be used for virtual clinical trials and population studies. Through their further refinement, development, and application into clinical practice, digital twins could be crucial in the era of personalized medicine, revolutionizing how diseases are detected and managed. Although significant challenges remain in the development of digital twins, a structural modification to the current operating models is occurring, and radiologists can guide the introduction of such technology into healthcare.