An Assessment of Imaging Informatics for Precision Medicine in Cancer

Value assessment of artificial intelligence in medical imaging: a scoping review

BACKGROUND

Artificial intelligence (AI) is seen as one of the major disrupting forces in the future healthcare system. However, the assessment of the value of these new technologies is still unclear, and no agreed international health technology assessment-based guideline exists. This study provides an overview of the available literature in the value assessment of AI in the field of medical imaging.

METHODS

We performed a systematic scoping review of published studies between January 2016 and September 2020 using 10 databases (Medline, Scopus, ProQuest, Google Scholar, and six related databases of grey literature). Information about the context (country, clinical area, and type of study) and mentioned domains with specific outcomes and items were extracted. An existing domain classification, from a European assessment framework, was used as a point of departure, and extracted data were grouped into domains and content analysis of data was performed covering predetermined themes.

RESULTS

Seventy-nine studies were included out of 5890 identified articles. An additional seven studies were identified by searching reference lists, and the analysis was performed on 86 included studies. Eleven domains were identified: (1) health problem and current use of technology, (2) technology aspects, (3) safety assessment, (4) clinical effectiveness, (5) economics, (6) ethical analysis, (7) organisational aspects, (8) patients and social aspects, (9) legal aspects, (10) development of AI algorithm, performance metrics and validation, and (11) other aspects. The frequency of mentioning a domain varied from 20 to 78% within the included papers. Only 15/86 studies were actual assessments of AI technologies. The majority of data were statements from reviews or papers voicing future needs or challenges of AI research, i.e. not actual outcomes of evaluations.

CONCLUSIONS

This review regarding value assessment of AI in medical imaging yielded 86 studies including 11 identified domains. The domain classification based on European assessment framework proved useful and current analysis added one new domain. Included studies had a broad range of essential domains about addressing AI technologies highlighting the importance of domains related to legal and ethical aspects.

Multiplex Immunofluorescence and the Digital Image Analysis Workflow for Evaluation of the Tumor Immune Environment in Translational Research

A robust understanding of the tumor immune environment has important implications for cancer diagnosis, prognosis, research, and immunotherapy. Traditionally, immunohistochemistry (IHC) has been regarded as the standard method for detecting proteins in situ, but this technique allows for the evaluation of only one cell marker per tissue sample at a time. However, multiplexed imaging technologies enable the multiparametric analysis of a tissue section at the same time. Also, through the curation of specific antibody panels, these technologies enable researchers to study the cell subpopulations within a single immunological cell group. Thus, multiplexed imaging gives investigators the opportunity to better understand tumor cells, immune cells, and the interactions between them. In the multiplexed imaging technology workflow, once the protocol for a tumor immune micro environment study has been defined, histological slides are digitized to produce high-resolution images in which regions of interest are selected for the interrogation of simultaneously expressed immunomarkers (including those co-expressed by the same cell) by using an image analysis software and algorithm. Most currently available image analysis software packages use similar machine learning approaches in which tissue segmentation first defines the different components that make up the regions of interest and cell segmentation, then defines the different parameters, such as the nucleus and cytoplasm, that the software must utilize to segment single cells. Image analysis tools have driven dramatic evolution in the field of digital pathology over the past several decades and provided the data necessary for translational research and the discovery of new therapeutic targets. The next step in the growth of digital pathology is optimization and standardization of the different tasks in cancer research, including image analysis algorithm creation, to increase the amount of data generated and their accuracy in a short time as described herein. The aim of this review is to describe this process, including an image analysis algorithm creation for multiplex immunofluorescence analysis, as an essential part of the optimization and standardization of the different processes in cancer research, to increase the amount of data generated and their accuracy in a short time.

Characterizing Immune Responses in Whole Slide Images of Cancer With Digital Pathology and Pathomics

Purpose of Review

Our goal is to show how readily available Pathomics tissue analytics can be used to study tumor immune interactions in cancer. We provide a brief overview of how Pathomics complements traditional histopathologic examination of cancer tissue samples. We highlight a novel Pathomics application, Tumor-TILs, that quantitatively measures and generates maps of tumor infiltrating lymphocytes in breast, pancreatic, and lung cancer by leveraging deep learning computer vision applications to perform automated analyses of whole slide images.

Recent Findings

Tumor-TIL maps have been generated to analyze WSIs from thousands of cases of breast, pancreatic, and lung cancer. We report the availability of these tools in an effort to promote collaborative research and motivate future development of ensemble Pathomics applications to discover novel biomarkers and perform a wide range of correlative clinicopathologic research in cancer immunopathology and beyond.

Summary

Tumor immune interactions in cancer are a fascinating aspect of cancer pathobiology with particular significance due to the emergence of immunotherapy. We present simple yet powerful specialized Pathomics methods that serve as powerful clinical research tools and potential standalone clinical screening tests to predict clinical outcomes and treatment responses for precision medicine applications in immunotherapy.

AI in Medical Imaging Informatics: Current Challenges and Future Directions

This paper reviews state-of-the-art research solutions across the spectrum of medical imaging informatics, discusses clinical translation, and provides future directions for advancing clinical practice. More specifically, it summarizes advances in medical imaging acquisition technologies for different modalities, highlighting the necessity for efficient medical data management strategies in the context of AI in big healthcare data analytics. It then provides a synopsis of contemporary and emerging algorithmic methods for disease classification and organ/ tissue segmentation, focusing on AI and deep learning architectures that have already become the de facto approach. The clinical benefits of in-silico modelling advances linked with evolving 3D reconstruction and visualization applications are further documented. Concluding, integrative analytics approaches driven by associate research branches highlighted in this study promise to revolutionize imaging informatics as known today across the healthcare continuum for both radiology and digital pathology applications. The latter, is projected to enable informed, more accurate diagnosis, timely prognosis, and effective treatment planning, underpinning precision medicine.

Deep learning workflow in radiology: a primer

Interest for deep learning in radiology has increased tremendously in the past decade due to the high achievable performance for various computer vision tasks such as detection, segmentation, classification, monitoring, and prediction. This article provides step-by-step practical guidance for conducting a project that involves deep learning in radiology, from defining specifications, to deployment and scaling. Specifically, the objectives of this article are to provide an overview of clinical use cases of deep learning, describe the composition of multi-disciplinary team, and summarize current approaches to patient, data, model, and hardware selection. Key ideas will be illustrated by examples from a prototypical project on imaging of colorectal liver metastasis. This article illustrates the workflow for liver lesion detection, segmentation, classification, monitoring, and prediction of tumor recurrence and patient survival. Challenges are discussed, including ethical considerations, cohorting, data collection, anonymization, and availability of expert annotations. The practical guidance may be adapted to any project that requires automated medical image analysis.

Enhancing Clinical Data and Clinical Research Data with Biomedical Ontologies - Insights from the Knowledge Representation Perspective

Objectives : There exists a communication gap between the biomedical informatics community on one side and the computer science/artificial intelligence community on the other side regarding the meaning of the terms “semantic integration" and “knowledge representation“. This gap leads to approaches that attempt to provide one-to-one mappings between data elements and biomedical ontologies. Our aim is to clarify the representational differences between traditional data management and semantic-web-based data management by providing use cases of clinical data and clinical research data re-representation. We discuss how and why one-to-one mappings limit the advantages of using Semantic Web Technologies (SWTs).

Methods : We employ commonly used SWTs, such as Resource Description Framework (RDF) and Ontology Web Language (OWL). We reuse pre-existing ontologies and ensure shared ontological commitment by selecting ontologies from a framework that fosters community-driven collaborative ontology development for biomedicine following the same set of principles.

Results : We demonstrate the results of providing SWT-compliant re-representation of data elements from two independent projects managing clinical data and clinical research data. Our results show how one-to-one mappings would hinder the exploitation of the advantages provided by using SWT.

Conclusions : We conclude that SWT-compliant re-representation is an indispensable step, if using the full potential of SWT is the goal. Rather than providing one-to-one mappings, developers should provide documentation that links data elements to graph structures to specify the re-representation.

Methods for Segmentation and Classification of Digital Microscopy Tissue Images

High-resolution microscopy images of tissue specimens provide detailed information about the morphology of normal and diseased tissue. Image analysis of tissue morphology can help cancer researchers develop a better understanding of cancer biology. Segmentation of nuclei and classification of tissue images are two common tasks in tissue image analysis. Development of accurate and efficient algorithms for these tasks is a challenging problem because of the complexity of tissue morphology and tumor heterogeneity. In this paper we present two computer algorithms; one designed for segmentation of nuclei and the other for classification of whole slide tissue images. The segmentation algorithm implements a multiscale deep residual aggregation network to accurately segment nuclear material and then separate clumped nuclei into individual nuclei. The classification algorithm initially carries out patch-level classification via a deep learning method, then patch-level statistical and morphological features are used as input to a random forest regression model for whole slide image classification. The segmentation and classification algorithms were evaluated in the MICCAI 2017 Digital Pathology challenge. The segmentation algorithm achieved an accuracy score of 0.78. The classification algorithm achieved an accuracy score of 0.81. These scores were the highest in the challenge.

How clinical imaging can assess cancer biology

Human cancers represent complex structures, which display substantial inter- and intratumor heterogeneity in their genetic expression and phenotypic features. However, cancers usually exhibit characteristic structural, physiologic, and molecular features and display specific biological capabilities named hallmarks. Many of these tumor traits are imageable through different imaging techniques. Imaging is able to spatially map key cancer features and tumor heterogeneity improving tumor diagnosis, characterization, and management. This paper aims to summarize the current and emerging applications of imaging in tumor biology assessment.

An annotated test-retest collection of prostate multiparametric MRI

Multiparametric Magnetic Resonance Imaging (mpMRI) is widely used for characterizing prostate cancer. Standard of care use of mpMRI in clinic relies on visual interpretation of the images by an expert. mpMRI is also increasingly used as a quantitative imaging biomarker of the disease. Little is known about repeatability of such quantitative measurements, and no test-retest datasets have been available publicly to support investigation of the technical characteristics of the MRI-based quantification in the prostate. Here we present an mpMRI dataset consisting of baseline and repeat prostate MRI exams for 15 subjects, manually annotated to define regions corresponding to lesions and anatomical structures, and accompanied by region-based measurements. This dataset aims to support further investigation of the repeatability of mpMRI-derived quantitative prostate measurements, study of the robustness and reliability of the automated analysis approaches, and to support development and validation of new image analysis techniques. The manuscript can also serve as an example of the use of DICOM for standardized encoding of the image annotation and quantification results.

Impact of sex, body mass index and initial pathologic diagnosis age on the incidence and prognosis of different types of cancer

Cancer represents a significant challenge for humankind, as early diagnosis and treatment are difficult to achieve. To systemically investigate the effect of sex, body mass index (BMI) and age on cancer incidence and prognosis, the data from 14,504 cases of cancer were downloaded from The Cancer Genome Atlas (TCGA). BMI was used to categorize each person as underweight, normal weight, overweight or obese. Two‑ and five‑year survival rates were applied to estimate the prognosis for each cancer type. All data were statistically analyzed. We identified that males were more susceptible to lung, liver and skin cancer when compared with females, whereas females were more susceptible to thyroid, breast and adrenal cortex cancer. High BMI (>25) was positively associated with the occurrence of cancer, although patients with high BMI at the time of initial diagnosis had higher two/five‑year survival rates. The survival rates for cancer were positively correlated with the age at initial pathologic diagnosis. Some types of cancer were associated with particularly young ages of onset, including adrenocortical carcinoma, cervical and endocervical cancers, brain lower grade glioma, pheochromocytoma and paraganglioma, testicular germ cell tumors and thyroid carcinoma. Hence, the early diagnosis and prognosis for these cancers need to be improved. In conclusion, sex, BMI and age are associated with the incidence and survival rates for cancers. These results could be used to supplement precision and personalized medicine.

Sensor, Signal, and Imaging Informatics

Objective: To summarize significant contributions to sensor, signal, and imaging informatics published in 2016. Methods: We conducted an extensive search using PubMed® and Web of Science® to identify the scientific contributions published in 2016 that addressed sensors, signals, and imaging in medical informatics. The three section editors selected 15 candidate best papers by consensus. Each candidate article was reviewed by the section editors and at least two other external reviewers. The final selection of the six best papers was conducted by the editorial board of the Yearbook. Results: The selected papers of 2016 demonstrate the important scientific advances in management and analysis of sensor, signal, and imaging information. Conclusion: The growing volume of signal and imaging data provides exciting new challenges and opportunities for research in medical informatics. Evolving technologies provide faster and more effective approaches for pattern recognition and diagnostic evaluation. The papers selected here offer a small glimpse of the high-quality scientific work published in 2016 in the domain of sensor, signal, and imaging informatics.