Visual Analytics for Hypothesis-Driven Exploration in Computational Pathology

HistoMSC: Density and topology analysis for AI-based visual annotation of histopathology whole slide images

We introduce an end-to-end framework for the automated visual annotation of histopathology whole slide images. Our method integrates deep learning models to achieve precise localization and classification of cell nuclei with spatial data aggregation to extend classes of sparsely distributed nuclei across the entire slide. We introduce a novel and cost-effective approach to localization, leveraging a U-Net architecture and a ResNet-50 backbone. The performance is boosted through color normalization techniques, helping achieve robustness under color variations resulting from diverse scanners and staining reagents. The framework is complemented by a YOLO detection architecture, augmented with generative methods. For classification, we use context patches around each nucleus, fed to various deep architectures. Sparse nuclei-level annotations are then aggregated using kernel density estimation, followed by color-coding and isocontouring. This reduces visual clutter and provides per-pixel probabilities with respect to pathology taxonomies. Finally, we use Morse-Smale theory to generate abstract annotations, highlighting extrema in the density functions and potential spatial interactions in the form of abstract graphs. Thus, our visualization allows for exploration at scales ranging from individual nuclei to the macro-scale. We tested the effectiveness of our framework in an assessment by six pathologists using various neoplastic cases. Our results demonstrate the robustness and usefulness of the proposed framework in aiding histopathologists in their analysis and interpretation of whole slide images.

Visualization and Visual Analytics Approaches for Image and Video Datasets: A Survey

Image and video data analysis has become an increasingly important research area with applications in different domains such as security surveillance, healthcare, augmented and virtual reality, video and image editing, activity analysis and recognition, synthetic content generation, distance education, telepresence, remote sensing, sports analytics, art, non-photorealistic rendering, search engines, and social media. Recent advances in Artificial Intelligence (AI) and particularly deep learning have sparked new research challenges and led to significant advancements, especially in image and video analysis. These advancements have also resulted in significant research and development in other areas such as visualization and visual analytics, and have created new opportunities for future lines of research. In this survey paper, we present the current state of the art at the intersection of visualization and visual analytics, and image and video data analysis. We categorize the visualization papers included in our survey based on different taxonomies used in visualization and visual analytics research. We review these papers in terms of task requirements, tools, datasets, and application areas. We also discuss insights based on our survey results, trends and patterns, the current focus of visualization research, and opportunities for future research.