1
|
Al-Naami N, Medoc N, Magnani M, Ghoniem M. Improved Visual Saliency of Graph Clusters with Orderable Node-Link Layouts. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2025; 31:1028-1038. [PMID: 39259626 DOI: 10.1109/tvcg.2024.3456167] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/13/2024]
Abstract
Graphs are often used to model relationships between entities. The identification and visualization of clusters in graphs enable insight discovery in many application areas, such as life sciences and social sciences. Force-directed graph layouts promote the visual saliency of clusters, as they bring adjacent nodes closer together, and push non-adjacent nodes apart. At the same time, matrices can effectively show clusters when a suitable row/column ordering is applied, but are less appealing to untrained users not providing an intuitive node-link metaphor. It is thus worth exploring layouts combining the strengths of the node-link metaphor and node ordering. In this work, we study the impact of node ordering on the visual saliency of clusters in orderable node-link diagrams, namely radial diagrams, arc diagrams and symmetric arc diagrams. Through a crowdsourced controlled experiment, we show that users can count clusters consistently more accurately, and to a large extent faster, with orderable node-link diagrams than with three state-of-the art force-directed layout algorithms, i.e., 'Linlog', 'Backbone' and 'sfdp'. The measured advantage is greater in case of low cluster separability and/or low compactness. A free copy of this paper and all supplemental materials are available at https://osf.io/kc3dg/.
Collapse
|
2
|
Mildau K, Ehlers H, Meisenburg M, Del Pup E, Koetsier RA, Torres Ortega LR, de Jonge NF, Singh KS, Ferreira D, Othibeng K, Tugizimana F, Huber F, van der Hooft JJJ. Effective data visualization strategies in untargeted metabolomics. Nat Prod Rep 2024. [PMID: 39620439 PMCID: PMC11610048 DOI: 10.1039/d4np00039k] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2024] [Indexed: 12/11/2024]
Abstract
Covering: 2014 to 2023 for metabolomics, 2002 to 2023 for information visualizationLC-MS/MS-based untargeted metabolomics is a rapidly developing research field spawning increasing numbers of computational metabolomics tools assisting researchers with their complex data processing, analysis, and interpretation tasks. In this article, we review the entire untargeted metabolomics workflow from the perspective of information visualization, visual analytics and visual data integration. Data visualization is a crucial step at every stage of the metabolomics workflow, where it provides core components of data inspection, evaluation, and sharing capabilities. However, due to the large number of available data analysis tools and corresponding visualization components, it is hard for both users and developers to get an overview of what is already available and which tools are suitable for their analysis. In addition, there is little cross-pollination between the fields of data visualization and metabolomics, leaving visual tools to be designed in a secondary and mostly ad hoc fashion. With this review, we aim to bridge the gap between the fields of untargeted metabolomics and data visualization. First, we introduce data visualization to the untargeted metabolomics field as a topic worthy of its own dedicated research, and provide a primer on cutting-edge visualization research into data visualization for both researchers as well as developers active in metabolomics. We extend this primer with a discussion of best practices for data visualization as they have emerged from data visualization studies. Second, we provide a practical roadmap to the visual tool landscape and its use within the untargeted metabolomics field. Here, for several computational analysis stages within the untargeted metabolomics workflow, we provide an overview of commonly used visual strategies with practical examples. In this context, we will also outline promising areas for further research and development. We end the review with a set of recommendations for developers and users on how to make the best use of visualizations for more effective and transparent communication of results.
Collapse
Affiliation(s)
- Kevin Mildau
- Bioinformatics Group, Wageningen University & Research, Wageningen, The Netherlands.
| | - Henry Ehlers
- Visualization Group, Institute of Visual Computing and Human-Centered Technology, TU Wien, Vienna, Austria.
| | - Mara Meisenburg
- Adaptation Physiology Group, Wageningen University & Research, Wageningen, The Netherlands
| | - Elena Del Pup
- Bioinformatics Group, Wageningen University & Research, Wageningen, The Netherlands.
| | - Robert A Koetsier
- Bioinformatics Group, Wageningen University & Research, Wageningen, The Netherlands.
| | | | - Niek F de Jonge
- Bioinformatics Group, Wageningen University & Research, Wageningen, The Netherlands.
| | - Kumar Saurabh Singh
- Bioinformatics Group, Wageningen University & Research, Wageningen, The Netherlands.
- Maastricht University Faculty of Science and Engineering, Plant Functional Genomics Maastricht, Limburg, The Netherlands
- Faculty of Environment, Science and Economy, University of Exeter, Penryl Cornwall, UK
| | | | - Kgalaletso Othibeng
- Department of Biochemistry, University of Johannesburg, Johannesburg, South Africa
| | - Fidele Tugizimana
- Department of Biochemistry, University of Johannesburg, Johannesburg, South Africa
| | - Florian Huber
- Centre for Digitalisation and Digitality, Düsseldorf University of Applied Sciences, Düsseldorf, Germany
| | - Justin J J van der Hooft
- Bioinformatics Group, Wageningen University & Research, Wageningen, The Netherlands.
- Department of Biochemistry, University of Johannesburg, Johannesburg, South Africa
| |
Collapse
|
3
|
Filipov V, Arleo A, Bogl M, Miksch S. On Network Structural and Temporal Encodings: A Space and Time Odyssey. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2024; 30:5847-5860. [PMID: 37647194 DOI: 10.1109/tvcg.2023.3310019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/01/2023]
Abstract
The dynamic network visualization design space consists of two major dimensions: network structural and temporal representation. As more techniques are developed and published, a clear need for evaluation and experimental comparisons between them emerges. Most studies explore the temporal dimension and diverse interaction techniques supporting the participants, focusing on a single structural representation. Empirical evidence about performance and preference for different visualization approaches is scattered over different studies, experimental settings, and tasks. This paper aims to comprehensively investigate the dynamic network visualization design space in two evaluations. First, a controlled study assessing participants' response times, accuracy, and preferences for different combinations of network structural and temporal representations on typical dynamic network exploration tasks, with and without the support of standard interaction methods. Second, the best-performing combinations from the first study are enhanced based on participants' feedback and evaluated in a heuristic-based qualitative study with visualization experts on a real-world network. Our results highlight node-link with animation and playback controls as the best-performing combination and the most preferred based on ratings. Matrices achieve similar performance to node-link in the first study but have considerably lower scores in our second evaluation. Similarly, juxtaposition exhibits evident scalability issues in more realistic analysis contexts.
Collapse
|
4
|
Giacomo ED, Didimo W, Liotta G, Montecchiani F, Tappini A. Comparative Study and Evaluation of Hybrid Visualizations of Graphs. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2024; 30:3503-3515. [PMID: 37018276 DOI: 10.1109/tvcg.2022.3233389] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Hybrid visualizations combine different metaphors into a single network layout, in order to help humans in finding the "right way" of displaying the different portions of the network, especially when it is globally sparse and locally dense. We investigate hybrid visualizations in two complementary directions: (i) On the one hand, we evaluate the effectiveness of different hybrid visualization models through a comparative user study; (ii) On the other hand, we estimate the usefulness of an interactive visualization that integrates all the considered hybrid models together. The results of our study provide some hints about the usefulness of the different hybrid visualizations for specific tasks of analysis and indicates that integrating different hybrid models into a single visualization may offer a valuable tool of analysis.
Collapse
|
5
|
Peeters J, Bot DM, Rovelo Ruiz G, Aerts J. Snowflake: visualizing microbiome abundance tables as multivariate bipartite graphs. FRONTIERS IN BIOINFORMATICS 2024; 4:1331043. [PMID: 38375239 PMCID: PMC10875061 DOI: 10.3389/fbinf.2024.1331043] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2023] [Accepted: 01/23/2024] [Indexed: 02/21/2024] Open
Abstract
Current visualizations in microbiome research rely on aggregations in taxonomic classifications or do not show less abundant taxa. We introduce Snowflake: a new visualization method that creates a clear overview of the microbiome composition in collected samples without losing any information due to classification or neglecting less abundant reads. Snowflake displays every observed OTU/ASV in the microbiome abundance table and provides a solution to include the data's hierarchical structure and additional information obtained from downstream analysis (e.g., alpha- and beta-diversity) and metadata. Based on the value-driven ICE-T evaluation methodology, Snowflake was positively received. Experts in microbiome research found the visualizations to be user-friendly and detailed and liked the possibility of including and relating additional information to the microbiome's composition. Exploring the topological structure of the microbiome abundance table allows them to quickly identify which taxa are unique to specific samples and which are shared among multiple samples (i.e., separating sample-specific taxa from the core microbiome), and see the compositional differences between samples. An R package for constructing and visualizing Snowflake microbiome composition graphs is available at https://gitlab.com/vda-lab/snowflake.
Collapse
Affiliation(s)
- Jannes Peeters
- Data Science Institute, Hasselt University, Diepenbeek, Belgium
| | - Daniël M. Bot
- Data Science Institute, Hasselt University, Diepenbeek, Belgium
| | - Gustavo Rovelo Ruiz
- Expertise Center for Digital Media, Hasselt University—Flanders Make, Diepenbeek, Belgium
| | - Jan Aerts
- Visual Data Analysis Lab, Department of Biosystems, KU Leuven, Leuven, Belgium
| |
Collapse
|
6
|
Abdelaal M, Schiele ND, Angerbauer K, Kurzhals K, Sedlmair M, Weiskopf D. Comparative Evaluation of Bipartite, Node-Link, and Matrix-Based Network Representations. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2023; 29:896-906. [PMID: 36191101 DOI: 10.1109/tvcg.2022.3209427] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
This work investigates and compares the performance of node-link diagrams, adjacency matrices, and bipartite layouts for visualizing networks. In a crowd-sourced user study ( n=150), we measure the task accuracy and completion time of the three representations for different network classes and properties. In contrast to the literature, which covers mostly topology-based tasks (e.g., path finding) in small datasets, we mainly focus on overview tasks for large and directed networks. We consider three overview tasks on networks with 500 nodes: (T1) network class identification, (T2) cluster detection, and (T3) network density estimation, and two detailed tasks: (T4) node in-degree vs. out-degree and (T5) representation mapping, on networks with 50 and 20 nodes, respectively. Our results show that bipartite layouts are beneficial for revealing the overall network structure, while adjacency matrices are most reliable across the different tasks.
Collapse
|
7
|
A Natural Language Interface for Automatic Generation of Data Flow Diagram using Web Extraction Techniques. JOURNAL OF KING SAUD UNIVERSITY - COMPUTER AND INFORMATION SCIENCES 2023. [DOI: 10.1016/j.jksuci.2023.01.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
|
8
|
Joos L, Jaeger-Honz S, Schreiber F, Keim DA, Klein K. Visual Comparison of Networks in VR. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2022; 28:3651-3661. [PMID: 36048995 DOI: 10.1109/tvcg.2022.3203001] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Networks are an important means for the representation and analysis of data in a variety of research and application areas. While there are many efficient methods to create layouts for networks to support their visual analysis, approaches for the comparison of networks are still underexplored. Especially when it comes to the comparison of weighted networks, which is an important task in several areas, such as biology and biomedicine, there is a lack of efficient visualization approaches. With the availability of affordable high-quality virtual reality (VR) devices, such as head-mounted displays (HMDs), the research field of immersive analytics emerged and showed great potential for using the new technology for visual data exploration. However, the use of immersive technology for the comparison of networks is still underexplored. With this work, we explore how weighted networks can be visually compared in an immersive VR environment and investigate how visual representations can benefit from the extended 3D design space. For this purpose, we develop different encodings for 3D node-link diagrams supporting the visualization of two networks within a single representation and evaluate them in a pilot user study. We incorporate the results into a more extensive user study comparing node-link representations with matrix representations encoding two networks simultaneously. The data and tasks designed for our experiments are similar to those occurring in real-world scenarios. Our evaluation shows significantly better results for the node-link representations, which is contrary to comparable 2D experiments and indicates a high potential for using VR for the visual comparison of networks.
Collapse
|
9
|
Carbonic: A Framework for Creating and Visualizing Complex Compound Graphs. APPLIED SCIENCES-BASEL 2022. [DOI: 10.3390/app12157541] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 02/04/2023]
Abstract
Advances in data generation and acquisition have resulted in a volume of available data of such magnitude that our ability to interpret and extract valuable knowledge from them has been surpassed. Our capacity to analyze data is hampered not only by their amount or their dimensionality, but also by their relationships and by the complexity of the systems they model. Compound graphs allow us to represent the existing relationships between nodes that are themselves hierarchically structured, so they are a natural substrate to support multiscale analysis of complex graphs. This paper presents Carbonic, a framework for interactive multiscale visual exploration and editing of compound graphs that incorporates several strategies for complexity management. It combines the representation of graphs at multiple levels of abstraction, with techniques for reducing the number of visible elements and for reducing visual cluttering. This results in a tool that allows both the exploration of existing graphs and the visual creation of compound graphs following a top-down approach that allows simultaneously observing the entities and their relationships at different scales. The results show the applicability of the developed framework to two use cases, demonstrating the usefulness of Carbonic for moving from information to knowledge.
Collapse
|
10
|
Angori L, Didimo W, Montecchiani F, Pagliuca D, Tappini A. Hybrid Graph Visualizations With ChordLink: Algorithms, Experiments, and Applications. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2022; 28:1288-1300. [PMID: 32784142 DOI: 10.1109/tvcg.2020.3016055] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Many real-world networks are globally sparse but locally dense. Typical examples are social networks, biological networks, and information networks. This double structural nature makes it difficult to adopt a homogeneous visualization model that clearly conveys both an overview of the network and the internal structure of its communities at the same time. As a consequence, the use of hybrid visualizations has been proposed. For instance, NodeTrix combines node-link and matrix-based representations (Henry et al., 2007). In this article we describe ChordLink, a hybrid visualization model that embeds chord diagrams, used to represent dense subgraphs, into a node-link diagram, which shows the global network structure. The visualization makes it possible to interactively highlight the structure of a community while keeping the rest of the layout stable. We discuss the intriguing algorithmic challenges behind the ChordLink model, present a prototype system that implements it, and illustrate case studies on real-world networks.
Collapse
|
11
|
Burch M, Ten Brinke KB, Castella A, Peters GKS, Shteriyanov V, Vlasvinkel R. Dynamic graph exploration by interactively linked node-link diagrams and matrix visualizations. Vis Comput Ind Biomed Art 2021; 4:23. [PMID: 34491465 PMCID: PMC8423958 DOI: 10.1186/s42492-021-00088-8] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2021] [Accepted: 07/21/2021] [Indexed: 11/30/2022] Open
Abstract
The visualization of dynamic graphs is a challenging task owing to the various properties of the underlying relational data and the additional time-varying property. For sparse and small graphs, the most efficient approach to such visualization is node-link diagrams, whereas for dense graphs with attached data, adjacency matrices might be the better choice. Because graphs can contain both properties, being globally sparse and locally dense, a combination of several visual metaphors as well as static and dynamic visualizations is beneficial. In this paper, a visually and algorithmically scalable approach that provides views and perspectives on graphs as interactively linked node-link and adjacency matrix visualizations is described. As the novelty of this technique, insights such as clusters or anomalies from one or several combined views can be used to influence the layout or reordering of the other views. Moreover, the importance of nodes and node groups can be detected, computed, and visualized by considering several layout and reordering properties in combination as well as different edge properties for the same set of nodes. As an additional feature set, an automatic identification of groups, clusters, and outliers is provided over time, and based on the visual outcome of the node-link and matrix visualizations, the repertoire of the supported layout and matrix reordering techniques is extended, and more interaction techniques are provided when considering the dynamics of the graph data. Finally, a small user experiment was conducted to investigate the usability of the proposed approach. The usefulness of the proposed tool is illustrated by applying it to a graph dataset, such as e co-authorships, co-citations, and a Comprehensible Perl Archive Network distribution.
Collapse
Affiliation(s)
- Michael Burch
- Eindhoven University of Technology, 5600MB, Eindhoven, The Netherlands.
| | | | - Adrien Castella
- Eindhoven University of Technology, 5600MB, Eindhoven, The Netherlands
| | | | - Vasil Shteriyanov
- Eindhoven University of Technology, 5600MB, Eindhoven, The Netherlands
| | - Rinse Vlasvinkel
- Eindhoven University of Technology, 5600MB, Eindhoven, The Netherlands
| |
Collapse
|
12
|
Linhares CDG, Ponciano JR, Paiva JGS, Travençolo BAN, Rocha LEC. A comparative analysis for visualizing the temporal evolution of contact networks: a user study. J Vis (Tokyo) 2021. [DOI: 10.1007/s12650-021-00759-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
|
13
|
Horak T, Berger P, Schumann H, Dachselt R, Tominski C. Responsive Matrix Cells: A Focus+Context Approach for Exploring and Editing Multivariate Graphs. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2021; 27:1644-1654. [PMID: 33074814 DOI: 10.1109/tvcg.2020.3030371] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
Matrix visualizations are a useful tool to provide a general overview of a graph's structure. For multivariate graphs, a remaining challenge is to cope with the attributes that are associated with nodes and edges. Addressing this challenge, we propose responsive matrix cells as a focus+context approach for embedding additional interactive views into a matrix. Responsive matrix cells are local zoomable regions of interest that provide auxiliary data exploration and editing facilities for multivariate graphs. They behave responsively by adapting their visual contents to the cell location, the available display space, and the user task. Responsive matrix cells enable users to reveal details about the graph, compare node and edge attributes, and edit data values directly in a matrix without resorting to external views or tools. We report the general design considerations for responsive matrix cells covering the visual and interactive means necessary to support a seamless data exploration and editing. Responsive matrix cells have been implemented in a web-based prototype based on which we demonstrate the utility of our approach. We describe a walk-through for the use case of analyzing a graph of soccer players and report on insights from a preliminary user feedback session.
Collapse
|
14
|
A. Leite R, Gschwandtner T, Miksch S, Gstrein E, Kuntner J. NEVA: Visual Analytics to Identify Fraudulent Networks. COMPUTER GRAPHICS FORUM : JOURNAL OF THE EUROPEAN ASSOCIATION FOR COMPUTER GRAPHICS 2020; 39:344-359. [PMID: 33132468 PMCID: PMC7584106 DOI: 10.1111/cgf.14042] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Revised: 02/21/2020] [Indexed: 06/11/2023]
Abstract
Trust-ability, reputation, security and quality are the main concerns for public and private financial institutions. To detect fraudulent behaviour, several techniques are applied pursuing different goals. For well-defined problems, analytical methods are applicable to examine the history of customer transactions. However, fraudulent behaviour is constantly changing, which results in ill-defined problems. Furthermore, analysing the behaviour of individual customers is not sufficient to detect more complex structures such as networks of fraudulent actors. We propose NEVA (Network dEtection with Visual Analytics), a Visual Analytics exploration environment to support the analysis of customer networks in order to reduce false-negative and false-positive alarms of frauds. Multiple coordinated views allow for exploring complex relations and dependencies of the data. A guidance-enriched component for network pattern generation, detection and filtering support exploring and analysing the relationships of nodes on different levels of complexity. In six expert interviews, we illustrate the applicability and usability of NEVA.
Collapse
Affiliation(s)
- Roger A. Leite
- Faculty of InformaticsVienna University of Technology (TU Wien)ViennaAustria
| | | | - Silvia Miksch
- Faculty of InformaticsVienna University of Technology (TU Wien)ViennaAustria
| | | | | |
Collapse
|
15
|
Behrisch M, Schreck T, Pfister H. GUIRO: User-Guided Matrix Reordering. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS 2019:1-1. [PMID: 31442977 DOI: 10.1109/tvcg.2019.2934300] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
Matrix representations are one of the main established and empirically proven to be effective visualization techniques for relational (or network) data. However, matrices-similar to node-link diagrams-are most effective if their layout reveals the underlying data topology. Given the many developed algorithms, a practical problem arises: "Which matrix reordering algorithm should I choose for my dataset at hand?" To make matters worse, different reordering algorithms applied to the same dataset may let significantly different visual matrix patterns emerge. This leads to the question of trustworthiness and explainability of these fully automated, often heuristic, black-box processes. We present GUIRO, a Visual Analytics system that helps novices, network analysts, and algorithm designers to open the black-box. Users can investigate the usefulness and expressiveness of 70 accessible matrix reordering algorithms. For network analysts, we introduce a novel model space representation and two interaction techniques for a user-guided reordering of rows or columns, and especially groups thereof (submatrix reordering). These novel techniques contribute to the understanding of the global and local dataset topology. We support algorithm designers by giving them access to 16 reordering quality metrics and visual exploration means for comparing reordering implementations on a row/column permutation level. We evaluated GUIRO in a guided explorative user study with 12 subjects, a case study demonstrating its usefulness in a real-world scenario, and through an expert study gathering feedback on our design decisions. We found that our proposed methods help even inexperienced users to understand matrix patterns and allow a user-guided steering of reordering algorithms. GUIRO helps to increase the transparency of matrix reordering algorithms, thus helping a broad range of users to get a better insight into the complex reordering process, in turn supporting data and reordering algorithm insights.
Collapse
|