Visual Analytic Tools and Techniques in Population Health and Health Services Research: Protocol for a Scoping Review

Assessing a Digital Scorecard on Global Immunization Progress: Stakeholder Views and Implications for Enhancing Performance and Accountability

Global health agencies and regional and national stakeholders collaborated to develop the Immunization Agenda 2030 Scorecard, a digital data visualization platform displaying global, regional, and country-level immunization progress. The scorecard serves to focus attention and enable strategic actions around the measures visualized. To assess the scorecard's usability, appropriateness, and context for use, we interviewed 15 immunization officers working across five global regions. To further understand the implementation context, we also reviewed the characteristics of 15 public platforms visualizing population health data. We integrated thematic findings across both methods. Many platforms highlight service gaps and enable comparisons between geographies to foster political pressure for service improvements. We observed heterogeneity regarding the platforms' focus areas and participants' leading concerns, which were management capacity and resourcing. Furthermore, one-third of platforms were out of date. Results yielded recommendations for the scorecard, which participants felt was well suited to focus the attention of decision makers on key immunization data. A simpler design coupled with implementation strategies that more actively engage policymakers would better align the scorecard with other public platforms engaging intended users. For population health platforms to serve as effective accountability mechanisms, studying implementation determinants, including usability testing, is vital to meet stakeholder needs.

Modelado de tópicos aplicado al análisis del papel del aprendizaje automático en revisiones sistemáticas

El objetivo de la investigación fue analizar el papel del aprendizaje automático de datos en las revisiones sistemáticas de literatura. Se aplicó la técnica de Procesamiento de Lenguaje Natural denominada modelado de tópicos, a un conjunto de títulos y resúmenes recopilados de la base de datos Scopus. Especificamente se utilizó la técnica de Asignación Latente de Dirichlet (LDA), a partir de la cual se lograron descubrir y comprender las temáticas subyacentes en la colección de documentos. Los resultados mostraron la utilidad de la técnica utilizada en la revisión exploratoria de literatura, al permitir agrupar los resultados por temáticas. Igualmente, se pudo identificar las áreas y actividades específicas donde más se ha aplicado el aprendizaje automático, en lo referente a revisiones de literatura. Se concluye que la técnica LDA es una estrategia fácil de utilizar y cuyos resultados permiten abordar una amplia colección de documentos de manera sistemática y coherente, reduciendo notablemente el tiempo de la revisión.

Interactive Visualization Applications in Population Health and Health Services Research: Systematic Scoping Review

BACKGROUND

Simple visualizations in health research data, such as scatter plots, heat maps, and bar charts, typically present relationships between 2 variables. Interactive visualization methods allow for multiple related facets such as numerous risk factors to be studied simultaneously, leading to data insights through exploring trends and patterns from complex big health care data. The technique presents a powerful tool that can be used in combination with statistical analysis for knowledge discovery, hypothesis generation and testing, and decision support.

OBJECTIVE

The primary objective of this scoping review is to describe and summarize the evidence of interactive visualization applications, methods, and tools being used in population health and health services research (HSR) and their subdomains in the last 15 years, from January 1, 2005, to March 30, 2019. Our secondary objective is to describe the use cases, metrics, frameworks used, settings, target audience, goals, and co-design of applications.

METHODS

We adapted standard scoping review guidelines with a peer-reviewed search strategy: 2 independent researchers at each stage of screening and abstraction, with a third independent researcher to arbitrate conflicts and validate findings. A comprehensive abstraction platform was built to capture the data from diverse bodies of literature, primarily from the computer science and health care sectors. After screening 11,310 articles, we present findings from 56 applications from interrelated areas of population health and HSR, as well as their subdomains such as epidemiologic surveillance, health resource planning, access, and use and costs among diverse clinical and demographic populations.

RESULTS

In this companion review to our earlier systematic synthesis of the literature on visual analytics applications, we present findings in 6 major themes of interactive visualization applications developed for 8 major problem categories. We found a wide application of interactive visualization methods, the major ones being epidemiologic surveillance for infectious disease, resource planning, health service monitoring and quality, and studying medication use patterns. The data sources included mostly secondary administrative and electronic medical record data. In addition, at least two-thirds of the applications involved participatory co-design approaches while introducing a distinct category, embedded research, within co-design initiatives. These applications were in response to an identified need for data-driven insights into knowledge generation and decision support. We further discuss the opportunities stemming from the use of interactive visualization methods in studying global health; inequities, including social determinants of health; and other related areas. We also allude to the challenges in the uptake of these methods.

CONCLUSIONS

Visualization in health has strong historical roots, with an upward trend in the use of these methods in population health and HSR. Such applications are being fast used by academic and health care agencies for knowledge discovery, hypotheses generation, and decision support.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/14019.

Visual Analytics: A Method to Explore Natural Histories of Oral Epithelial Dysplasia

Risk assessment and follow-up of oral potentially malignant disorders in patients with mild or moderate oral epithelial dysplasia is an ongoing challenge for improved oral cancer prevention. Part of the challenge is a lack of understanding of how observable features of such dysplasia, gathered as data by clinicians during follow-up, relate to underlying biological processes driving progression. Current research is at an exploratory phase where the precise questions to ask are not known. While traditional statistical and the newer machine learning and artificial intelligence methods are effective in well-defined problem spaces with large datasets, these are not the circumstances we face currently. We argue that the field is in need of exploratory methods that can better integrate clinical and scientific knowledge into analysis to iteratively generate viable hypotheses. In this perspective, we propose that visual analytics presents a set of methods well-suited to these needs. We illustrate how visual analytics excels at generating viable research hypotheses by describing our experiences using visual analytics to explore temporal shifts in the clinical presentation of epithelial dysplasia. Visual analytics complements existing methods and fulfills a critical and at-present neglected need in the formative stages of inquiry we are facing.

Policy-makers' views on translating burden of disease estimates in health policies: bridging the gap through data visualization

Background

Knowledge Translation (KT) and data visualization play a vital role in the dissemination of data and information to improve healthcare systems. A better understanding of KT and its utility requires examining its processes, and how these interact with available data tools and platforms and various users.

In this context, the aim of this paper is to understand how relevant users interact with data visualization tools, in particular Global Burden of Disease (GBD) visualizations, while also examining KT processes related to data visualization.

Methods

A qualitative case-study consisting of semi-structured interviews with eight policy officers. Interviewees were selected by the Institute for Health Metrics and Evaluation (IHME) from three countries: Canada, Kenya and New Zealand. Data were analyzed through framework coding, using qualitative analysis software.

Results

Policy officers’ responses indicated that data can prompt action by engaging users, and effective delivery and translation of data was enhanced by data visualization tools. GBD was considered valuable for use in policy (e.g., planning and prioritizing health policy; facilitating accountability; and tracking and monitoring progress and trends over time and between countries). Using GBD and data visualization tools, participants quickly and easily accessed key information and turned it into actionable messages; engaging visuals captured decision-makers’ attention while providing information in a digestible, time-saving manner. However, participants emphasized an overall disconnect between research and public health. Functionality and technical issues, e.g., absence of tool guides and tool complexity, as well as lacking buy-in and awareness of certain tools from those less familiar with GBD, were named as major barriers. In order to address this “know-do” gap, user-friendly knowledge translation tools were stated as crucial, as was the importance of collaboration and leveraging different insights from data generators and users to inform health policy.

Conclusions

Policy officers aware of KT are willing to utilize data visualization tools as they value them as an engaging and important mechanism to foster the use of GBD data in policy-making. To further facilitate policy officers’ efforts to effectively use GBD data in policy and practice, further research is required into how users perceive and interact with data visualization and other KT tools.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13690-021-00537-z.

Visual Analytic Tools and Techniques in Population Health and Health Services Research: Scoping Review

BACKGROUND

Visual analytics (VA) promotes the understanding of data with visual, interactive techniques, using analytic and visual engines. The analytic engine includes automated techniques, whereas common visual outputs include flow maps and spatiotemporal hot spots.

OBJECTIVE

This scoping review aims to address a gap in the literature, with the specific objective to synthesize literature on the use of VA tools, techniques, and frameworks in interrelated health care areas of population health and health services research (HSR).

METHODS

Using the 2018 PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines, the review focuses on peer-reviewed journal articles and full conference papers from 2005 to March 2019. Two researchers were involved at each step, and another researcher arbitrated disagreements. A comprehensive abstraction platform captured data from diverse bodies of the literature, primarily from the computer and health sciences.

RESULTS

After screening 11,310 articles, findings from 55 articles were synthesized under the major headings of visual and analytic engines, visual presentation characteristics, tools used and their capabilities, application to health care areas, data types and sources, VA frameworks, frameworks used for VA applications, availability and innovation, and co-design initiatives. We found extensive application of VA methods used in areas of epidemiology, surveillance and modeling, health services access, use, and cost analyses. All articles included a distinct analytic and visualization engine, with varying levels of detail provided. Most tools were prototypes, with 5 in use at the time of publication. Seven articles presented methodological frameworks. Toward consistent reporting, we present a checklist, with an expanded definition for VA applications in health care, to assist researchers in sharing research for greater replicability. We summarized the results in a Tableau dashboard.

CONCLUSIONS

With the increasing availability and generation of big health care data, VA is a fast-growing method applied to complex health care data. What makes VA innovative is its capability to process multiple, varied data sources to demonstrate trends and patterns for exploratory analysis, leading to knowledge generation and decision support. This is the first review to bridge a critical gap in the literature on VA methods applied to the areas of population health and HSR, which further indicates possible avenues for the adoption of these methods in the future. This review is especially important in the wake of COVID-19 surveillance and response initiatives, where many VA products have taken center stage.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/14019.

Experts' perceptions on the use of visual analytics for complex mental healthcare planning: an exploratory study

BACKGROUND

Health experts including planners and policy-makers face complex decisions in diverse and constantly changing healthcare systems. Visual analytics may play a critical role in supporting analysis of complex healthcare data and decision-making. The purpose of this study was to examine the real-world experience that experts in mental healthcare planning have with visual analytics tools, investigate how well current visualisation techniques meet their needs, and suggest priorities for the future development of visual analytics tools of practical benefit to mental healthcare policy and decision-making.

METHODS

Health expert experience was assessed by an online exploratory survey consisting of a mix of multiple choice and open-ended questions. Health experts were sampled from an international pool of policy-makers, health agency directors, and researchers with extensive and direct experience of using visual analytics tools for complex mental healthcare systems planning. We invited them to the survey, and the experts' responses were analysed using statistical and text mining approaches.

RESULTS

The forty respondents who took part in the study recognised the complexity of healthcare systems data, but had most experience with and preference for relatively simple and familiar visualisations such as bar charts, scatter plots, and geographical maps. Sixty-five percent rated visual analytics as important to their field for evidence-informed decision-making processes. Fifty-five percent indicated that more advanced visual analytics tools were needed for their data analysis, and 67.5% stated their willingness to learn new tools. This was reflected in text mining and qualitative synthesis of open-ended responses.

CONCLUSIONS

This exploratory research provides readers with the first self-report insight into expert experience with visual analytics in mental healthcare systems research and policy. In spite of the awareness of their importance for complex healthcare planning, the majority of experts use simple, readily available visualisation tools. We conclude that co-creation and co-development strategies will be required to support advanced visual analytics tools and skills, which will become essential in the future of healthcare.