Applying machine-learning to rapidly analyze large qualitative text datasets to inform the COVID-19 pandemic response: comparing human and machine-assisted topic analysis techniques

Student and Attending Preceptor Perceptions of Longitudinal Clinic as a Surgical Education and Assessment Tool

INTRODUCTION

As undergraduate medical education becomes increasingly longitudinal, particular attention is needed to maintain sufficient exposure to surgical disciplines. While traditional medical student clerkships are isolated 4 to 8-wk blocks on a single service, one unique adjunct to the traditional clerkship model is the continuity clinic (CC): a weekly longitudinal requirement that occurs either during the traditional clerkship or following clerkships while conducting independent research. This study compares attending surgeon and medical student perceptions of important characteristics in evaluating a student in CC and the perceived utility of this experience in assessment and preparation for subinternships.

METHODS

Attending preceptors in surgical specialties and medical students from two academic institutions who completed a surgical CC were surveyed on the importance of various characteristics in CC evaluation. Thirteen characteristics were ranked by importance (1-most important, 13-least important). Subjects were asked to rate the value of CC for evaluation and educational purposes. Students also completed presurveys/postsurveys examining their confidence in clinical skills before and after CC. Analysis was performed using Fisher's exact, Mann-Whitney, and unpaired t-tests where appropriate. Free-text comments were analyzed using natural language processing topic modeling.

RESULTS

Altogether, 67 medical students and 38 attending surgeons completed the survey. Students ranked hard skills as more important and soft skills as less important in CC evaluation compared to attendings. Students ranked knowledge related to interpretation of radiologic or laboratory results, surgical planning, and routine disease processes higher than attending surgeons. Students ranked hard skills such as patient presentation and documentation, and soft skills such as interpersonal and communication skills and professionalism significantly lower than attendings. Following participation in CC, students reported increased confidence in several skills, including perioperative consultation, preoperative assessment, surgical planning, and disease surveillance, as well as improved self-evaluation of preparedness for subinternship. Forty-two percent of students planned to request a letter of recommendation from their CC preceptors, and attendings rated the assessment value of CC as at least equivalent to a subinternship (mean 5.6/10, 1-worse than subinternship, 10-better than subinternship for assessment).

CONCLUSIONS

CC is an educational tool that facilitates maintenance and improvement in student confidence in clinical skills in the perioperative setting. In evaluating performance, students tended to rank hard skills as more important and soft skills as less important than their attending preceptors. Notably, attendings saw CC as a comparable assessment tool to subinternships. As undergraduate medical education continues to implement longitudinal experiences and research years, CC should be considered as a strategy to improve perioperative education and promote self-efficacy in clinical skills.

Why Research From Lower- and Middle-Income Countries Matters to Evidence-Based Intervention: A State of the Science Review of ACT Research as an Example

Despite the global nature of psychological issues, an overwhelming majority of research originates from a small segment of the world's population living in high-income countries (HICs). This disparity risks distorting our understanding of psychological phenomena by underrepresenting the cultural and contextual diversity of human experience. Research from lower- and middle-income countries (LMIC) is also less frequently cited, both because it is seemingly viewed as a "special case" and because it is less well known due to language differences and biases in indexing algorithms. Acknowledging and actively addressing this imbalance is crucial for a more inclusive, diverse, and effective science of evidence-based intervention. In this state-of-the-science review, we used a machine learning method to identify key topics in LMIC research on Acceptance and Commitment Therapy (ACT), choosing ACT due to the significant body of work from LMICs. We also examined one indication of study quality (study size), and overall citations. Research in LMICs was often nonindexed, leading to lower citations, but study size could not explain a lack of indexing. Many objectively identified topics in ACT research became invisible when LMIC research was ignored. Specific countries exhibited potentially important differences in the topics. We conclude that strong and affirmative actions are needed by scientific associations and others to ensure that research from LMICs is conducted, known, indexed, and used by CBT researchers and others interested in evidence-based intervention science.

Making the most of big qualitative datasets: a living systematic review of analysis methods

Introduction

Qualitative data provides deep insights into an individual's behaviors and beliefs, and the contextual factors that may shape these. Big qualitative data analysis is an emerging field that aims to identify trends and patterns in large qualitative datasets. The purpose of this review was to identify the methods used to analyse large bodies of qualitative data, their cited strengths and limitations and comparisons between manual and digital analysis approaches.

Methods

A multifaceted approach has been taken to develop the review relying on academic, gray and media-based literature, using approaches such as iterative analysis, frequency analysis, text network analysis and team discussion.

Results

The review identified 520 articles that detailed analysis approaches of big qualitative data. From these publications a diverse range of methods and software used for analysis were identified, with thematic analysis and basic software being most common. Studies were most commonly conducted in high-income countries, and the most common data sources were open-ended survey responses, interview transcripts, and first-person narratives.

Discussion

We identified an emerging trend to expand the sources of qualitative data (e.g., using social media data, images, or videos), and develop new methods and software for analysis. As the qualitative analysis field may continue to change, it will be necessary to conduct further research to compare the utility of different big qualitative analysis methods and to develop standardized guidelines to raise awareness and support researchers in the use of more novel approaches for big qualitative analysis.

Systematic review registration

https://osf.io/hbvsy/?view_only=.

The Applications of Artificial Intelligence (AI)-Driven Tools in Virus-Like Particles (VLPs) Research

Viral-like particles (VLPs) represent versatile nanoscale structures mimicking the morphology and antigenic characteristics of viruses, devoid of genetic material, making them promising candidates for various biomedical applications. The integration of artificial intelligence (AI) into VLP research has catalyzed significant advancements in understanding, production, and therapeutic applications of these nanostructures. This comprehensive review explores the collaborative utilization of AI tools, computational methodologies, and state-of-the-art technologies within the VLP domain. AI's involvement in bioinformatics facilitates sequencing and structure prediction, unraveling genetic intricacies and three-dimensional configurations of VLPs. Furthermore, AI-enabled drug discovery enables virtual screening, demonstrating promise in identifying compounds to inhibit VLP activity. In VLP production, AI optimizes processes by providing strategies for culture conditions, nutrient concentrations, and growth kinetics. AI's utilization in image analysis and electron microscopy expedites VLP recognition and quantification. Moreover, network analysis of protein-protein interactions through AI tools offers an understanding of VLP interactions. The integration of multi-omics data via AI analytics provides a comprehensive view of VLP behavior. Predictive modeling utilizing machine learning algorithms aids in forecasting VLP stability, guiding optimization efforts. Literature mining facilitated by text mining algorithms assists in summarizing information from the VLP knowledge corpus. Additionally, AI's role in laboratory automation enhances experimental efficiency. Addressing data security concerns, AI ensures the protection of sensitive information in the digital era of VLP research. This review serves as a roadmap, providing insights into AI's current and future applications in VLP research, thereby guiding innovative directions in medicine and beyond.

User feedback on the NHS test & Trace Service during COVID-19: The use of machine learning to analyse free-text data from 37,914 England adults

Objectives

The UK government's approach to the pandemic relies on a test, trace and isolate strategy, mainly implemented via the digital NHS Test & Trace Service. Feedback on user experience is central to the successful development of public-facing Services. As the situation dynamically changes and data accumulate, interpretation of feedback by humans becomes time-consuming and unreliable. The specific objectives were to 1) evaluate a human-in-the-loop machine learning technique based on structural topic modelling in terms of its Service ability in the analysis of vast volumes of free-text data, 2) generate actionable themes that can be used to increase user satisfaction of the Service.

Methods

We evaluated an unsupervised Topic Modelling approach, testing models with 5-40 topics and differing covariates. Two human coders conducted thematic analysis to interpret the topics. We identified a Structural Topic Model with 25 topics and metadata as covariates as the most appropriate for acquiring insights.

Results

Results from analysis of feedback by 37,914 users from May 2020 to March 2021 highlighted issues with the Service falling within three major themes: multiple contacts and incompatible contact method and incompatible contact method, confusion around isolation dates and tracing delays, complex and rigid system.

Conclusions

Structural Topic Modelling coupled with thematic analysis was found to be an effective technique to rapidly acquire user insights. Topic modelling can be a quick and cost-effective method to provide high quality, actionable insights from free-text feedback to optimize public health Services.