Journey to the centre of the cell: Virtual reality immersion into scientific data

Label-Free Intracellular Multi-Specificity in Yeast Cells by Phase-Contrast Tomographic Flow Cytometry

In-flow phase-contrast tomography provides a 3D refractive index of label-free cells in cytometry systems. Its major limitation, as with any quantitative phase imaging approach, is the lack of specificity compared to fluorescence microscopy, thus restraining its huge potentialities in single-cell analysis and diagnostics. Remarkable results in introducing specificity are obtained through artificial intelligence (AI), but only for adherent cells. However, accessing the 3D fluorescence ground truth and obtaining accurate voxel-level co-registration of image pairs for AI training is not viable for high-throughput cytometry. The recent statistical inference approach is a significant step forward for label-free specificity but remains limited to cells' nuclei. Here, a generalized computational strategy based on a self-consistent statistical inference to achieve intracellular multi-specificity is shown. Various subcellular compartments (i.e., nuclei, cytoplasmic vacuoles, the peri-vacuolar membrane area, cytoplasm, vacuole-nucleus contact site) can be identified and characterized quantitatively at different phases of the cells life cycle by using yeast cells as a biological model. Moreover, for the first time, virtual reality is introduced for handling the information content of multi-specificity in single cells. Full fruition is proofed for exploring and interacting with 3D quantitative biophysical parameters of the identified compartments on demand, thus opening the route to a metaverse for 3D microscopy.

Virtual reality and augmented reality in radiation oncology education – A review and expert commentary

The field of radiation oncology is rapidly advancing through technological and biomedical innovation backed by robust research evidence. However, cancer professionals are notoriously time-poor, meaning there is a need for high quality, accessible and tailored oncologic education programs. While traditional teaching methods including lectures and other in-person delivery formats remain important, digital learning (DL) has provided additional teaching options that can be delivered flexibly and on-demand from anywhere in the world.

While evidence of this digital migration has been evident for some time now, it has not always been met with the same enthusiasm by the teaching community, in part due to questions about its pedagogical effectiveness. Many of these reservations have been driven by a rudimentary utilisation of the medium and inexperience with digital best-practice. With increasing familiarity and understanding of the medium, increasingly sophisticated and pedagogically-driven learning solutions can be produced.

This article will review the application of immersive digital learning tools in radiation oncology education. This includes first and second-generation Virtual Reality (VR) environments and Augmented Reality (AR). It will explore the data behind, and best-practice application of, each of these tools as well as giving practical tips for educators who are looking to implement (or refine) their use of these learning methods. It includes a discussion of how to match the digital learning methods to the content being taught and ends with a horizon scan of where the digital medium may take us in the future. This article is the second in a two-part series, with the companion piece being on Screen-Based Digital Learning Methods in Radiation Oncology.

Overall, the digital space is well-placed to cater to the evolving educational needs of oncology learners. Further uptake over the next decade is likely to be driven by the desire for flexible on demand delivery, high-yield products, engaging delivery methods and programs that are tailored to individual learning needs. Educational programs that embrace these principles will have unique opportunities to thrive in this space.

The Role of Immersion for Improving Extended Reality Analysis of Personalized Flow Simulations

PURPOSE

Computational models of flow in patient-derived arterial geometries have become a key paradigm of biomedical research. These fluid models are often challenging to visualize due to high spatial heterogeneity and visual complexity. Virtual immersive environments can offer advantageous visualization of spatially heterogeneous and complex systems. However, as different VR devices offer varying levels of immersion, there remains a crucial lack of understanding regarding what level of immersion is best suited for interactions with patient-specific flow models.

METHODS

We conducted a quantitative user evaluation with multiple VR devices testing an important use of hemodynamic simulations-analysis of surface parameters within complex patient-specific geometries. This task was compared for the semi-immersive zSpace 3D monitor and the fully immersive HTC Vive system.

RESULTS

The semi-immersive device was more accurate than the fully immersive device. The two devices showed similar results for task duration and performance (accuracy/duration). The accuracy of the semi-immersive device was also higher for arterial geometries of greater complexity and branching.

CONCLUSION

This assessment demonstrates that the level of immersion plays a significant role in the accuracy of assessing arterial flow models. We found that the semi-immersive VR device was a generally optimal choice for arterial visualization.

Advancing pharmacy and healthcare with virtual digital technologies

Digitalisation of the healthcare sector promises to revolutionise patient healthcare globally. From the different technologies, virtual tools including artificial intelligence, blockchain, virtual, and augmented reality, to name but a few, are providing significant benefits to patients and the pharmaceutical sector alike, ranging from improving access to clinicians and medicines, as well as improving real-time diagnoses and treatments. Indeed, it is envisioned that such technologies will communicate together in real-time, as well as with their physical counterparts, to create a large-scale, cyber healthcare system. Despite the significant benefits that virtual-based digital health technologies can bring to patient care, a number of challenges still remain, ranging from data security to acceptance within the healthcare sector. This review provides a timely account of the benefits and challenges of virtual health interventions, as well an outlook on how such technologies can be transitioned from research-focused towards real-world healthcare and pharmaceutical applications to transform treatment pathways for patients worldwide.

New Approach to Accelerated Image Annotation by Leveraging Virtual Reality and Cloud Computing

Three-dimensional imaging is at the core of medical imaging and is becoming a standard in biological research. As a result, there is an increasing need to visualize, analyze and interact with data in a natural three-dimensional context. By combining stereoscopy and motion tracking, commercial virtual reality (VR) headsets provide a solution to this critical visualization challenge by allowing users to view volumetric image stacks in a highly intuitive fashion. While optimizing the visualization and interaction process in VR remains an active topic, one of the most pressing issue is how to utilize VR for annotation and analysis of data. Annotating data is often a required step for training machine learning algorithms. For example, enhancing the ability to annotate complex three-dimensional data in biological research as newly acquired data may come in limited quantities. Similarly, medical data annotation is often time-consuming and requires expert knowledge to identify structures of interest correctly. Moreover, simultaneous data analysis and visualization in VR is computationally demanding. Here, we introduce a new procedure to visualize, interact, annotate and analyze data by combining VR with cloud computing. VR is leveraged to provide natural interactions with volumetric representations of experimental imaging data. In parallel, cloud computing performs costly computations to accelerate the data annotation with minimal input required from the user. We demonstrate multiple proof-of-concept applications of our approach on volumetric fluorescent microscopy images of mouse neurons and tumor or organ annotations in medical images.

Impact of Virtual Reality on Student Motivation in a High School Science Course

It is essential to expose students to real situations in science courses, to experience how classroom concepts are reflected in the real world. However, the materials and methods available are not always very adequate; for example, chemistry courses involve the supervision of reagents to avoid risky situations, in addition to the costs, logistics of preparing materials, and possible adverse environmental factors. As an alternative solution, the following experience was carried out using virtual reality (VR) equipment, with very realistic applications that allowed 304 fourth semester high school students to have an immersive, interactive, and contextualized experience of the disciplinary contents. The students were asked about their perception regarding the motivation and acceptance of the use of virtual reality. The results were 72% positive for attention, 61% positive for relevance, 64% positive for trust, and 71% positive for satisfaction. Also, they mentioned their intention to continue using this resource and create lines of research to study the different aspects that could form a disciplinary proposal for an entire course based on virtual reality.

Nanoscape, a data-driven 3D real-time interactive virtual cell environment

Our understanding of cellular and structural biology has reached unprecedented levels of detail, and computer visualisation techniques can be used to create three-dimensional (3D) representations of cells and their environment that are useful in both teaching and research. However, extracting and integrating the relevant scientific data, and then presenting them in an effective way, can pose substantial computational and aesthetic challenges. Here we report how computer artists, experts in computer graphics and cell biologists have collaborated to produce a tool called Nanoscape that allows users to explore and interact with 3D representations of cells and their environment that are both scientifically accurate and visually appealing. We believe that using Nanoscape as an immersive learning application will lead to an improved understanding of the complexities of cellular scales, densities and interactions compared with traditional learning modalities.

Application of Virtual Reality Based on 3D-CTA in Intracranial Aneurysm Surgery

As a popular technology in the field of human-computer interaction, virtual reality (VR) brings a brand new sensory experience to users by generating the environment. In recent years, while introducing the application of virtual reality technology, researchers have done a lot of work around virtual reality in many fields, such as the application of virtual reality technology in medical procedures. Combining the immersive and expandable features of virtual reality can improve the safety and accuracy of surgery. This article mainly introduces the application of 3D-CTA virtual reality technology in intracranial aneurysm surgery and aims to provide some ideas and directions for the improvement and progress of intracranial aneurysm surgery. This paper presents a research method based on virtual reality technology 3D-CTA in intracranial aneurysm surgery, including the application overview of 3D-CTA in intracranial aneurysm surgery and the virtual reality algorithm based on 3D-CTA for intracranial arteries. In addition, there is also the application of virtual reality CTA technology in the design of the intracranial aneurysm application system. Experimental results show that the average accuracy of 3D-CTA diagnosis based on virtual reality is 90.81%, and it can be put into use in the next step.

Cellular sociology regulates the hierarchical spatial patterning and organization of cells in organisms

Advances in single-cell biotechnology have increasingly revealed interactions of cells with their surroundings, suggesting a cellular society at the microscale. Similarities between cells and humans across multiple hierarchical levels have quantitative inference potential for reaching insights about phenotypic interactions that lead to morphological forms across multiple scales of cellular organization, namely cells, tissues and organs. Here, the functional and structural comparisons between how cells and individuals fundamentally socialize to give rise to the spatial organization are investigated. Integrative experimental cell interaction assays and computational predictive methods shape the understanding of societal perspective in the determination of the cellular interactions that create spatially coordinated forms in biological systems. Emerging quantifiable models from a simpler biological microworld such as bacterial interactions and single-cell organisms are explored, providing a route to model spatio-temporal patterning of morphological structures in humans. This analogical reasoning framework sheds light on structural patterning principles as a result of biological interactions across the cellular scale and up.

Bioscience education 2030 and beyond: Where will technology take the curriculum?

This brief review explores the ever-increasing role that technological affordances may play in the 21C biochemistry and molecular biology curriculum. We consider the need to develop digital and creative fluencies in our students and the importance of creativity and visualization in learning science. The potential of virtual reality (VR) platforms to complement these goals are discussed with a number of examples. Finally, we look into the future where to see how VR might fit into a future curriculum.

Expression of TRPC1 and SBEM protein in breast cancer tissue and its relationship with clinicopathological features and prognosis of patients

This study investigated the relationship of the expression of transient receptor potential channel 1 (TRPC1), small breast epithelial mucin (SBEM) in breast cancer tissues with clinical pathological features and prognosis of patients. Altogether 50 patients with breast cancer who were treated in Weifang People's hospital from April 2017 to November 2018 were selected, and the mRNA and protein differences of TRPC1 and SBEM in breast cancer patients and normal breast cancer tissues were detected by qRT-PCR and Western blot. Spearman test was used for correlation analysis. Logistic univariate and multivariate analysis were performed on the risk factors related to breast cancer metastasis in breast cancer patients. The expression of TRPC1 and SBEM in breast cancer tissues was significantly higher than that in normal breast tissues (P<0.001). The mRNA expression of TRPC1, SBEM and protein was not related to age, tumor size and tissue grade of breast cancer patients, but related to TNM stage, clinical stage and lymph node metastasis (P<0.001). The relative expression of TRPC1 was positively correlated with clinical stage of breast cancer (r=0.992, P<0.001). The relative expression of SBEM was positively correlated with the clinical stage of breast cancer (r=0.853, P<0.001). The relative expression of TRPC1 was positively correlated with TNM staging of breast cancer (r=0.860, P<0.001). The relative expression of SBEM was positively correlated with TNM staging of breast cancer (r=0.880, P<0.001). Multivariate conditional Logistic regression analysis showed that TNM staging, TRPC1, SBEM were independent risk factors for malignant breast cancer metastasis. On the contrary, expression of TRPC1 and SBEM in breast cancer tissues was up-regulated. TRPC1 and SBEM may be involved in the process of breast cancer occurrence, development and metastasis, and can be used as potential tissue biomarkers in diagnosis of breast cancer metastasis and disease assessment.

Exploration of the Application of Virtual Reality and Internet of Things in Film and Television Production Mode

In order to reduce some of the problems of technological restructuring and insufficient expansion in the current film and television production mode, the application of emerging technologies such as artificial intelligence (AI), virtual reality (VR), and Internet of Things (IoT) in the film and television industry is introduced in this research. First, a topical crawler tool was constructed to grab relevant texts about “AI”, “VR”, and “IoT” crossover “film and television”, and the grasping accuracy rate and recall rate of this tool were compared. Then, based on the extracted text, the data of recent development in related fields were extracted. The AdaBoost algorithm was used to improve the BP (Back Propagation) neural network (BPNN). This model was used to predict the future development scale of related fields. Finally, a virtual character interaction system based on IoT-sensor technology was built and its performance was tested. The results showed that the topical crawler tool constructed in this study had higher recall rate and accuracy than other tools, and a total of 188 texts related to AI, VR, and IoT crossover television films were selected after Naive Bayes classification. In addition, the error of the BPNN prediction model based on the AdaBoost algorithm was less than 20%, and it can effectively predict the future development scale of AI and other fields. In addition, the virtual character interaction system based on IoT technology constructed in this study has a high motion recognition rate, produces a strong sense of immersion among users, and can realize real-time capture and imitation of character movements. In a word, the field of AI and VR crossover film and television has great development prospects in the future. Therefore, the application of IoT technology in building the virtual-character interaction system can improve the effect of VR or AI film and television production.

A workflow for visualizing human cancer biopsies using large-format electron microscopy

Recent developments in large format electron microscopy have enabled generation of images that provide detailed ultrastructural information on normal and diseased cells and tissues. Analyses of these images increase our understanding of cellular organization and interactions and disease-related changes therein. In this manuscript, we describe a workflow for two-dimensional (2D) and three-dimensional (3D) imaging, including both optical and scanning electron microscopy (SEM) methods, that allow pathologists and cancer biology researchers to identify areas of interest from human cancer biopsies. The protocols and mounting strategies described in this workflow are compatible with 2D large format EM mapping, 3D focused ion beam-SEM and serial block face-SEM. The flexibility to use diverse imaging technologies available at most academic institutions makes this workflow useful and applicable for most life science samples. Volumetric analysis of the biopsies studied here revealed morphological, organizational and ultrastructural aspects of the tumor cells and surrounding environment that cannot be revealed by conventional 2D EM imaging. Our results indicate that although 2D EM is still an important tool in many areas of diagnostic pathology, 3D images of ultrastructural relationships between both normal and cancerous cells, in combination with their extracellular matrix, enables cancer researchers and pathologists to better understand the progression of the disease and identify potential therapeutic targets.

Deadeye Visualization Revisited: Investigation of Preattentiveness and Applicability in Virtual Environments

Visualizations rely on highlighting to attract and guide our attention. To make an object of interest stand out independently from a number of distractors, the underlying visual cue, e.g., color, has to be preattentive. In our prior work, we introduced Deadeye as an instantly recognizable highlighting technique that works by rendering the target object for one eye only. In contrast to prior approaches, Deadeye excels by not modifying any visual properties of the target. However, in the case of 2D visualizations, the method requires an additional setup to allow dichoptic presentation, which is a considerable drawback. As a follow-up to requests from the community, this paper explores Deadeye as a highlighting technique for 3D visualizations, because such stereoscopic scenarios support dichoptic presentation out of the box. Deadeye suppresses binocular disparities for the target object, so we cannot assume the applicability of our technique as a given fact. With this motivation, the paper presents quantitative evaluations of Deadeye in VR, including configurations with multiple heterogeneous distractors as an important robustness challenge. After confirming the preserved preattentiveness (all average accuracies above 90%) under such real-world conditions, we explore VR volume rendering as an example application scenario for Deadeye. We depict a possible workflow for integrating our technique, conduct an exploratory survey to demonstrate benefits and limitations, and finally provide related design implications.

Peppy: A virtual reality environment for exploring the principles of polypeptide structure

A key learning outcome for undergraduate biochemistry classes is a thorough understanding of the principles of protein structure. Traditional approaches to teaching this material, which include two-dimensional (2D) images on paper, physical molecular modeling kits, and projections of 3D structures into 2D, are unable to fully capture the dynamic 3D nature of proteins. We have built a virtual reality application, Peppy, aimed at facilitating teaching of the principles of protein secondary structure. Rather than attempt to model molecules with the same fidelity to the underlying physical chemistry as existing, research-oriented molecular modelling approaches, we took the more straightforward approach of harnessing the Unity video game physics engine. Indeed, the simplicity and limitations of our model are strengths in a teaching context, provoking questions and thus deeper understanding. Peppy allows exploration of the relative effects of hydrogen bonding (and electrostatic interactions more generally), backbone φ/ψ angles, basic chemical structure, and steric effects on a polypeptide structure in an accessible format that is novel, dynamic, and fun to use. Apart from describing the implementation and use of Peppy, we discuss the outcomes of deploying Peppy in undergraduate biochemistry courses.

Molecular Graphics: Bridging Structural Biologists and Computer Scientists

Visualization of molecular structures is one of the most common tasks carried out by structural biologists, typically using software, such as Chimera, COOT, PyMOL, or VMD. In this Perspective article, we outline how past developments in computer graphics and data visualization have expanded the understanding of biomolecular function, and we summarize recent advances that promise to further transform structural biology. We also highlight how progress in molecular graphics has been impeded by communication barriers between two communities: the computer scientists driving these advances, and the structural and computational biologists who stand to benefit. By pointing to canonical papers and explaining technical progress underlying new graphical developments in simple terms, we aim to improve communication between these communities; this, in turn, would help shape future developments in molecular graphics.

A Virtual Tour of the Cell: Impact of Virtual Reality on Student Learning and Engagement in the STEM Classroom

This study assesses the educational impact of virtual reality (VR) on student learning and engagement in a sophomore-level cell biology course taught at Otterbein University. Using VR, students toured the cell and its environment within the human body. Next, students completed a team challenge where they worked as pairs to match images of cell components taken from the virtual reality application with the appropriate name. Finally, students were given a voluntary survey that presented questions about their perceptions of the VR experience and the associated cell sorting activity. Survey results revealed that a majority of students enjoyed the VR experience and felt that it had a positive impact on their education. These results indicate the potential supportive role that VR may have in a variety of undergraduate courses.

Study of Commodity VR for Computational Material Sciences

Recent advancements in virtual reality (VR) devices and software environments make it possible to easily incorporate this technology for many applications, including computational materials science. For studying three-dimensional (3D) structure models and related chemical information, we focused on using a commodity VR device (VIVE) and an authoring tool (Unity). To visualize 3D chemical structures, disturbances like judder due to dropped frames should be eliminated from the VR experience to improve simulations. We propose a simple evaluation method that is straightforward for the nonexpert or novice VR user. We examine the major visualization representations including ball, ball and stick, and isosurface systems. For systematic benchmark measurements, a pendulum from the VR device was used to generate periodic oscillatory motion during measurements of a time series in frames per second (fps). For VIVE with a refresh rate of 90 Hz, judder occurred when less than 90 fps. We demonstrated the system size limitations for the results of molecular dynamics simulations of phase separation of ABA block copolymers and experimental observations of filler morphologies in rubber.

Applying Modern Virtual and Augmented Reality Technologies to Medical Images and Models

Recent technological innovations have created new opportunities for the increased adoption of virtual reality (VR) and augmented reality (AR) applications in medicine. While medical applications of VR have historically seen greater adoption from patient-as-user applications, the new era of VR/AR technology has created the conditions for wider adoption of clinician-as-user applications. Historically, adoption to clinical use has been limited in part by the ability of the technology to achieve a sufficient quality of experience. This article reviews the definitions of virtual and augmented reality and briefly covers the history of their development. Currently available options for consumer-level virtual and augmented reality systems are presented, along with a discussion of technical considerations for their adoption in the clinical environment. Finally, a brief review of the literature of medical VR/AR applications is presented prior to introducing a comprehensive conceptual framework for the viewing and manipulation of medical images in virtual and augmented reality. Using this framework, we outline considerations for placing these methods directly into a radiology-based workflow and show how it can be applied to a variety of clinical scenarios.

A Survey: Virtual Reality Model for Medical Diagnosis

Virtual reality is a new and a very revolutionizing concept and its progress in the field of medical diagnosis is taking this technology to another level. Simulators of virtual reality give essential aptitude for preparing in a controlled domain, operating patients free of pressure without supervision. These skills acquired can then be utilized in the training room. In medical applications virtual reality can be utilized for better picture control, enhanced picture understanding, enhanced quantitative correlations, and better planning of surgery. Not only has the virtual environment provided in the virtual reality helped the patients to cope with stress associated before the surgery but also helped in the reduction of pain. In this paper we have mainly focused on incorporating Virtual Reality in treatment of three diseases-breast cancers, colon cancer and Alzheimer’s. Also, we have made a comparison of the traditional methods, which already exist to treat the above diseases with the methods incorporating virtual reality. Finally we have stated the benefits of using virtual reality over traditional methods.

Molecular Visualization on the Holodeck

Can virtual reality be useful for visualizing and analyzing molecular structures and three-dimensional (3D) microscopy? Uses we are exploring include studies of drug binding to proteins and the effects of mutations, building accurate atomic models in electron microscopy and x-ray density maps, understanding how immune system cells move using 3D light microscopy, and teaching schoolchildren about biomolecules that are the machinery of life. Virtual reality (VR) offers immersive display with a wide field of view and head tracking for better perception of molecular architectures and uses 6-degree-of-freedom hand controllers for simple manipulation of 3D data. Conventional computer displays with trackpad, mouse and keyboard excel at two-dimensional tasks such as writing and studying research literature, uses for which VR technology is at present far inferior. Adding VR to the conventional computing environment could improve 3D capabilities if new user-interface problems can be solved. We have developed three VR applications: ChimeraX for analyzing molecular structures and electron and light microscopy data, AltPDB for collaborative discussions around atomic models, and Molecular Zoo for teaching young students characteristics of biomolecules. Investigations over three decades have produced an extensive literature evaluating the potential of VR in research and education. Consumer VR headsets are now affordable to researchers and educators, allowing direct tests of whether the technology is valuable in these areas. We survey here advantages and disadvantages of VR for molecular biology in the context of affordable and dramatically more powerful VR and graphics hardware than has been available in the past.