1
|
Zaman N, Ong J, Waisberg E, Masalkhi M, Lee AG, Tavakkoli A, Zuckerbrod S. Advanced Visualization Engineering for Vision Disorders: A Clinically Focused Guide to Current Technology and Future Applications. Ann Biomed Eng 2024; 52:178-207. [PMID: 37861913 DOI: 10.1007/s10439-023-03379-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2023] [Accepted: 10/04/2023] [Indexed: 10/21/2023]
Abstract
Head-mounted visualization technology, often in the form of virtual, augmented, and mixed reality (VAMR), has revolutionized how visual disorders may be approached clinically. In this manuscript, we review the available literature on VAMR for visual disorders and provide a clinically oriented guide to how VAMR technology has been deployed for visual impairments. The chief areas of clinical investigation with VAMR are divided include (1) vision assessment, (2) vision simulation, and (3) vision rehabilitation. We discuss in-depth the current literature of these areas in VAMR and upcoming/future applications to combat the detrimental impact of visual impairment worldwide.
Collapse
Affiliation(s)
- Nasif Zaman
- Human-Machine Perception Laboratory, University of Nevada, Reno, NV, USA
| | - Joshua Ong
- Michigan Medicine, University of Michigan, Ann Arbor, MI, USA
| | - Ethan Waisberg
- University College Dublin School of Medicine, Belfield, Dublin 4, Ireland.
| | - Mouayad Masalkhi
- University College Dublin School of Medicine, Belfield, Dublin 4, Ireland
| | - Andrew G Lee
- Department of Ophthalmology, Blanton Eye Institute, Houston Methodist Hospital, Houston, TX, USA
- The Houston Methodist Research Institute, Houston Methodist Hospital, Houston, TX, USA
- Departments of Ophthalmology, Neurology, and Neurosurgery, Weill Cornell Medicine, New York, NY, USA
- Department of Ophthalmology, University of Texas Medical Branch, Galveston, TX, USA
- University of Texas MD Anderson Cancer Center, Houston, TX, USA
- Texas A&M College of Medicine, Bryan, TX, USA
- Department of Ophthalmology, The University of Iowa Hospitals and Clinics, Iowa City, IA, USA
| | - Alireza Tavakkoli
- Human-Machine Perception Laboratory, University of Nevada, Reno, NV, USA
| | | |
Collapse
|
2
|
Żelechowski M, Faludi B, Karnam M, Gerig N, Rauter G, Cattin PC. Automatic patient positioning based on robot rotational workspace for extended reality. Int J Comput Assist Radiol Surg 2023; 18:1951-1959. [PMID: 37296352 PMCID: PMC10589133 DOI: 10.1007/s11548-023-02967-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/11/2023] [Accepted: 05/16/2023] [Indexed: 06/12/2023]
Abstract
PURPOSE Understanding the properties and aspects of the robotic system is essential to a successful medical intervention, as different capabilities and limits characterize each. Robot positioning is a crucial step in the surgical setup that ensures proper reachability to the desired port locations and facilitates docking procedures. This very demanding task requires much experience to master, especially with multiple trocars, increasing the barrier of entry for surgeons in training. METHODS Previously, we demonstrated an Augmented Reality-based system to visualize the rotational workspace of the robotic system and proved it helps the surgical staff to optimize patient positioning for single-port interventions. In this work, we implemented a new algorithm to allow for an automatic, real-time robotic arm positioning for multiple ports. RESULTS Our system, based on the rotational workspace data of the robotic arm and the set of trocar locations, can calculate the optimal position of the robotic arm in milliseconds for the positional and in seconds for the rotational workspace in virtual and augmented reality setups. CONCLUSIONS Following the previous work, we extended our system to support multiple ports to cover a broader range of surgical procedures and introduced the automatic positioning component. Our solution can decrease the surgical setup time and eliminate the need to repositioning the robot mid-procedure and is suitable both for the preoperative planning step using VR and in the operating room-running on an AR headset.
Collapse
Affiliation(s)
- Marek Żelechowski
- Center for medical Image Analysis & Navigation (CIAN), Department of Biomedical Engineering, University of Basel, Basel, Switzerland.
| | - Balázs Faludi
- Center for medical Image Analysis & Navigation (CIAN), Department of Biomedical Engineering, University of Basel, Basel, Switzerland
| | - Murali Karnam
- Bio-Inspired RObots for MEDicine-Laboratory (BIROMED-lab), Department of Biomedical Engineering, University of Basel, Basel, Switzerland
| | - Nicolas Gerig
- Bio-Inspired RObots for MEDicine-Laboratory (BIROMED-lab), Department of Biomedical Engineering, University of Basel, Basel, Switzerland
| | - Georg Rauter
- Bio-Inspired RObots for MEDicine-Laboratory (BIROMED-lab), Department of Biomedical Engineering, University of Basel, Basel, Switzerland
| | - Philippe C Cattin
- Center for medical Image Analysis & Navigation (CIAN), Department of Biomedical Engineering, University of Basel, Basel, Switzerland
| |
Collapse
|
3
|
Chan HS, Tang YM, Do CW, Ho Yin Wong H, Chan LYL, To S. Design and assessment of amblyopia, strabismus, and myopia treatment and vision training using virtual reality. Digit Health 2023; 9:20552076231176638. [PMID: 37312939 PMCID: PMC10259136 DOI: 10.1177/20552076231176638] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Accepted: 05/02/2023] [Indexed: 06/15/2023] Open
Abstract
Background Virtual reality is a relatively new intervention that has the potential to be used in the treatment of eye and vision problems. This article reviews the use of virtual reality-related interventions in amblyopia, strabismus, and myopia research. Methods Sources covered in the review included 48 peer-reviewed research published between January 2000 and January 2023 from five electronic databases (ACM Digital Library, IEEE Xplore, PubMed, ScienceDirect and Web of Science). To prevent any missing relevant articles, the keywords, and terms used in the search included "VR", "virtual reality", "amblyopia", "strabismus," and "myopia". Quality assessment and data extraction were performed independently by two authors to form a narrative synthesis to summarize findings from the included research. Results Total number of 48 references were reviewed. There were 31 studies published on amblyopia, 18 on strabismus, and 6 on myopia, with 7 studies overlapping amblyopia and strabismus. In terms of technology, smartphone-based virtual reality headset viewers were utilized more often in amblyopia research, but commercial standalone virtual reality headsets were used more frequently in myopia and strabismus-related research. The software and virtual environment were mostly developed based on vision therapy and dichoptic training paradigms. Conclusion It has been suggested that virtual reality technology offers a potentially effective tool for amblyopia, strabismus, and myopia studies. Nonetheless, a variety of factors, especially the virtual environment and systems employed in the data presented, must be explored before determining whether virtual reality can be effectively applied in clinical settings. This review is significant as the technology in virtual reality software and application design features have been investigated and considered for future reference.
Collapse
Affiliation(s)
- Hoi Sze Chan
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hung Hom, Hong Kong
| | - Yuk Ming Tang
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hung Hom, Hong Kong
| | - Chi Wai Do
- School of Optometry, The Hong Kong Polytechnic University, Hung Hom, Hong Kong
| | - Horace Ho Yin Wong
- School of Optometry, The Hong Kong Polytechnic University, Hung Hom, Hong Kong
| | - Lily YL Chan
- School of Optometry, The Hong Kong Polytechnic University, Hung Hom, Hong Kong
| | - Suet To
- Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hung Hom, Hong Kong
| |
Collapse
|
4
|
Ong J, Hariprasad SM, Chhablani J. Into the RetinaVerse: A New Frontier of Retina in the Metaverse. Ophthalmic Surg Lasers Imaging Retina 2022; 53:595-600. [PMID: 36378613 DOI: 10.3928/23258160-20221017-01] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|
5
|
Ma MKI, Saha C, Poon SHL, Yiu RSW, Shih KC, Chan YK. Virtual Reality and Augmented Reality- Emerging Screening and Diagnostic Techniques in Ophthalmology: a Systematic Review. Surv Ophthalmol 2022; 67:1516-1530. [PMID: 35181279 DOI: 10.1016/j.survophthal.2022.02.001] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Revised: 02/07/2022] [Accepted: 02/11/2022] [Indexed: 11/24/2022]
Abstract
In healthcare, virtual reality (VR) and augmented reality (AR) have been applied extensively for many purposes. Similar to other technologies such as telemedicine and artificial intelligence, VR and AR may improve clinical diagnosis and screening services in ophthalmology by alleviating current problems, including workforce shortage, diagnostic error, and underdiagnosis. In the past decade a number of studies and products have used VR and AR concepts to build clinical tests for ophthalmology, but comprehensive reviews on these studies are limited. Therefore, we conducted a systematic review on the use of VR and AR as a diagnostic and screening tool in ophthalmology. We identified 26 studies that implemented a variety of VR and AR tests on different conditions, including VR cover tests for binocular vision disorder, VR perimetry for glaucoma, and AR slit lamp biomicroscopy for retinal diseases. In general, while VR and AR tools can become standardized, automated, and cost-effective tests with good user experience, several weaknesses, including unsatisfactory accuracy, weak validation, and hardware limitations, have prevented these VR and AR tools from having wider clinical application. Also, a comparison between VR and AR is made to explain why studies have predominantly used VR rather than AR.
Collapse
Affiliation(s)
| | - Chinmoy Saha
- Department of Ophthalmology, Li Ka Shing Faculty of Medicine, University of Hong Kong
| | | | | | - Kendrick Co Shih
- Department of Ophthalmology, Li Ka Shing Faculty of Medicine, University of Hong Kong
| | - Yau Kei Chan
- Department of Ophthalmology, Li Ka Shing Faculty of Medicine, University of Hong Kong.
| |
Collapse
|
6
|
Iskander M, Ogunsola T, Ramachandran R, McGowan R, Al-Aswad LA. Virtual Reality and Augmented Reality in Ophthalmology: A Contemporary Prospective. Asia Pac J Ophthalmol (Phila) 2021; 10:244-252. [PMID: 34383716 PMCID: PMC9167643 DOI: 10.1097/apo.0000000000000409] [Citation(s) in RCA: 21] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022] Open
Abstract
PURPOSE Most published systematic reviews have focused on the use of virtual reality (VR)/augmented reality (AR) technology in ophthalmology as it relates to surgical training. To date, this is the first review that investigates the current state of VR/AR technology applied more broadly to the entire field of ophthalmology. METHODS PubMed, Embase, and CINAHL databases were searched systematically from January 2014 through December 1, 2020. Studies that discussed VR and/or AR as it relates to the field of ophthalmology and provided information on the technology used were considered. Abstracts, non-peer-reviewed literature, review articles, studies that reported only qualitative data, and studies without English translations were excluded. RESULTS A total of 77 studies were included in this review. Of these, 28 evaluated the use of VR/AR in ophthalmic surgical training/assessment and guidance, 7 in clinical training, 23 in diagnosis/screening, and 19 in treatment/therapy. 15 studies used AR, 61 used VR, and 1 used both. Most studies focused on the validity and usability of novel technologies. CONCLUSIONS Ophthalmology is a field of medicine that is well suited for the use of VR/AR. However, further longitudinal studies examining the practical feasibility, efficacy, and safety of such novel technologies, the cost-effectiveness, and medical/legal considerations are still needed. We believe that time will indeed foster further technological advances and lead to widespread use of VR/AR in routine ophthalmic practice.
Collapse
Affiliation(s)
- Mina Iskander
- Department of Ophthalmology, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, US
| | - Titilola Ogunsola
- Department of Ophthalmology, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, US
| | - Rithambara Ramachandran
- Department of Ophthalmology, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, US
| | - Richard McGowan
- Department of Epidemiology and Health Promotion, NYU College of Dentistry, New York, New York, US
| | - Lama A. Al-Aswad
- Department of Ophthalmology, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, US
- Department of Population Health, NYU Langone Health, NYU Grossman School of Medicine, New York, New York, US
| |
Collapse
|
7
|
Versek C, Banijamali SMA, Bex P, Lashkari K, Kamarthi S, Sridhar S. Portable Diagnostic System for Age-Related Macular Degeneration Screening Using Visual Evoked Potentials. Eye Brain 2021; 13:111-127. [PMID: 33953628 PMCID: PMC8092944 DOI: 10.2147/eb.s295745] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2020] [Accepted: 03/04/2021] [Indexed: 11/23/2022] Open
Abstract
Background Delayed Dark-Adapted vision Recovery (DAR) is a biomarker for Age-related Macular Degeneration (AMD), however its measurement is burdensome for patients and examiners. Methods In this study, we developed a portable, wireless and user-friendly system that employs a headset with a smartphone to deliver controlled photo-bleach and monocular pattern reversal stimuli, while using custom electroencephalography (EEG) electrodes and electronics in order to measure Dark-Adapted Visual Evoked Potentials (DAVEP) objectively and separately at the peripheral and central visual field. This is achieved in one comfortable 20-minute session, without requiring subject reporting. DAVEP responses post photo-bleach for up to 15 minutes were measured concurrently from both eyes in 12 AMD-patients, 1 degenerative myopia patient, and 8 controls who had no diagnosed macular vision loss. Results Robust positive polarity DAVEP responses were observed at 200-500 ms from stimulus onset to scotopic stimuli that have been seldom reported and analyzed previously. The amplitude recovery of the DAVEP response was significantly delayed in AMD patients as compared to controls. We developed DAVEP1 scores, a simple metric for DAR, which classified 90% of subject eyes correctly, indicating the presence of AMD in at least one eye of all pre-confirmed subjects with this diagnosis. Conclusion We developed a user-friendly, portable VEP system and DAVEP1 metric, which show a high potential to identify DAR-deficits in AMD-patients. This novel technology could aid in early diagnosis of AMD.
Collapse
Affiliation(s)
| | | | - Peter Bex
- Department of Psychology, Northeastern University, Boston, MA, USA
| | - Kameran Lashkari
- Department of Bioengineering, University of Massachusetts Dartmouth, Dartmouth, MA, USA
| | - Sagar Kamarthi
- Department of Mechanical and Industrial Engineering, Northeastern University, Boston, MA, USA
| | - Srinivas Sridhar
- NeuroFieldz Inc, Newton, MA, USA.,Departments of Physics, Bioengineering, and Chemical Engineering, Northeastern University, Boston, MA, USA.,Division of Radiation Oncology, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
8
|
Taulu S, Larson E. Unified Expression of the Quasi-Static Electromagnetic Field: Demonstration With MEG and EEG Signals. IEEE Trans Biomed Eng 2021; 68:992-1004. [PMID: 32746058 PMCID: PMC8486342 DOI: 10.1109/tbme.2020.3009053] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
Objective: Electromagnetic recordings are useful for non-invasive measurement of human brain activity. They typically sample electric potentials on the scalp or the magnetic field outside the head using electroencephalography (EEG) or magnetoencephalography (MEG), respectively. EEG and MEG are not, however, symmetric counterparts: EEG samples a scalar field via a line integral over the electric field between two points, while MEG samples projections of a vector-valued field by small sensors. Here we present a unified mathematical formalism for electromagnetic measurements, leading to useful interpretations and signal processing methods for EEG and MEG. Methods: We represent electric and magnetic fields as solutions of Laplace’s equation under the quasi-static approximation, each field representable as an expansion of the same vector spherical harmonics (VSH) but differently weighted by electro- and magnetostatic multipole moments, respectively. Results: We observe that the electric and the magnetic fields are mathematically symmetric but couple to the underlying electric source distribution in distinct ways via their corresponding multipole moments, which have concise mathematical forms. The VSH model also allows us to construct linear bases for MEG and EEG for signal processing and analysis, including interference suppression methods and system calibration. Conclusion: The VSH model is a powerful and simple approach for modeling quasi-static electromagnetic fields. Significance: Our formalism provides a unified framework for interpreting resolution questions, and paves the way for new processing and analysis methods.
Collapse
|
9
|
Telemedicine in ophthalmology. Part 2. “special teleophthalmology”. OPHTHALMOLOGY JOURNAL 2021. [DOI: 10.17816/ov46314] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022] Open
Abstract
In recent years, telemedicine (TM) has been gradually introduced into ophthalmology in the form of teleophthalmology (TO), because most of the eye diseases can be photographed and transmitted via Internet. The most wide development of TO has involved the field of diabetic retinopathy (DR) diagnosis, primarily due to the high prevalence of diabetes mellitus in the world. Examples of the well-established operation of remote DR screening centers exist in different countries of the world. There are many studies published, which compare a remote examination with a personal one, and according to their data, TO screening is no worse, than traditional screening. In addition to DR, TO also covers the diagnosis of glaucoma, age-related macular degeneration, and other ophthalmic conditions. In this article, we present an overview of modern TO centers in different countries, the features of their organization and global results that have been achieved during their existence. New technologies developed to facilitate the work of such centers will also be touched: image analysis algorithms, portable diagnostic equipment, medical information systems. The prospects for introducing TO into Russian medical practice are considered separately.
Collapse
|
10
|
Comparison of Visual Stimuli for Steady-State Visual Evoked Potential-Based Brain-Computer Interfaces in Virtual Reality Environment in terms of Classification Accuracy and Visual Comfort. COMPUTATIONAL INTELLIGENCE AND NEUROSCIENCE 2019; 2019:9680697. [PMID: 31354804 PMCID: PMC6636533 DOI: 10.1155/2019/9680697] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/26/2019] [Accepted: 06/03/2019] [Indexed: 11/24/2022]
Abstract
Recent studies on brain-computer interfaces (BCIs) based on the steady-state visual evoked potential (SSVEP) have demonstrated their use to control objects or generate commands in virtual reality (VR) environments. However, most SSVEP-based BCI studies performed in VR environments have adopted visual stimuli that are typically used in conventional LCD environments without considering the differences in the rendering devices (head-mounted displays (HMDs) used in the VR environments). The proximity between the visual stimuli and the eyes in HMDs can readily cause eyestrain, degrading the overall performance of SSVEP-based BCIs. Therefore, in the present study, we have tested two different types of visual stimuli—pattern-reversal checkerboard stimulus (PRCS) and grow/shrink stimulus (GSS)—on young healthy participants wearing HMDs. Preliminary experiments were conducted to investigate the visual comfort of each participant during the presentation of the visual stimuli. In subsequent online avatar control experiments, we observed considerable differences in the classification accuracy of individual participants based on the type of visual stimuli used to elicit SSVEP. Interestingly, there was a close relationship between the subjective visual comfort score and the online performance of the SSVEP-based BCI: most participants showed better classification accuracy under visual stimulus they were more comfortable with. Our experimental results suggest the importance of an appropriate visual stimulus to enhance the overall performance of the SSVEP-based BCIs in VR environments. In addition, it is expected that the appropriate visual stimulus for a certain user might be readily selected by surveying the user's visual comfort for different visual stimuli, without the need for the actual BCI experiments.
Collapse
|