A non-contact interactive stereo display system for exploring human anatomy

Stereoscopic display based on Virtual Reality (VR) can facilitate clinicians observing 3 D anatomical models with the depth cue which lets them understand the spatial relationship between different anatomical structures intuitively. However, there are few input devices available in the sterile field of the operating room for controlling 3 D anatomical models. This paper presents a cost-effective VR application for stereo display of 3 D anatomical models with non-contact interaction. The system is integrated with hand gesture interaction and voice interaction to achieve non-contact interaction. Hand gesture interaction is based on Leap Motion. Voice interaction is implemented based on Bing Speech for English language and Aitalk for Chinese language. A local database is designed to record the anatomical terminologies organized in a tree structure, and provided to the speech recognition engine for querying these uncommon words. Ten participants were asked to practice the proposed system and compare it with the common interactive manners. The results show that our system is more efficient than the common interactive manner and prove the feasibility and practicability of the proposed system used in the sterile field.

A non-contact interactive stereo display system for exploring human anatomy

Stereoscopic display based on Virtual Reality (VR) can facilitate doctors to observe the 3 D virtual anatomical models with the depth cues, assist them in intuitively investigating the spatial relationship between different anatomical structures without mental imagination. However, there is few input device can be used in controlling the virtual anatomical models in the sterile operating room. This paper presents a cost-effective VR application system for demonstration of 3 D virtual anatomical models with non-contact interaction and stereo display. The system is integrated with hand gesture interaction and voice interaction to achieve non-contact interaction. Hand gesture interaction is implemented based on a Leap Motion controller mounted on the Oculus Rift DK2. Voice is converted into operation using Bing Speech for English language and Aitalk for Chinese language, respectively. A local relationship database is designed to record the anatomical terminologies to speech recognition engine to query these uncommon words. The hierarchical nature of these terminologies is also recorded in a tree structure. In the experiments, ten participants were asked to perform the evaluation on the proposed system. The results show that our system is more efficient than traditional interactive manner and verify the feasibility and practicability in the sterile operating room.

Insight into Vergence-Accommodation Mismatch

Prolonged use of conventional stereo displays causes viewer discomfort and fatigue because of the vergence-accommodation conflict. We used a novel volumetric display to examine how viewing distance, the sign of the vergence-accommodation conflict, and the temporal properties of the conflict affect discomfort and fatigue. In the first experiment, we presented a fixed conflict at short, medium, and long viewing distances. We compared subjects' symptoms in that condition and one in which there was no conflict. We observed more discomfort and fatigue with a given vergence-accommodation conflict at the longer distances. The second experiment compared symptoms when the conflict had one sign compared to when it had the opposite sign at short, medium, and long distances. We observed greater symptoms with uncrossed disparities at long distances and with crossed disparities at short distances. The third experiment compared symptoms when the conflict changed rapidly as opposed to slowly. We observed more serious symptoms when the conflict changed rapidly. These findings help define comfortable viewing conditions for stereo displays.

A novel stereoscopic projection display system for CT images of fractures

The present study proposed a novel projection display system based on a virtual reality enhancement environment. The proposed system displays stereoscopic images of fractures and enhances the computed tomography (CT) images. The diagnosis and treatment of fractures primarily depend on the post-processing of CT images. However, two-dimensional (2D) images do not show overlapping structures in fractures since they are displayed without visual depth and these structures are too small to be simultaneously observed by a group of clinicians. Stereoscopic displays may solve this problem and allow clinicians to obtain more information from CT images. Hardware with which to generate stereoscopic images was designed. This system utilized the conventional equipment found in meeting rooms. The off-axis algorithm was adopted to convert the CT images into stereo image pairs, which were used as the input for a stereo generator. The final stereoscopic images were displayed using a projection system. Several CT fracture images were imported into the system for comparison with traditional 2D CT images. The results showed that the proposed system aids clinicians in group discussions by producing large stereoscopic images. The results demonstrated that the enhanced stereoscopic CT images generated by the system appear clearer and smoother, such that the sizes, displacement and shapes of bone fragments are easier to assess. Certain fractures that were previously not visible on 2D CT images due to vision overlap became vividly evident in the stereo images. The proposed projection display system efficiently, economically and accurately displayed three-dimensional (3D) CT images. The system may help clinicians improve the diagnosis and treatment of fractures.

Stereo display with time-multiplexed focal adjustment

In stereo displays, binocular disparity creates a striking impression of depth. However, such displays present focus cues-blur and accommodation-that specify a different depth than disparity, thereby causing a conflict. This conflict causes several problems including misperception of the 3D layout, difficulty fusing binocular images, and visual fatigue. To address these problems, we developed a display that preserves the advantages of conventional stereo displays, while presenting correct or nearly correct focus cues. In our new stereo display each eye views a display through a lens that switches between four focal distances at very high rate. The switches are synchronized to the display, so focal distance and the distance being simulated on the display are consistent or nearly consistent with one another. Focus cues for points in-between the four focal planes are simulated by using a depth-weighted blending technique. We will describe the design of the new display, discuss the retinal images it forms under various conditions, and describe an experiment that illustrates the effectiveness of the display in maximizing visual performance while minimizing visual fatigue.