Post Transmission Digital Video Enhancement for People with Visual Impairments

The Effect of Zoom Magnification and Large Display on Video Comprehension in Individuals With Central Vision Loss

Purpose

A larger display at the same viewing distance provides relative-size magnification for individuals with central vision loss (CVL). However, the resulting large visible area of the display is expected to result in more head rotation, which may cause discomfort. We created a zoom magnification technique that placed the center of interest (COI) in the center of the display to reduce the need for head rotation.

Methods

In a 2 × 2 within-subject study design, 23 participants with CVL viewed video clips from 1.5 m (4.9 feet) shown with or without zoom magnification, and with a large (208 cm/82" diagonal, 69°) or a typical (84 cm/33", 31°) screen. Head position was tracked and a custom questionnaire was used to measure discomfort.

Results

Video comprehension was better with the large screen (P < 0.001) and slightly worse with zoom magnification (P = 0.03). Oddly, head movements did not vary with screen size (P = 0.63), yet were greater with zoom magnification (P = 0.001). This finding was unexpected, because the COI remains in the center with zoom magnification, but moves widely with a large screen and no magnification.

Conclusions

This initial attempt to implement the zoom magnification method had flaws that may have decreased its effectiveness. In the future, we propose alternative implementations for zoom magnification, such as variable magnification.

Translational Relevance

We present the first explicit demonstration that relative-size magnification improves the video comprehension of people with CVL when viewing video.

An implementation of Bubble Magnification did not improve the video comprehension of individuals with central vision loss

PURPOSE

People with central vision loss (CVL) watch television, videos and movies, but often report difficulty and have reduced video comprehension. An approach to assist viewing videos is electronic magnification of the video itself, such as Bubble Magnification.

METHODS

We created a Bubble Magnification technique that displayed a magnified segment around the centre of interest (COI) as determined by the gaze of participants with normal vision. The 15 participants with CVL viewed video clips shown with 2× and 3× Bubble Magnification, and unedited. We measured video comprehension and gaze coherence.

RESULTS

Video comprehension was significantly worse with both 2× (p = 0.01) and 3× Bubble Magnification (p < 0.001) than the unedited video. There was no difference in gaze coherence across conditions (p ≥ 0.58). This was unexpected because we expected a benefit in both video comprehension and gaze coherence. This initial attempt to implement the Bubble Magnification method had flaws that probably reduced its effectiveness.

CONCLUSIONS

In the future, we propose alternative implementations of Bubble Magnification, such as variable magnification and bubble size. This study is a first step in the development of an intelligent-magnification approach to providing a vision rehabilitation aid to assist people with CVL.

An augmented-reality edge enhancement application for Google Glass

PURPOSE

Google Glass provides a platform that can be easily extended to include a vision enhancement tool. We have implemented an augmented vision system on Glass, which overlays enhanced edge information over the wearer's real-world view, to provide contrast-improved central vision to the Glass wearers. The enhanced central vision can be naturally integrated with scanning.

METHODS

Google Glass' camera lens distortions were corrected by using an image warping. Because the camera and virtual display are horizontally separated by 16 mm, and the camera aiming and virtual display projection angle are off by 10°, the warped camera image had to go through a series of three-dimensional transformations to minimize parallax errors before the final projection to the Glass' see-through virtual display. All image processes were implemented to achieve near real-time performance. The impacts of the contrast enhancements were measured for three normal-vision subjects, with and without a diffuser film to simulate vision loss.

RESULTS

For all three subjects, significantly improved contrast sensitivity was achieved when the subjects used the edge enhancements with a diffuser film. The performance boost is limited by the Glass camera's performance. The authors assume that this accounts for why performance improvements were observed only with the diffuser filter condition (simulating low vision).

CONCLUSIONS

Improvements were measured with simulated visual impairments. With the benefit of see-through augmented reality edge enhancement, natural visual scanning process is possible and suggests that the device may provide better visual function in a cosmetically and ergonomically attractive format for patients with macular degeneration.

Visual search with image modification in age-related macular degeneration

PURPOSE

AMD results in loss of central vision and a dependence on low-resolution peripheral vision. While many image enhancement techniques have been proposed, there is a lack of quantitative comparison of the effectiveness of enhancement. We developed a natural visual search task that uses patients' eye movements as a quantitative and functional measure of the efficacy of image modification.

METHODS

Eye movements of 17 patients (mean age = 77 years) with AMD were recorded while they searched for target objects in natural images. Eight different image modification methods were implemented and included manipulations of local image or edge contrast, color, and crowding. In a subsequent task, patients ranked their preference of the image modifications.

RESULTS

Within individual participants, there was no significant difference in search duration or accuracy across eight different image manipulations. When data were collapsed across all image modifications, a multivariate model identified six significant predictors for normalized search duration including scotoma size and acuity, as well as interactions among scotoma size, age, acuity, and contrast (P < 0.05). Additionally, an analysis of image statistics showed no correlation with search performance across all image modifications. Rank ordering of enhancement methods based on participants' preference revealed a trend that participants preferred the least modified images (P < 0.05).

CONCLUSIONS

There was no quantitative effect of image modification on search performance. A better understanding of low- and high-level components of visual search in natural scenes is necessary to improve future attempts at image enhancement for low vision patients. Different search tasks may require alternative image modifications to improve patient functioning and performance.

Effects of contour enhancement on low-vision preference and visual search

PURPOSE

To determine whether image enhancement improves visual search performance and whether enhanced images were also preferred by subjects with vision impairment.

METHODS

Subjects (n = 24) with vision impairment (vision: 20/52 to 20/240) completed visual search and preference tasks for 150 static images that were enhanced to increase object contours' visual saliency. Subjects were divided into two groups and were shown three enhancement levels. Original and medium enhancements were shown to both groups. High enhancement was shown to group 1, and low enhancement was shown to group 2. For search, subjects pointed to an object that matched a search target displayed at the top left of the screen. An "integrated search performance" measure (area under the curve of cumulative correct response rate over search time) quantified performance. For preference, subjects indicated the preferred side when viewing the same image with different enhancement levels on side-by-side high-definition televisions.

RESULTS

Contour enhancement did not improve performance in the visual search task. Group 1 subjects significantly (p < 0.001) rejected the High enhancement, and showed no preference for medium enhancement over the original images. Group 2 subjects significantly preferred (p < 0.001) both the medium and the low enhancement levels over original. Contrast sensitivity was correlated with both preference and performance; subjects with worse contrast sensitivity performed worse in the search task (ρ = 0.77, p < 0.001) and preferred more enhancement (ρ = -0.47, p = 0.02). No correlation between visual search performance and enhancement preference was found. However, a small group of subjects (n = 6) in a narrow range of mid-contrast sensitivity performed better with the enhancement, and most (n = 5) also preferred the enhancement.

CONCLUSIONS

Preferences for image enhancement can be dissociated from search performance in people with vision impairment. Further investigations are needed to study the relationships between preference and performance for a narrow range of mid-contrast sensitivity where a beneficial effect of enhancement may exist.

Development and evaluation of vision rehabilitation devices

We have developed a range of vision rehabilitation devices and techniques for people with impaired vision due to either central vision loss or severely restricted peripheral visual field. We have conducted evaluation studies with patients to test the utilities of these techniques in an effort to document their advantages as well as their limitations. Here we describe our work on a visual field expander based on a head mounted display (HMD) for tunnel vision, a vision enhancement device for central vision loss, and a frequency domain JPEG/MPEG based image enhancement technique. All the evaluation studies included visual search paradigms that are suitable for conducting indoor controllable experiments.

Visual search performance of patients with vision impairment: effect of JPEG image enhancement

PURPOSE

To measure natural image search performance in patients with central vision impairment. To evaluate the performance effect for a JPEG based image enhancement technique using the visual search task.

METHODS

One hundred and fifty JPEG images were presented on a touch screen monitor in either an enhanced or original version to 19 patients (visual acuity 0.4-1.2 logMAR, 6/15 to 6/90, 20/50 to 20/300) and seven normally sighted controls (visual acuity -0.12 to 0.1 logMAR, 6/4.5 to 6/7.5, 20/15 to 20/25). Each image fell into one of three categories: faces, indoors, and collections. The enhancement was realized by moderately boosting a mid-range spatial frequency band in the discrete cosine transform (DCT) coefficients of the image luminance component. Participants pointed to an object in a picture that matched a given target displayed at the upper-left corner of the monitor. Search performance was quantified by the percentage of correct responses, the median search time of correct responses, and an 'integrated performance' measure - the area under the curve of cumulative correct response rate over search time.

RESULTS

Patients were able to perform the search tasks but their performance was substantially worse than the controls. Search performances for the three image categories were significantly different (p <= 0.001) for all the participants, with searching for faces being the most difficult. When search time and correct response were analyzed separately, the effect of enhancement led to increase in one measure but decrease in another for many patients. Using the integrated performance, it was found that search performance declined with decrease in acuity (p = 0.005). An improvement with enhancement was found mainly for the patients whose acuity ranged from 0.4 to 0.8 logMAR (6/15 to 6/38, 20/50 to 20/125). Enhancement conferred a small but significant improvement in integrated performance for indoor and collection images (p = 0.025) in the patients.

CONCLUSION

Search performance for natural images can be measured in patients with impaired vision to evaluate the effect of image enhancement. Patients with moderate vision loss might benefit from the moderate level of enhancement used here.

IMAGE ENHANCEMENT FOR IMPAIRED VISION: THE CHALLENGE OF EVALUATION

With the aging of the population, the prevalence of eye diseases and thus of vision impairment is increasing. The TV watching habits of people with vision impairments are comparable to normally sighted people1, however their vision loss prevents them from fully benefiting from this medium. For over 20 years we have been developing video image-enhancement techniques designed to assist people with visual impairments, particularly those due to central retinal vision loss. A major difficulty in this endeavor is the lack of evaluation techniques to assess and compare the effectiveness of various enhancement methods. This paper reviews our approaches to image enhancement and the results we have obtained, with special emphasis on the difficulties encountered in the evaluation of the benefits of enhancement and the solutions we have developed to date.

Improved content aware scene retargeting for retinitis pigmentosa patients

Background

In this paper we present a novel scene retargeting technique to reduce the visual scene while maintaining the size of the key features. The algorithm is scalable to implementation onto portable devices, and thus, has potential for augmented reality systems to provide visual support for those with tunnel vision. We therefore test the efficacy of our algorithm on shrinking the visual scene into the remaining field of view for those patients.

Methods

Simple spatial compression of visual scenes makes objects appear further away. We have therefore developed an algorithm which removes low importance information, maintaining the size of the significant features. Previous approaches in this field have included seam carving, which removes low importance seams from the scene, and shrinkability which dynamically shrinks the scene according to a generated importance map. The former method causes significant artifacts and the latter is inefficient. In this work we have developed a new algorithm, combining the best aspects of both these two previous methods. In particular, our approach is to generate a shrinkability importance map using as seam based approach. We then use it to dynamically shrink the scene in similar fashion to the shrinkability method. Importantly, we have implemented it so that it can be used in real time without prior knowledge of future frames.

Results

We have evaluated and compared our algorithm to the seam carving and image shrinkability approaches from a content preservation perspective and a compression quality perspective. Also our technique has been evaluated and tested on a trial included 20 participants with simulated tunnel vision. Results show the robustness of our method at reducing scenes up to 50% with minimal distortion. We also demonstrate efficacy in its use for those with simulated tunnel vision of 22 degrees of field of view or less.

Conclusions

Our approach allows us to perform content aware video resizing in real time using only information from previous frames to avoid jitter. Also our method has a great benefit over the ordinary resizing method and even over other image retargeting methods. We show that the benefit derived from this algorithm is significant to patients with fields of view 20° or less.

Designing and testing scene enhancement algorithms for patients with retina degenerative disorders

Background

Retina degenerative disorders represent the primary cause of blindness in UK and in the developed world. In particular, Age Related Macular Degeneration (AMD) and Retina Pigmentosa (RP) diseases are of interest to this study. We have therefore created new image processing algorithms for enhancing the visual scenes for them.

Methods

In this paper we present three novel image enhancement techniques aimed at enhancing the remaining visual information for patients suffering from retina dystrophies. Currently, the only effective way to test novel technology for visual enhancement is to undergo testing on large numbers of patients. To test our techniques, we have therefore built a retinal image processing model and compared the results to data from patient testing. In particular we focus on the ability of our image processing techniques to achieve improved face detection and enhanced edge perception.

Results

Results from our model are compared to actual data obtained from testing the performance of these algorithms on 27 patients with an average visual acuity of 0.63 and an average contrast sensitivity of 1.22. Results show that Tinted Reduced Outlined Nature (TRON) and Edge Overlaying algorithms are most beneficial for dynamic scenes such as motion detection. Image Cartoonization was most beneficial for spatial feature detection such as face detection. Patient's stated that they would most like to see Cartoonized images for use in daily life.

Conclusions

Results obtained from our retinal model and from patients show that there is potential for these image processing techniques to improve visual function amongst the visually impaired community. In addition our methodology using face detection and efficiency of perceived edges in determining potential benefit derived from different image enhancement algorithms could also prove to be useful in quantitatively assessing algorithms in future studies.

Measuring perceived video quality of MPEG enhancement by people with impaired vision

We used a new method to measure the perceived quality of contrast-enhanced motion video. Patients with impaired vision (n=24) and normally sighted subjects (n=6) adjusted the level of MPEG-based enhancement of 8 videos (4 min each) drawn from 4 categories. They selected the level of enhancement that provided the preferred view of the videos, using a decreasing-step-size staircase procedure. Most patients made consistent selections of the preferred level of enhancement, indicating an appreciation of and a perceived benefit from the MPEG-based enhancement. The selections varied between patients and were correlated with letter contrast sensitivity, but the selections were not affected by training, experience, or video category. We measured just noticeable differences directly for videos and mapped the image manipulation (enhancement in our case) onto an approximately linear perceptual space. These tools and approaches will be of value in other evaluations of the image quality of motion video manipulations.