Improving retinal image resolution with iterative weighted shift-and-add

Retinal image registration as a tool for supporting clinical applications

BACKGROUND AND OBJECTIVE

The study of small vessels allows for the analysis and diagnosis of diseases with strong vasculopathy. This type of vessels can be observed non-invasively in the retina via fundoscopy. The analysis of these vessels can be facilitated by applications built upon Retinal Image Registration (RIR), such as mosaicing, Super Resolution (SR) or eye shape estimation. RIR is challenging due to possible changes in the retina across time, the utilization of diverse acquisition devices with varying properties, or the curved shape of the retina.

METHODS

We employ the Retinal Image Registration through Eye Modelling and Pose Estimation (REMPE) framework, which simultaneously estimates the cameras' relative poses, as well as eye shape and orientation to develop RIR applications and to study their effectiveness.

RESULTS

We assess quantitatively the suitability of the REMPE framework towards achieving SR and eye shape estimation. Additionally, we provide indicative results demonstrating qualitatively its usefulness in the context of longitudinal studies, mosaicing, and multiple image registration. Besides the improvement over registration accuracy, demonstrated via registration applications, the most important novelty presented in this work is the eye shape estimation and the generation of 3D point meshes. This has the potential for allowing clinicians to perform measurements on 3D representations of the eye, instead of doing so in 2D images that contain distortions induced because of the projection on the image space.

CONCLUSIONS

RIR is very effective in supporting applications such as SR, eye shape estimation, longitudinal studies, mosaicing and multiple image registration. Its improved registration accuracy compared to the state of the art translates directly in improved performance when supporting the aforementioned applications.

Retinal image registration based on keypoint correspondences, spherical eye modeling and camera pose estimation

In this work, an image registration method for two retinal images is proposed. The proposed method utilizes keypoint correspondences and assumes a spherical model of the eye. Image registration is treated as a pose estimation problem, which requires estimation of the rigid transformation that relates the two images. Using this estimate, one image can be warped so that it is registered to the coordinate frame of the other. Experimental evaluation shows improved accuracy over state-of-the-art approaches as well as robustness to noise and spurious keypoint correspondences. Experiments also indicate the method's applicability to diagnostic image enhancement and comparative analysis of images from different examinations.

An experimental evaluation of the accuracy of keypoints-based retinal image registration

This work regards an investigation of the accuracy of a state-of-the-art, keypoint-based retinal image registration approach, as to the type of keypoint features used to guide the registration process. The employed registration approach is a local method that incorporates the notion of a 3D retinal surface imaged from different viewpoints and has been shown, experimentally, to be more accurate than competing approaches. The correspondences obtained between SIFT, SURF, Harris-PIIFD and vessel bifurcations are studied, either individually or in combinations. The combination of SIFT features with vessel bifurcations was found to perform better than other combinations or any individual feature type, alone. The registration approach is also comparatively evaluated against representative methods of the state-of-the-art in retinal image registration, using a benchmark dataset that covers a broad range of cases regarding the overlap of the acquired images and the anatomical characteristics of the imaged retinas.

Retinal image registration through simultaneous camera pose and eye shape estimation

In this paper, a retinal image registration method is proposed. The approach utilizes keypoint correspondences and assumes that the human eye has a spherical or ellipsoidal shape. The image registration problem amounts to solving a camera 3D pose estimation problem and, simultaneously, an eye 3D shape estimation problem. The camera pose estimation problem is solved by estimating the relative pose between the views from which the images were acquired. The eye shape estimation problem parameterizes the shape and orientation of an ellipsoidal model for the eye. Experimental evaluation shows 17.91% reduction of registration error and 47.52% reduction of the error standard deviation over state of the art methods.

Enhanced sampling of 2D interference patterns

We propose a simple analysis to improve the resolution of interference patterns which consist of straight fringes. As the pattern is rotated with respect to the detector, each row or column in the camera perceives it in a slightly shifted manner. We support this proposed method by analyzing both simulated and experimental interference patterns, and verify it using an interferogram obtained from a spectrally complex light source. The results imply that this technique could be implemented in different aspects of image analysis common in many fields in physics.

Retinal image registration under the assumption of a spherical eye

We propose a method for registering a pair of retinal images. The proposed approach employs point correspondences and assumes that the human eye has a spherical shape. The image registration problem is formulated as a 3D pose estimation problem, solved by estimating the rigid transformation that relates the views from which the two images were acquired. Given this estimate, each image can be warped upon the other so that pixels with the same coordinates image the same retinal point. Extensive experimental evaluation shows improved accuracy over state of the art methods, as well as robustness to noise and spurious keypoint matches. Experiments also indicate the method's applicability to the comparative analysis of images from different examinations that may exhibit changes and its applicability to diagnostic support.

Point spread function estimation from projected speckle illumination

The resolution of an imaging apparatus is ideally limited by the diffraction properties of the light passing through the system aperture, but in many practical cases, inhomogeneities in the light propagating medium or imperfections in the optics degrade the image resolution. Here we introduce a powerful and practical new approach for estimating the point spread function (PSF) of an imaging system on the basis of PSF Estimation from Projected Speckle Illumination (PEPSI). PEPSI uses the fact that the speckles' phase randomness cancels the effects of the aberrations in the illumination path, thereby providing an objective pattern for measuring the deformation of the imaging path. Using this approach, both wide-field-of-view and local-PSF estimation can be obtained by calibration-free, single-speckle-pattern projection. Finally, we demonstrate the feasibility of using PEPSI estimates for resolution improvement in iterative maximum likelihood deconvolution.

Super resolution for fundoscopy based on 3D image registration

An approach to the generation of super-resolution (SR) images from fundoscopy images is proposed that is based on the 3D registration of the original fundoscopy images. The proposed approach utilizes a simple 3D registration method to enable the application of conventional SR techniques which, otherwise, employ 2D image registration. Qualitative and quantitative comparative evaluation shows that the obtained results improve image definition and alleviate noise.

Pre-processing, registration and selection of adaptive optics corrected retinal images

PURPOSE

In this paper, the aim is to demonstrate enhanced processing of sequences of fundus images obtained using a commercial AO flood illumination system. The purpose of the work is to (1) correct for uneven illumination at the retina (2) automatically select the best quality images and (3) precisely register the best images.

METHODS

Adaptive optics corrected retinal images are pre-processed to correct uneven illumination using different methods; subtracting or dividing by the average filtered image, homomorphic filtering and a wavelet based approach. These images are evaluated to measure the image quality using various parameters, including sharpness, variance, power spectrum kurtosis and contrast. We have carried out the registration in two stages; a coarse stage using cross-correlation followed by fine registration using two approaches; parabolic interpolation on the peak of the cross-correlation and maximum-likelihood estimation. The angle of rotation of the images is measured using a combination of peak tracking and Procrustes transformation.

RESULTS

We have found that a wavelet approach (Daubechies 4 wavelet at 6th level decomposition) provides good illumination correction with clear improvement in image sharpness and contrast. The assessment of image quality using a 'Designer metric' works well when compared to visual evaluation, although it is highly correlated with other metrics. In image registration, sub-pixel translation measured using parabolic interpolation on the peak of the cross-correlation function and maximum-likelihood estimation are found to give very similar results (RMS difference 0.047 pixels). We have confirmed that correcting rotation of the images provides a significant improvement, especially at the edges of the image. We observed that selecting the better quality frames (e.g. best 75% images) for image registration gives improved resolution, at the expense of poorer signal-to-noise. The sharpness map of the registered and de-rotated images shows increased sharpness over most of the field of view.

CONCLUSION

Adaptive optics assisted images of the cone photoreceptors can be better pre-processed using a wavelet approach. These images can be assessed for image quality using a 'Designer Metric'. Two-stage image registration including correcting for rotation significantly improves the final image contrast and sharpness.

Improving retinal imaging by corneal refractive index matching

Imaging the retina at high resolution requires a dilated pupil, which in turn exposes more corneal irregularities. We diminish the optical errors of the cornea by refractive index matching. Lens-fitted goggles were used for corneal immersion, to reduce its aberrations, while keeping the ocular power. An additional aspheric plate reduced the residual ocular spherical aberration. A comparison of the index-matching-based retinal images with those acquired directly shows resolution improvement for subjects with normal extent of ocular aberrations. A simulation of the point spread function, obtained from an averaged ocular and corneal wavefront error, also reveals substantial improvement when using corneal index matching. The demonstrated improvement using index matching may enable further improvement of current retinal imaging techniques or relaxing requirements for active ocular aberration correction.

Estimation of the ocular point spread function by retina modeling

Retinal imaging often suffers from blurring aberrations. With knowledge of the blurring point spread function (PSF), better images can be reconstructed by deconvolution techniques. We demonstrate a method to enhance the contrast of retinal cells by estimating the ocular PSF. This is done by finding the cells' positions and their intensity distributions and using these as a model for the image. The feasibility of this method is demonstrated by Wiener deconvolution both for adaptively and nonadaptively corrected images.