Landmark detection for fusion of fundus and MRI toward a patient-specific multimodal eye model

An Automatic Framework to Create Patient-specific Eye Models From 3D Magnetic Resonance Images for Treatment Selection in Patients With Uveal Melanoma

PURPOSE

The optimal treatment strategy for uveal melanoma (UM) relies on many factors, the most important being tumor size and location. Building on recent developments in high-resolution 3D ocular magnetic resonance imaging (MRI), we developed an automatic image-processing framework to create patient-specific eye models and to subsequently determine the full 3D tumor shape and size automatically.

METHODS AND MATERIALS

From 15 patients with UM, 3D inversion-recovery gradient-echo (T1-weighted) and 3D fat-suppressed spin-echo (T2-weighted) images were acquired with a 7T MRI scanner. First, the sclera and cornea were segmented from the T2-weighted image by mesh-fitting. The T1- and T2-weighted images were then coregistered. From the registered T1-weighted image, the lens, vitreous body, retinal detachment, and tumor were segmented. Fuzzy C-means clustering was used to differentiate the tumor from retinal detachments. The tumor model was verified and (if needed) edited by an ophthalmic MRI specialist. Subsequently, the prominence and largest basal diameter of the tumor were measured automatically based on the verified contours. These results were compared with manual assessments on the original images and with ultrasound measurements to show the errors in manual analysis.

RESULTS

The framework successfully created an eye model fully automatically in 12 cases. In these cases, a Dice similarity coefficient (mean surface distance) of 97.7%±0.84% (0.17±0.11 mm) was achieved for the sclera, 96.8%±1.05% (0.20±0.06 mm) for the vitreous body, 91.6%±4.83% (0.15±0.06 mm) for the lens, and 86.0%±7.4% (0.35±0.27 mm) for the tumor. The manual assessments deviated, on average, 0.39±0.31 mm in prominence and 1.7±1.22 mm in basal diameter from the automatic measurements.

CONCLUSIONS

The described framework combined information from T1- and T2-weighted images to accurately determine tumor boundaries in 3D. The proposed process may have a direct effect on clinical workflow, as it enables an accurate 3D assessment of tumor dimensions, which directly influences therapy selection.

Three-dimensional MRI-based treatment planning approach for non-invasive ocular proton therapy

PURPOSE

To develop a high-resolution three-dimensional (3D) magnetic resonance imaging (MRI)-based treatment planning approach for uveal melanomas (UM) in proton therapy.

MATERIALS/METHODS

For eight patients with UM, a segmentation of the gross tumor volume (GTV) and organs-at-risk (OARs) was performed on T1- and T2-weighted 7 Tesla MRI image data to reconstruct the patient MR-eye. An extended contour was defined with a 2.5-mm isotropic margin derived from the GTV. A broad beam algorithm, which we have called πDose, was implemented to calculate relative proton absorbed doses to the ipsilateral OARs. Clinically favorable gazing angles of the treated eye were assessed by calculating a global weighted-sum objective function, which set penalties for OARs and extreme gazing angles. An optimizer, which we have named OPT'im-Eye-Tool, was developed to tune the parameters of the functions for sparing critical-OARs.

RESULTS

In total, 441 gazing angles were simulated for every patient. Target coverage including margins was achieved in all the cases (V95%  > 95%). Over the whole gazing angles solutions space, maximum dose (Dmax ) to the optic nerve and the macula, and mean doses (Dmean ) to the lens, the ciliary body and the sclera were calculated. A forward optimization was applied by OPT'im-Eye-Tool in three different prioritizations: iso-weighted, optic nerve prioritized, and macula prioritized. In each, the function values were depicted in a selection tool to select the optimal gazing angle(s). For example, patient 4 had a T2 equatorial tumor. The optimization applied for the straight gazing angle resulted in objective function values of 0.46 (iso-weighted situation), 0.90 (optic nerve prioritization) and 0.08 (macula prioritization) demonstrating the impact of that angle in different clinical approaches.

CONCLUSIONS

The feasibility and suitability of a 3D MRI-based treatment planning approach have been successfully tested on a cohort of eight patients diagnosed with UM. Moreover, a gaze-angle trade-off dose optimization with respect to OARs sparing has been developed. Further validation of the whole treatment process is the next step in the goal to achieve both a non-invasive and a personalized proton therapy treatment.

Ophthalmic Magnetic Resonance Imaging: Where Are We (Heading To)?

Magnetic resonance imaging of the eye and orbit (MReye) is a cross-domain research field, combining (bio)physics, (bio)engineering, physiology, data sciences and ophthalmology. A growing number of reports document technical innovations of MReye and promote their application in preclinical research and clinical science. Realizing the progress and promises, this review outlines current trends in MReye. Examples of MReye strategies and their clinical relevance are demonstrated. Frontier applications in ocular oncology, refractive surgery, ocular muscle disorders and orbital inflammation are presented and their implications for explorations into ophthalmic diseases are provided. Substantial progress in anatomically detailed, high-spatial resolution MReye of the eye, orbit and optic nerve is demonstrated. Recent developments in MReye of ocular tumors are explored, and its value for personalized eye models derived from machine learning in the treatment planning of uveal melanoma and evaluation of retinoblastoma is highlighted. The potential of MReye for monitoring drug distribution and for improving treatment management and the assessment of individual responses is discussed. To open a window into the eye and into (patho)physiological processes that in the past have been largely inaccessible, advances in MReye at ultrahigh magnetic field strengths are discussed. A concluding section ventures a glance beyond the horizon and explores future directions of MReye across multiple scales, including in vivo electrolyte mapping of sodium and other nuclei. This review underscores the need for the (bio)medical imaging and ophthalmic communities to expand efforts to find solutions to the remaining unsolved problems and technical obstacles of MReye, with the objective to transfer methodological advancements driven by MR physics into genuine clinical value.

Automatic vocal tract landmark localization from midsagittal MRI data

The various speech sounds of a language are obtained by varying the shape and position of the articulators surrounding the vocal tract. Analyzing their variations is crucial for understanding speech production, diagnosing speech disorders and planning therapy. Identifying key anatomical landmarks of these structures on medical images is a pre-requisite for any quantitative analysis and the rising amount of data generated in the field calls for an automatic solution. The challenge lies in the high inter- and intra-speaker variability, the mutual interaction between the articulators and the moderate quality of the images. This study addresses this issue for the first time and tackles it by means of Deep Learning. It proposes a dedicated network architecture named Flat-net and its performance are evaluated and compared with eleven state-of-the-art methods from the literature. The dataset contains midsagittal anatomical Magnetic Resonance Images for 9 speakers sustaining 62 articulations with 21 annotated anatomical landmarks per image. Results show that the Flat-net approach outperforms the former methods, leading to an overall Root Mean Square Error of 3.6 pixels/0.36 cm obtained in a leave-one-out procedure over the speakers. The implementation codes are also shared publicly on GitHub.

Personalized Anatomic Eye Model From T1-Weighted Volume Interpolated Gradient Echo Magnetic Resonance Imaging of Patients With Uveal Melanoma

PURPOSE

We present a 3-dimensional patient-specific eye model from magnetic resonance imaging (MRI) for proton therapy treatment planning of uveal melanoma (UM). During MRI acquisition of UM patients, the point fixation can be difficult and, together with physiological blinking, can introduce motion artifacts in the images, thus challenging the model creation. Furthermore, the unclear boundary of the small objects (eg, lens, optic nerve) near the muscle or of the tumors with hemorrhage and tantalum clips can limit model accuracy.

METHODS AND MATERIALS

A dataset of 37 subjects, including 30 healthy eyes of volunteers and 7 eyes of UM patients, was investigated. In our previous work, active shape model was successfully applied to retinoblastoma eye segmentation in T1-weighted 3T MRI. Here, we evaluate this method in a more challenging setting, based on 1.5T MRI acquisition and different datasets of awake adult eyes with UM. The lens and cornea together with the sclera, vitreous humor, and optic nerve were automatically segmented and validated against manual delineations of a senior ocular radiation oncologist, in terms of the Dice similarity coefficient and Hausdorff distance.

RESULTS

Leave-one-out cross validation (mixing both volunteers and UM patients) yielded median Dice similarity coefficient values (respective of Hausdorff distance) of 94.5% (1.64 mm) for the sclera, 92.2% (1.73 mm) for the vitreous humor, 88.3% (1.09 mm) for the lens, and 81.9% (1.86 mm) for the optic nerve. The average computation time for an eye was 10 seconds.

CONCLUSIONS

To our knowledge, our work is the first attempt to automatically segment adult eyes, including patients with UM. Our results show that automated active shape model segmentation can succeed in the presence of motion, tumors, and tantalum clips. These results are promising for inclusion in clinical practice.

Multi-channel MRI segmentation of eye structures and tumors using patient-specific features

Retinoblastoma and uveal melanoma are fast spreading eye tumors usually diagnosed by using 2D Fundus Image Photography (Fundus) and 2D Ultrasound (US). Diagnosis and treatment planning of such diseases often require additional complementary imaging to confirm the tumor extend via 3D Magnetic Resonance Imaging (MRI). In this context, having automatic segmentations to estimate the size and the distribution of the pathological tissue would be advantageous towards tumor characterization. Until now, the alternative has been the manual delineation of eye structures, a rather time consuming and error-prone task, to be conducted in multiple MRI sequences simultaneously. This situation, and the lack of tools for accurate eye MRI analysis, reduces the interest in MRI beyond the qualitative evaluation of the optic nerve invasion and the confirmation of recurrent malignancies below calcified tumors. In this manuscript, we propose a new framework for the automatic segmentation of eye structures and ocular tumors in multi-sequence MRI. Our key contribution is the introduction of a pathological eye model from which Eye Patient-Specific Features (EPSF) can be computed. These features combine intensity and shape information of pathological tissue while embedded in healthy structures of the eye. We assess our work on a dataset of pathological patient eyes by computing the Dice Similarity Coefficient (DSC) of the sclera, the cornea, the vitreous humor, the lens and the tumor. In addition, we quantitatively show the superior performance of our pathological eye model as compared to the segmentation obtained by using a healthy model (over 4% DSC) and demonstrate the relevance of our EPSF, which improve the final segmentation regardless of the classifier employed.

Automatic Segmentation of the Eye in 3D Magnetic Resonance Imaging: A Novel Statistical Shape Model for Treatment Planning of Retinoblastoma

PURPOSE

Proper delineation of ocular anatomy in 3-dimensional (3D) imaging is a big challenge, particularly when developing treatment plans for ocular diseases. Magnetic resonance imaging (MRI) is presently used in clinical practice for diagnosis confirmation and treatment planning for treatment of retinoblastoma in infants, where it serves as a source of information, complementary to the fundus or ultrasonographic imaging. Here we present a framework to fully automatically segment the eye anatomy for MRI based on 3D active shape models (ASM), and we validate the results and present a proof of concept to automatically segment pathological eyes.

METHODS AND MATERIALS

Manual and automatic segmentation were performed in 24 images of healthy children's eyes (3.29 ± 2.15 years of age). Imaging was performed using a 3-T MRI scanner. The ASM consists of the lens, the vitreous humor, the sclera, and the cornea. The model was fitted by first automatically detecting the position of the eye center, the lens, and the optic nerve, and then aligning the model and fitting it to the patient. We validated our segmentation method by using a leave-one-out cross-validation. The segmentation results were evaluated by measuring the overlap, using the Dice similarity coefficient (DSC) and the mean distance error.

RESULTS

We obtained a DSC of 94.90 ± 2.12% for the sclera and the cornea, 94.72 ± 1.89% for the vitreous humor, and 85.16 ± 4.91% for the lens. The mean distance error was 0.26 ± 0.09 mm. The entire process took 14 seconds on average per eye.

CONCLUSION

We provide a reliable and accurate tool that enables clinicians to automatically segment the sclera, the cornea, the vitreous humor, and the lens, using MRI. We additionally present a proof of concept for fully automatically segmenting eye pathology. This tool reduces the time needed for eye shape delineation and thus can help clinicians when planning eye treatment and confirming the extent of the tumor.