Skip to content
21 Sep

Multi-Scale Multi-Modal Retinal Image Fusion: From Neurons to Vessels

Allocated Projects

Project allocated in 2016-2017

Adaptive Optics (AO) is an optical imaging method that enables the observation of individual photoreceptor cells in the eye. With such micrometre level observations, clinicians can evaluate the condition of the retina of an individual with extreme precision. By longitudinally assessing the state of the photoreceptors, it is possible to identify diseases early, develop more sensitive and robust outcome measures for clinical trials, and potentially optimise intervention to preserve sight.

Despite these impressive advantages, the technology has significant limitations arising from its impressive magnification. Thus, this project has the following goals:
Expand the field of view: As any keyhole observation with an extreme magnification, AO has a very limited field of view. Random motions of the eye and fixation of gaze at different targets causes rapid motions and create images that cannot be easily registered to one another. Hence, the development of mosaicking algorithms that can montage the entire layer of photoreceptors of the human retina would be a valuable diagnostic tool.
Assist in relocalisation: Longitudinal studies require that the same regions of the retina be observed over regular time intervals. There exists no consistent way, however, to understand whether the exact same location is observed at every repeat visit. As a result, software technologies that can identify the currently observed photoreceptor patch within the mosaic previously acquired would greatly assist in disease tracking and longitudinal monitoring in both observational and interventional studies.
Co-register multi-modal images: To allow for the best retinal structural assessments, the entire cell map of the human retina would be registered with complementary modalities, including fundus images and angiography. Using the retinal vessels as landmarks, a multi-scale multi-modal image of the human retina can be acquired comprising information from the cellular level up to the higher spatial components of the retinal vessels.

The field of the project is Ophthalmology. Its methodological theme is the analysis and enhancement of multi-scale multi-modal retinal images that provide information from the micrometre to the centimetre range.

The computation improvement of the capabilities of Adaptive Optics (AO) will have significant implications to the accurate and timely detection of disease in humans, as well as the tracking of the state of their sight. Adaptive Optics (AO) retinal imaging was first demonstrated close to 20 years ago, but despite being the only in vivo cellular neuroimaging modality, it has not achieved its true full potential, predominantly due to unsustainably high personnel costs of image processing and analysis. There is a need to enable longitudinal tracking of individual photoreceptors through automated image registration and analysis software that will also significantly reduce AO imaging and interpretation personnel costs.

All these problems are amenable to software solutions that exploit the concepts of super-resolution, mosaicking etc. Additionally, there is a need to more precisely in an automated fashion be able to co-register AO photoreceptor mosaic images with complementary imaging modalities such as optical coherence tomography, fundoscopy, and angiography. Such software will allow more comprehensive retinal structural assessments, both cross-sectionally and longitudinally, and is bound to attract industrial interest as well.


Image Acquisition

Key Partners