Assessment and training of functional vision in rehabilitation using Virtual Reality

Update Nov. 2024



This project is under the supervision of Gera de Haan, Joost Heutink and Frans Cornelissen, centered at the Royal Dutch Visio and LEO lab at the UMCG. Royal Dutch Visio, as the main beneficiary, aims to improve the wellbeing of partially sighted or blind patients. This research is focused on the analysis of the Hemianopia patients to extract useful knowledge in this regard.

Personal Background and Interest:

Master’s study in Computer Science – Artificial Intelligence with experience in Medical Image Analysis, Pattern Recognition, Machine Learning, and Deep Learning.

Aim of the project: 

The aim of this project is to assess the accuracy, reliability, and feasibility of performing functional vision prediction and training, with a focus on standardized, simulated or real-world environments.

Current activities:

During my PhD, I developed and tested innovative AI-based methods and conducted experiments to address critical challenges in understanding and improving visual processing and quality of life for individuals with visual impairments.

Development of ACE-DNV for Gaze Event Classification

I created a novel method called ACE-DNV for the automatic classification of gaze events during dynamic natural viewing. This is the first method to perform gaze classification in real-world settings without relying on additional devices such as IMU sensors or depth cameras. Instead, it uses environmental footage and eye movement signals captured by commercially available mobile eye trackers. This breakthrough simplifies the study of visual attention and behavior, offering a robust foundation for assistive technologies.

Design of Gaze-SPV for Vision Restoration

I introduced Gaze-SPV, an AI-driven approach for taking a first step to integrating gaze information into cortical implant systems designed to restore partial vision for blind individuals. By incorporating eye movement data, which had been previously overlooked, Gaze-SPV enhances the translation of visual scenes into cortical stimuli. This method represents a step in developing more effective representation of the environment for object recognition in future vision restoration technologies.

Behavioral Validation and Framework Testing

To validate our proposed prosthetic vision representation method, I conducted a gaze-contingent behavioral experiment to assess their efficacy and practicality. This experiment provided empirical evidence on the usability and impact of these AI methods, while also introducing a versatile experimental framework that can be used in future studies to evaluate different visual encoding strategies.

Investigating Quality of Life in Glaucoma Patients

In the final phase of my PhD, I explored the relationship between the quality of life of glaucoma patients and the archetypal analysis of their visual field defect maps. This work identified meaningful patterns and correlations that could inform better glaucoma management and personalized treatment plans, aiming to improve patient outcomes.

Future directions:

The developed methods, such as ACE-DNV and Gaze-SPV, hold the potential to improve how gaze data is used to enhance assistive devices and cortical implants, improving accessibility and independence for visually impaired individuals. The behavioral frameworks and experimental insights can be expanded to refine encoding strategies and validate new applications.

My OptiVisT experience:

Working in interdisciplinary groups has been a rewarding experience. Collaborating with researchers from diverse fields has enabled me to tackle challenges through multiple perspectives. Additionally, visiting institutes across various cities and countries has improved my communication and collaboration skills, enriching both my professional and personal growth.