A gaze-based semi-autonomous wheelchair was developed to control mobility platforms by
decoding how the user looks at the environment to understand where they want to navigate
their mobility device. However, many natural eye movements are not relevant for action
intention decoding; only some are, which places a challenge on decoding, the so-called ‘Midas
touch’ Problem.
Herein, a new solution is presented, consisting of 1. deep computer vision to understand what
object a user is looking at in their field of view, with 2. an analysis of where on the object’s
bounding box the user is looking, to 3. use a simple machine learning classifier to determine
whether the overt visual attention on the object is predictive of a navigation intention to that
object.
This decoding system ultimately determines whether the user wants to drive to, e.g., a TV or
just looks at it. Crucially, we find that when users look at an object and imagine they were to
interact with it, the resulting eye movements from this motor imagery (akin to neural interfaces)
remain decodable.
Once a driving intention and thus also the location is detected, our system instructs our
autonomous wheelchair platform, the A.Eye-Drive, to navigate to the desired object while
avoiding static and moving obstacles.
Thus, we have realised a cognitive-level human interface for navigation purposes, as it requires
the user only to cognitively interact with the desired goal, not to continuously steer their
wheelchair to the target (low-level human interfacing).