The increasing deployment of autonomous systems in populated environments requires robots to possess a deep understanding of humans in their surroundings. It would be particularly desirable if this understanding could be achieved automatically through the use of vision-based sensing, and is integrated into existing robotic state estimation frameworks to leverage synergies between the two approaches. However, current robotic state estimation systems typically do not incorporate human states and their estimation into their frameworks or representation.
Although existing methods in human state estimation achieve high accuracy, they often lack computational efficiency and fail to produce a globally consistent and feasible estimation in line with the observed scene. To achieve human-robot understanding and facilitate higher-level interaction, autonomous systems need to integrate human state estimation as an integral part of scene understanding.
This thesis addresses these challenges by developing methods to estimate human states, including posture, shape, and relative position with respect to the observing robotic system, using different sensor modalities in standard SLAM frameworks. Additionally, this thesis proposes a learned human motion model to improve the accuracy of human state estimation and enable monocular, scale-aware camera tracking. Finally, a joint formulation of human and robotic state estimation is presented, demonstrating its potential to improve accuracy and performance for both sides. These contributions have the potential to enable robotic systems to understand humans in their surroundings and facilitate safe and effective human-robot interaction.