Autonomous Vehicles (AVs) hold the potential to revolutionise mobility systems around the world. The successful deployment of AVs requires improving the safety of AV decision making methods. A decision making method that has shown promising results is Reinforcement Learning (RL). Nevertheless, risk awareness and robustness are still open challenges in RL-based methods. This study focuses on these challenges.
We propose a novel framework for training risk-aware AV policies using RL and bespoke risk models. The framework increases post-hoc explainability, which is crucial for certifying the safety of AVs. Our first method augments the state vector with a collision prediction model based on Gaussian Processes. Experiments with a simulator and state-of-the-art RL algorithms show: risk awareness decreases collision rates by 15% and improves robustness in harsh braking situations, our collision prediction model surpasses other collision models, and our RL method outperforms the standard model-based Intelligent Driver Model.
In the future AVs will navigate complex multi-agent interactions with other AVs. Therefore, our second method leverages Multi-agent RL (MARL) with a reward function composed of a bespoke risk index. We study collaborative manoeuvring among AVs in the presence of an emergency vehicle, and evaluate in a simulator five cases: comparison to model-based baselines, trade-off between risk and efficiency, mixed human and autonomous traffic, scability, trade-off between collaboration and competitiveness. The results indicate autonomous traffic can increase the speed of the emergency vehicle by up to 34% while reducing overall collision rates by 45%.
Finally, we propose a Sim-to-real method for transferring policies trained on a simulator with MARL to the real world. The method reduces the reality gap between simulation and reality by 90%, by leveraging Domain Randomisation on a set of parameters. Furthermore, we deploy the trained policies on a fleet of miniature AVs.