Computational methods to study the behaviour of social insects

Abstract

In this thesis, I describe the development, benchmarking, and application of an interconnected pipeline of computer vision methods to study the behaviour of eusocial insects - specifically herbivorous leaf-cutter ants. With the popularisation of deep learning-driven inference tools in ecology, morphology, locomotion, and neuroscience, the need for large and high-quality training data arises. Here, I thus explore one possible avenue to tackle the unifying “data-hunger” through the use of synthetic data, generated from near-photorealistic 3D models of our target species. First, I describe the hard- and software implementation of a novel open-source photogrammetry platform to digitise arthropods. Second, I illustrate the procedural synthetic data generation process and demonstrate its applicability in a variety of computer vision applications, such as detection, tracking, pose-estimation, and semantic- and instance segmentation. Third, I describe how mixed datasets of real and synthetic samples can be used to inform the training of reference-free size estimation approaches using deep convolutional neural networks which perform at approximately human-level accuracy. Finally, I demonstrate the versatility of the developed methods in a small pilot study to extract behavioural patterns of over 50,000 freely moving leaf-cutter ants across 80 hours of video recordings. While, in the presented work, the focus lies predominantly on leaf-cutter ants, the developed toolset is ultimately species-agnostic. This work thus constitutes a significant contribution towards the democratisation of deep learning-driven inference for animal behaviour research.