Generative methods for scene association with 2D pairwise constraints

Abstract

This thesis is concerned with the task of efficiently recognising the particular instance of a scene depicted in a query image, with applications in robot navigation including loop closure, global localisation and topological navigation. Three novel frameworks are proposed, each based on learning scene models by tracking local features to form sets of landmarks. Recognition then proceeds by considering 2D constraints between pairs of local feature correspondences to efficiently approximate global scene geometry.

First, the inter-image and intra-image pairwise geometries are considered to reduce feature correspondences to a more succinct set for a RANSAC-based 3D geometry constraint. A Hough-transform voting scheme based on inter-image correspondences weakly prunes the set of correspondences, after which intra-image geometries constrain the relative image positions of correspondences to eliminate unrealistic configurations. This idea is first proposed in an image retrieval application, and then extended to scene recognition where relative landmark positions are learned explicitly per scene.

Second, a method is introduced to embed 2D pairwise geometry directly in an inverted index, to allow for fast scene recognition without 3D estimations. A set of discrete geometric words are extracted for a query image, and passed through the index to find examples of such pairwise configurations in the database. A global geometry constraint is then proposed by considering a maximum-clique approach to an adjacency matrix of correspondences.

Third, a global topological localisation system is investigated which learns a naive Bayesian network for each landmark, to efficiently approximate global geometry without a fully-connected model. Long-term robot navigation is then addressed by learning scene models in an incremental manner, and updating the dynamic properties of landmarks accordingly. Experiments were performed on a new challenging dataset obtained by manually walking along a 7km path in a park and urban district, to capture long-term effects over an 8 month period.