Endoscopic surveillance is a widely used method for moni-
toring abnormal changes in the gastrointestinal tract such as Barrett's
esophagus. Direct visual assessment, however, is both time consuming
and error prone, as it involves manual labelling of abnormalities on a
large set of images. To assist surveillance, this paper proposes an online
scene association scheme to summarise an endoscopic video into scenes,
on-the-y. This provides scene clustering based on visual contents, and
also facilitates topological localisation during navigation. The proposed
method is based on tracking and detection of visual landmarks on the
tissue surface. A generative model is proposed for online learning of pair-
wise geometrical relationships between landmarks. This enables robust
detection of landmarks and scene association under tissue deformation.
Detailed experimental comparison and validation have been conducted
on in vivo endoscopic videos to demonstrate the practical value of our
approach.