Towards bounding-box free panoptic segmentation
File(s)
Working paper
Author(s)
Bonde, Ujwal
Alcantarilla, Pablo F
Leutenegger, Stefan
Type
Working Paper
Abstract
In this work we introduce a new bounding-box free network (BBFNet) for
panoptic segmentation. Panoptic segmentation is an ideal problem for a
bounding-box free approach as it already requires per-pixel semantic class
labels. We use this observation to exploit class boundaries from an
off-the-shelf semantic segmentation network and refine them to predict instance
labels. Towards this goal BBFNet predicts coarse watershed levels and use it to
detect large instance candidates where boundaries are well defined. For smaller
instances, whose boundaries are less reliable, BBFNet also predicts instance
centers by means of Hough voting followed by mean-shift to reliably detect
small objects. A novel triplet loss network helps merging fragmented instances
while refining boundary pixels. Our approach is distinct from previous works in
panoptic segmentation that rely on a combination of a semantic segmentation
network with a computationally costly instance segmentation network based on
bounding boxes, such as Mask R-CNN, to guide the prediction of instance labels
using a Mixture-of-Expert (MoE) approach. We benchmark our non-MoE method on
Cityscapes and Microsoft COCO datasets and show competitive performance with
other MoE based approaches while outperfroming exisiting non-proposal based
approaches. We achieve this while been computationally more efficient in terms
of number of parameters and FLOPs. Video results are provided here
https://blog.slamcore.com/reducing-the-cost-of-understanding.
panoptic segmentation. Panoptic segmentation is an ideal problem for a
bounding-box free approach as it already requires per-pixel semantic class
labels. We use this observation to exploit class boundaries from an
off-the-shelf semantic segmentation network and refine them to predict instance
labels. Towards this goal BBFNet predicts coarse watershed levels and use it to
detect large instance candidates where boundaries are well defined. For smaller
instances, whose boundaries are less reliable, BBFNet also predicts instance
centers by means of Hough voting followed by mean-shift to reliably detect
small objects. A novel triplet loss network helps merging fragmented instances
while refining boundary pixels. Our approach is distinct from previous works in
panoptic segmentation that rely on a combination of a semantic segmentation
network with a computationally costly instance segmentation network based on
bounding boxes, such as Mask R-CNN, to guide the prediction of instance labels
using a Mixture-of-Expert (MoE) approach. We benchmark our non-MoE method on
Cityscapes and Microsoft COCO datasets and show competitive performance with
other MoE based approaches while outperfroming exisiting non-proposal based
approaches. We achieve this while been computationally more efficient in terms
of number of parameters and FLOPs. Video results are provided here
https://blog.slamcore.com/reducing-the-cost-of-understanding.
Date Issued
2020-02-19
Citation
2020
Publisher
arXiv
Copyright Statement
© 2020 The Author(s).
Sponsor
SLAMcore Ltd
Identifier
http://arxiv.org/abs/2002.07705v2
Grant Number
n/a
Subjects
cs.CV
cs.CV
cs.RO
Notes
13 pages, 6 figures
Publication Status
Published