Abstract
This paper proposes a joint color and depth segmentation scheme exploiting together geometrical clues and a learning stage. The approach starts from an initial over-segmentation based on spectral clustering. The input data is also fed to a Convolutional Neural Network (CNN) thus producing a per-pixel descriptor vector for each scene sample. An iterative merging procedure is then used to recombine the segments into the regions corresponding to the various objects and surfaces. The proposed algorithm starts by considering all the adjacent segments and computing a similarity metric according to the CNN features. The couples of segments with higher similarity are considered for merging. Finally the algorithm uses a NURBS surface fitting scheme on the segments in order to understand if the selected couples correspond to a single surface. The comparison with state-of-the-art methods shows how the proposed method provides an accurate and reliable scene segmentation.
The full paper can be downloaded from here
Videos
This section contains some videos showing the sequence of merging steps on 3 different scenes from the NYUDv2 dataset.
Scene 450
Scene 846
Scene 1110
Contacts
For any information you can write to lttm@dei.unipd.it . Have a look at our website http://lttm.dei.unipd.it for other works and datasets on this topic.
References
[1] L. Minto, G. Pagnutti, P. Zanuttigh, "Scene Segmentation Driven by Deep Learning and Surface Fitting", accepted for publication at Geometry Meets Deep Learning Workshop (ECCV 2016 workshop)
[2] G.Pagnutti, P. Zanuttigh, "Joint color and depth segmentation based on region merging and surface fitting", International Conference on Computer Vision Theory and Applications (VISAPP), Rome, Italy, 2016
xhtml/css website layout by Ben Goldman - http://realalibi.com