Multimedia Technology and Telecommunications Lab

Knowledge Transfer Techniques for Semantic Segmentation

Home Page


Research Areas







The main research directions in knowledge transfer techniques for semantic segmentation we are developing regard incremental learning and unsupervised domain adaptation.

Incremental Learning for Semantic Segmentation
Incremental learning for semantic segmentation is the ability of a learning system (e.g., a neural network) to learn the segmentation and the labeling of the new classes without forgetting or deteriorating too much the performance on previously learned ones.

  • In [1] the incremental learning problem for semantic segmentation is formally introduced. To tackle this task we propose to distill the knowledge of the previous model to retain the information about previously learned classes, whilst updating the current model to learn the new ones. We propose various approaches working both on the output logits and on intermediate features. In opposition to some recent frameworks, we do not store any image from previously learned classes and only the last model is needed to preserve high accuracy on these classes.


Unsupervised Domain Adapatation for Semantic Segmentation
Unsupervised domain adapatation for semantic segmentation is the task of aligning a network trained on source data to perform well on target data. Complex deep neural networks for this task require to be trained with a huge amount of labeled data, which is difficult and expensive to acquire. A recently proposed workaround is the usage of synthetic data, however the differences between real world and synthetic scenes limit the performance.
  • In [2] a novel unsupervised domain adaptation strategy is proposed to adapt a synthetic supervised training to real world data. The proposed learning strategy exploits three components: a standard supervised learning on synthetic data, an adversarial learning strategy able to exploit both labeled synthetic data and unlabeled real data and finally a self-teaching strategy working on unlabeled data only. The last component is guided by the segmentation confidence, estimated by the fully convolutional discriminator of the adversarial learning module, helping to further reduce the domain shift between synthetic and real data. Furthermore we weighted this loss on the basis of the class frequencies to enhance the performance on less common classes.

Related Papers:

[1]   U. Michieli and P. Zanuttigh, Incremental Learning Techniques for Semantic Segmentation, Proceedings of the International Conference on Computer Vision (ICCV), Workshop on Transferring and Adapting Source Knowledge in Computer Vision (TASK-CV), 2019.

[2]   M. Biasetton, U. Michieli, G. Agresti and P. Zanuttigh, Unsupervised Domain Adaptation for Semantic Segmentation of Urban Scenes, Proceedings of the International Conference on Computer Vision and Pattern Recognition (CVPR), Workshop on Autonomous Driving (WAD), 2019.