Introduction
Deep learning architectures exhibit a critical drop of performance due to catastrophic forgetting when they are required to incrementally learn new tasks. Contemporary incremental learning frameworks focus on image classification, while we are the first tp formally introduce the incremental/continual learning problem for semantic segmentation in which a pixel-wise labeling is considered.
[1] Incremental Learning Techniques for Semantic Segmentation
Deep learning architectures exhibit a critical drop of performance due to catastrophic forgetting when they are required to incrementally learn new tasks. Contemporary incremental learning frameworks focus on image classification and object detection while in this work we formally introduce the incremental learning problem for semantic segmentation in which a pixel-wise labeling is considered. To tackle this task we propose to distill the knowledge of the previous model to retain the information about previously learned classes, whilst updating the current model to learn the new ones. We propose various approaches working both on the output logits and on intermediate features. In opposition to some recent frameworks, we do not store any image from previously learned classes and only the last model is needed to preserve high accuracy on these classes. The experimental evaluation on the Pascal VOC2012 dataset shows the effectiveness of the proposed approaches.
The full paper can be downloaded from here
(with a couple of minor fixes w.r.t. the paper in the conference proceedings)
The method is illustrated in Figure 1.
[2] Knowledge Distillation for Incremental Learning in Semantic Segmentation
Deep learning architectures have shown remarkable results in scene understanding problems, however they exhibit a critical drop of performances when they are required to learn incrementally new tasks without forgetting old ones. This catastrophic forgetting phenomenon impacts on the deployment of artificial intelligence in real world scenarios where systems need to learn new and different representations over time. Current approaches for incremental learning deal only with image classification and object detection tasks, while in this work we formally introduce incremental learning for semantic segmentation. We tackle the problem applying various knowledge distillation techniques on the previous model. In this way, we retain the information about learned classes, whilst updating the current model to learn the new ones. We developed four main methodologies of knowledge distillation working on both output layers and internal feature representations. We do not store any image belonging to previous training stages and only the last model is used to preserve high accuracy on previously learned classes. Extensive experimental results on the Pascal VOC2012 and MSRC-v2 datasets show the effectiveness of the proposed approaches in several incremental learning scenarios.
The full paper can be downloaded from here
The method is illustrated in the Figure below.
Code
The code for the training and the evaluation of the proposed method is available on GitHub here. Furthermore, the datasets to replicate the results presented in Table 1 of the paper are available here for convenience.
Contacts
For any information on the method you can contact umberto.michieli@dei.unipd.it or
lttm@dei.unipd.it
References
[1] U. Michieli and P. Zanuttigh, "Incremental Learning Techniques for Semantic Segmentation", Proceedings of the International Conference on Computer Vision (ICCV), Workshop on Transferring and Adapting Source Knowledge in Computer Vision (TASK-CV), 2019.
[2] U. Michieli and P. Zanuttigh, "Knowledge Distillation for Incremental Learning in Semantic Segmentation", Computer Vision and Image Understanding (CVIU), Elsevier, 2021.
xhtml/css website layout by Ben Goldman - http://realalibi.com