Here you can find the code used for the training and experimental evaluation of the approach described in the paper "Continual Coarse-to-Fine Domain Adaptation in Semantic Segmentation".

Abstract

Deep neural networks are typically trained in a single shot for a specific task and data distribution but in real world settings both the task and the domain of application can change. The problem becomes even more challenging in dense predictive tasks, such as semantic segmentation and as such most approaches tackle the two problems separately. In this paper we introduce the novel task of coarse-to-fine learning of semantic segmentation architectures in presence of domain shift. We consider subsequent learning stages progressively refining the task at the semantic level; i.e., the finer set of semantic labels at each learning step is hierarchically derived from the coarser set of the previous step. We propose a new approach (CCDA) which combines a maximum squares minimization target for unsupervised domain adaptation, coarse-to-fine knowledge distillation and a new coarse-to-fine weight initialization rule. First, we employ the maximum squares loss to align source and target domains and, at the same time, to balance the gradients between well-classified and harder samples. Second, we introduce a novel coarse-to-fine knowledge distillation constraint to transfer network capabilities acquired on a coarser set of labels to a set of finer labels. Finally, we design a coarse-to-fine weight initialization rule to spread the weights from each coarse class to the respective finer classes. To evaluate our approach we design two benchmarks where source knowledge is extracted from the GTA5 dataset and it is transferred to either the Cityscapes or the IDD datasets and we show how it outperforms the main competitors.

Paper

The full paper can be downloaded from here.

Code

The code for the training and the evaluation of the proposed method will be made available here.

(Zip)

Method

The overall problem formulation and the proposed architecture are illustrated below.

 

 

Contacts

For any information you can contact lttm@dei.unipd.it

Have a look at our website http://lttm.dei.unipd.it for other works on this topic.

 

 

References

[1]   D. Shenaj, F. Barbato, U. Michieli and P. Zanuttigh, "Continual Coarse-to-Fine Domain Adaptation in Semantic Segmentation", Elsevier Image and Vision Computing, 2022

 

xhtml/css website layout by Ben Goldman  - http://realalibi.com