Here you can find the code used for the training and experimental evaluation of the approach described in the paper "Unsupervised Domain Adaptation for Mobile Semantic Segmentation based on Cycle Consistency and Feature Alignment".

Abstract

The supervised training of deep networks for semantic segmentation requires a huge amount of labeled real world data. To solve this issue, a commonly exploited workaround is to use synthetic data for training. However, deep networks show a critical performance drop when evaluated on data with slightly different statistical properties with respect to the training set. In this work, we propose a novel Unsupervised Domain Adaptation (UDA) strategy to address the domain shift issue between real world and synthetic representations. An adversarial model, based on the cycle consistency framework, performs the mapping between the synthetic and real domain. The data is then fed to a MobileNet-v2 architecture that performs the semantic segmentation task. An additional couple of discriminators, working at the feature level of the MobileNet-v2, allows to better align the features of the two domain distributions and to further improve the performance. Finally, the consistency of the semantic maps is exploited. After an initial supervised training on synthetic data, the whole UDA architecture is trained end-to-end considering all its components at once. Experimental results show how the proposed strategy is able to obtain impressive performances in adapting a segmentation network trained on synthetic data to real world scenarios. The usage of the lightweight MobileNet-v2 architecture allows its deployment on devices with limited computational resources as the ones employed in autonomous vehicles.

 

The full paper can be downloaded from here

 

 

Code

The code for the training and the evaluation of the proposed method is available here.

Method

The method is illustrated in Figure 1.

 

Results

The main quantitative and qualitative results are reported in the following.

 

Contacts

For any information on the method you can contact lttm@dei.unipd.it

Have a look at our website http://lttm.dei.unipd.it for other works on this topic.

 

 

References

[1]   M. Toldo, U. Michieli, G. Agresti, P. Zanuttigh, "Unsupervised Domain Adaptation for Mobile Semantic Segmentation based on Cycle Consistency and Feature Alignment", Accepted for publication in Image and Vision Computing

 

xhtml/css website layout by Ben Goldman  - http://realalibi.com