# Learning with Style: Continual Semantic Segmentation Across Tasks and Domains
 
 </div>
 
This is the official PyTorch implementation of our work: "Learning with Style: Continual Semantic Segmentation Across Tasks and Domains"

In this paper we propose to address a general continual learning setting for semantic segmentation, taking into account the semantic shift within both input and label spaces.
We resort to style transfer techniques to extend knowledge across domains and a robust distillation framework to recollect task knowledge under incremental domain shift. 
The devised framework (LwS, Learning with Style) generalizes incrementally acquired task knowledge across all the domains encountered mitigating catastrophic forgetting. 
Extensive experimental evaluation shows how our framework outperforms existing methods, ill-equipped to deal with continual semantic segmentation under both task and domain shift.


## Requirements
This repository uses the following libraries:
- python (3.9.7)
- pytorch (1.10.2)
- torchvision (0.11.3)
- torchsummary (1.5.1)
- tensorboardX (2.5)
- tqdm (4.64.0)
- imageio (2.9.0)
- numpy (1.21.5)
- pillow (8.4.0)
- wandb (0.12.1)
- pandas (1.4.0)

## Setup

Download the following datasets and place each of them inside the `datasets/{dataset_name}` folder.

- [**Cityscapes**](https://www.cityscapes-dataset.com/)
- [**BDD100K**](https://bdd-data.berkeley.edu/)
- [**IDD**](https://idd.insaan.iiit.ac.in/)
- [**Mapillary Vistas**](https://www.mapillary.com/dataset/vistas)
- [**Shift**](https://www.vis.xyz/pub/shift/)

Configuration files for data incremental splitting of Mapillary and Shift are already provided in `datasets`'s subfolders.


## Scripts

To run experiments execute:

> python main.py .. args ..

Most significative args:

- `lambda_distil`: weight of KD loss on new domain
- `lambda_ce_style`: weight of CE loss on old domains
- `lambda_kd_style`: weight of KD loss on old domains
- `incr_data_split`: incremental domain setup
- `incr_class_split`: incremental class setup

To test previously trained models execute:

> python test.py .. args ..

Command examples can be found inside the `scripts` folder.
