Here you can find the datasets used for the training and experimental evaluation of the approach described in the CVPR 2019 paper "Unsupervised Domain Adaptation for ToF Data Denoising with Adversarial Learning".

Abstract

Time-of-Flight data is typically affected by a high level of noise and by artifacts due to Multi-Path Interference (MPI). While various traditional approaches for ToF data improvement have been proposed, machine learning techniques have seldom been applied to this task, mostly due to the limited availability of real world training data with depth ground truth. In this paper, we avoid to rely on labeled real data in the learning framework. A Coarse-Fine CNN, able to exploit multi-frequency ToF data for MPI correction, is trained on synthetic data with ground truth in a supervised way. In parallel, an adversarial learning strategy for regression based on the Generative Adversarial Networks (GAN) framework is used to perform an unsupervised pixel-level domain adaptation from synthetic to real world data, exploiting unlabeled real world acquisitions. Experimental results demonstrate that the proposed approach is able to effectively denoise real world data and to outperform state-of-the-art techniques.

 

The full paper can be downloaded from here

The additional material can be downloaded from here

 

 

Time-of-Flight Datasets

For the training and evaluation of the proposed ToF denoising method, we used five datasets, called "S1", "S2", "S3", "S4" and "S5" in the paper.

The real world datasets "S2", "S3" and "S5" have been introduced in this work [1]. They have been used for the training and the evaluation of the proposed Domain Adapted CNN for ToF data denoising.

The synthetic dataset "S1" and the real world one "S4" have instead been introduced in [2] and can be downloaded from  here .

Table 1 summarizes the various datasets exploited in the paper and provides the links to download them.

Dataset Type GT # scenes Used for
S1 Synthetic Yes 40 Supervised training
S2 Real No 97 Adversarial training
S3 Real Yes 8 Validation
S4 Real Yes 8 Testing
S5 Real Yes 8 Testing
There is a zipped archive for each dataset:
  • the archive "S2.zip" contains the depth, amplitude and intensity maps (".mat" files) captured with the ToF camera at 10, 20, 30, 40, 50 and 60 MHz on the 97 scenes contained in the " S2" real dataset. The ToF data are recorded in an office environment with uncontrolled light condition. This dataset was used for the unsupervised training of the proposed Domain Adapted CNN for ToF data denoising.
    • Notice that no ground truth information is provided for this dataset (it is used for unsupervised adaptation).
    Scene 9 Scene 12 Scene 14 Scene 31
      Depth map [m] img img img img
      Amplitude img img img img
  • the archize "S3.zip" containg the depth, amplitude and intensity maps (".mat" files) captured with the ToF camera at 10, 20, 30, 40, 50 and 60 MHz on the 8 scenes contained in the S3 real validation set. The ToF data are recorded in a laboratory with no external illumination. This dataset is provided with the depth ground truth and it is used for validation purposes in the training process.
    • Scene 0 Scene 1 Scene 3 Scene 4
      Depth map [m] img img img img
      Depth ground
    truth [m]    		 img img img img
      Amplitude img img img img
  • the archive "S5.zip" contains the depth, amplitude and intensity maps (".mat" files) captured with the ToF camera at 10, 20, 30, 40, 50 and 60 MHz on the 8 scenes contained in the S5 real test set, the "Box dataset". The ToF data are recorded in a laboratory with no external illumination. This dataset is provided with the depth ground truth and it is used for the testing of the proposed Domain Adapted CNN for ToF data denoising.
    • Scene 0 Scene 1 Scene 2 Scene 3
      Depth map [m] img img img img
      Depth ground
    truth [m]    		 img img img img
      Amplitude img img img img

If you use these datasets please cite the works [1] and [2].

At the address http://lttm.dei.unipd.it/nuovo/datasets.html you can find other ToF and stereo datasets from our research group.

For any information on the data you can contact lttm@dei.unipd.it

 

Contacts

For any information you can write to gianluca.agresti@dei.unipd.it.
Have a look at our website http://lttm.dei.unipd.it for other works and datasets on this topic.

 

References

[1]   G. Agresti, H. Schaefer, P. Sartor, P. Zanuttigh, "Unsupervised Domain Adaptation for ToF Data Denoising with Adversarial Learning", International Conference on Computer Vision and Pattern Recognition (CVPR), 2019.

[2]   G. Agresti and P. Zanuttigh, Deep learning for multi-path error removal in ToF sensors, Geometry Meets Deep Learning, ECCVW18, Munich, Germany, 2018.

 

xhtml/css website layout by Ben Goldman  - http://realalibi.com