Abstract
Indirect Time-of-Flight cameras (iToF) are low-cost devices that provide depth images at an interactive frame rate. However, they are affected by different error sources, with the spotlight taken by Multi-Path Interference (MPI), a key challenge for this technology. Common data-driven approaches tend to focus on a direct estimation of the output depth values, ignoring the underlying transient propagation of the light in the scene. In this work instead, we propose a very compact architecture, leveraging on the direct-global subdivision of transient information for the removal of MPI and for the reconstruction of the transient information itself. The proposed model reaches state-of-the-art MPI correction performances both on synthetic and real data and proves to be very competitive also at extreme levels of noise; at the same time, it also makes a step towards reconstructing transient information from multi-frequency iToF data.
The full paper can be downloaded from here
Transient Dataset
For the training and evaluation of the proposed ToF denoising method we made use of 4 ToF datasets, called "S1", "S3", "S4" and "S5", and of the "Walls" transient dataset, which has been introduced in this work.
The "Walls" transient dataset can be downloaded from the following links: 1wall, 2walls_1, 2walls_2, 3walls.
The synthetic dataset "S1" and the real world one "S4" have instead
been introduced in [2] and can be downloaded from here.
The real world datasets "S3" and "S5" have been introduced in [3]
and can be downloaded here
Table 1 summarizes the various datasets exploited in the paper and provides the links to download them.
| Dataset | Type | Transient GT | # scenes | Used for |
| Walls | Synthetic | Yes | 222 | Training |
| S1 | Synthetic | No | 40 | Supervised training |
| S3 | Real | No | 8 | Validation |
| S4 | Real | No | 8 | Testing |
| S5 | Real | No | 8 | Testing |
-
The archives "1wall,
2walls_1, 2walls_2,
3walls"
contain the 222 transient images simulated using the Time of Flight
Tracer.
| |
|
|
|
|
| Depth map [m] |  |
 |
 |
|
If you use this dataset please cite the work [1].
At the address http://lttm.dei.unipd.it/nuovo/datasets.html
you can find other ToF and stereo datasets from our research
group.
Code
The code used for the implementation of this work can instead be found "here"Contacts
For any information you can write to adriano.simonetto@phd.unipd.it.
Have a look at our website http://lttm.dei.unipd.it
for other works and datasets on this topic.
References
[1] A. Simonetto, G. Agresti, P. Zanuttigh, H.Schäfer "Lightweight Deep Learning Architecture for MPI Correction and Transient Reconstruction", ArXiV, 2021.
[2] G. Agresti and P. Zanuttigh, Deep learning for multi-path error removal in ToF sensors, Geometry Meets Deep Learning, ECCVW18, Munich, Germany, 2018. [3] G. Agresti, H. Schaefer, P. Sartor, P. Zanuttigh, "Unsupervised Domain Adaptation for ToF Data Denoising with Adversarial Learning", International Conference on Computer Vision and Pattern Recognition (CVPR), 2019. [4] P. Pitts, A. Benedetti, M. Slaney, P. Chou, Time of Flight Tracer", Tech. rep., Microsoft, 2014.
xhtml/css website layout by Ben Goldman - http://realalibi.com