This software provides depth upsampling results based on the algorithm, called weighted mode filtering, proposed in [1]. Please refer to 'DepthEnhancement_Support.cpp' for more details. We assume that input depth maps are irregularly sampled. For instance, they are first obtained from ToF sensor and then warped into color camera coordinate. Thus, a binary function (here, 'm_DepthReli') is defined for indicating reliability (1: valid depth, 0: empty).
Note that this software does not have the part for temporal consistency (using optical flow vectors). Also, the installation of 'OpenCV' is required to use this software. The multiscale color measure (MCM), where a color distance is computed in the multiscale framework for preventing aliasing effects in the depth upsampling, is used, and its parameter (the number of levels) is automatically calculated using the resolution difference between input color and depth images. Please refer to [1] for more details.
[1] D. Min, J. Lu, and M. N. Do, "Depth video enhancement based on weighted mode filtering," IEEE Trans. Image Processing (TIP), Mar. 2012.
The parameters used in the code can be tuned in the 'Config_depth_enhancement.txt'. To run the code directly, please type the following: DepthEnhancement.exe Config_depth_enhancement.txt
* Parameters
2*'FilterSize'+1: the size of a window used in filtering. For instance, if 'FilterSize' is set to 2, N(p) is 5x5.
'Sigma_I' and 'Sigma_S': parameters for intensity and spatial Gaussian kernels (G_I and G_S)
'Depth relaxation': a relaxation parameter for input depth, where the relaxation is modeled with a Gaussian function G_r. Note that this value is set in a form of a bandwidth, not a sigma. For instance, if 'Depth relaxation' is set to 20 and input depth is d1, G_r has nonzero values between d1-20 and d1+20. The sigma of G_r is automatically calculated using the bandwidth 'Depth relaxation'.
* Input and output
Input: low-resolution depth sequences (e.g. 'Sub_TL0001.png')
high-resolution color sequences (e.g. 'L0001.png')
Output: high-resolution depth sequences (e.g. 'Tanks_DepthOut_0001.ppm')