Intrinsic Image Decomposition With Sparse and Non-local Priors

World’s FIRST 10K Best Paper Award – Platinum Award

 

 

 

Abstract

This paper proposes an new intrinsic decomposition method that decomposing a single RGB-D image into reflectance and shading components. We observe and verify that, shading image mainly contains smooth regions separated by curves, and its gradient distribution is sparse. We therefore use L1-norm to model the direct irradiance component--the main sub-component extracted from shading component. Moreover, non-local prior weighted by a bilateral kernel on a larger neighborhood is designed to fully exploit structural correlation in the reflectance component to improve the decomposition performance. The model is solved by the alternating direction method under the augmented Lagrangian multiplier (ADM-ALM) framework. Experimental results on both synthetic and real datasets demonstrate that the proposed method yields better results and enjoys lower complexity compared with the state-of-the-art methods.

 

Keywords: Intrinsic decomposition, RGB-D, sparse, non-local

 

 




Fig. 1. Results on one image from MPI-Sintel dataset: (a) input RGB image and depth map, (b) results of [8], (c) results of [15], and (d) our results.		  

Download

 

Source code: 

Please send an email to lik@tju.edu.cn to get the source code. The source code is only for the non-commercial use.

 

Evaluation on NYU Datasets



Fig. 2. Results on one image from NYU dataset: (a) input RGB image and depth map, (b) results of [8], (c) results of [15], and (d) our results..



Fig. 3. Results on one image from NYU dataset: (a) input RGB image and depth map, (b) results of [8], (c) results of [15], and (d) our results..


 

Evaluation on MPI-Sintel Datasets


Table 1 : Quantitative evaluation on MPI-Sintel dataset.



Table 2 : Quantitative evaluation on noisy MPI-Sintel dataset.


 

Running time


Table 3 : Comparison of running times.

Publications


[1] Yujie Wang, Kun Li, Jingyu Yang, Xinchen Ye, “Intrinsic image decomposition from a single RGB-D image with sparse and non-local priors”, IEEE International Conference on Multimedia and Expo (ICME),.July 10-15, 2017, Hongkong, China. [pdf]
References

[1]Grosse, Roger, et al. "Ground truth dataset and baseline evaluations for intrinsic image algorithms." IEEE, International Conference on Computer Vision IEEE, 2010:2335-2342.
[2] Dai, Haipeng, et al. "L0 co-intrinsic images decomposition." IEEE International Conference on Multimedia and Expo IEEE, 2014:1-6.
[3] Barron, Jonathan, and J. Malik. "Intrinsic Scene Properties from a Single RGB-D Image." Computer Vision and Pattern Recognition IEEE, 2013:17-24.
[4] Barrow, H. G., and J. M. Tenenbaum. "Recovering Intrinsic Scene Characteristics from Images." Journal of Fujian Commercial College (1978).
[5] Land, Edwin H., and J. J. Mccann. "Lightness and Retinex Theory." J Opt Soc Am 61.1(1971):1-11.
[6] Zhao, Qi, et al. "A Closed-Form Solution to Retinex with Nonlocal Texture Constraints. " IEEE Transactions on Pattern Analysis & Machine Intelligence 34.7(2012):1437-1444.
[7] MLAShen, Li, and C. Yeo. "Intrinsic images decomposition using a local and global sparse representation of reflectance." Computer Vision and Pattern Recognition IEEE, 2011:697-704.
[8] Chen, Qifeng, and V. Koltun. "A Simple Model for Intrinsic Image Decomposition with Depth Cues." (2013):241-248.
[9] Shi, Jian, et al. "Efficient intrinsic image decomposition for RGBD images." ACM Symposium on Virtual Reality Software and Technology ACM, 2015:17-25.
[10] Boyd, Stephen, et al. "Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers." Foundations & Trends? in Machine Learning 3.1(2011):1-122.
[11] Weiss, Y. "Deriving intrinsic images from image sequences." Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on IEEE, 2001:68-75 vol.2.
[12] Bousseau, Adrien, and S. Paris. "User-assisted intrinsic images." Acm Siggraph Asia ACM, 2009:1-10.
[13] Ye, Genzhi, et al. "Intrinsic video and applications." Acm Transactions on Graphics 33.4(2014):80.”
[14] Sunkavalli, Kalyan, et al. "Interactive intrinsic video editing." Acm Transactions on Graphics 33.6(2014):197.
[15] Jeon, Junho, et al. Intrinsic Image Decomposition Using Structure-Texture Separation and Surface Normals. Computer Vision – ECCV 2014. 2014:218-233.
[16] Buades, Antoni, B. Coll, and J. M. Morel. "A Non-Local Algorithm for Image Denoising." IEEE Computer Society Conference on Computer Vision & Pattern Recognition IEEE Computer Society, 2005:60-65.
[17] Yang, J., et al. "Color-Guided Depth Recovery From RGB-D Data Using an Adaptive Autoregressive Model." IEEE Transactions on Image Processing 23.8(2014):3443-3458.
[18] Butler, Daniel J., et al. A Naturalistic Open Source Movie for Optical Flow Evaluation. Computer Vision – ECCV 2012. Springer Berlin Heidelberg, 2012:611-625.