Non-Rigid Structure from Motion via Sparse Representation

Kun Li, Jingyu Yang, and Jianmin Jiang

Abstract

This paper proposes a new approach for non-rigid structure from motion with occlusion, based on sparse representation. We address the occlusion problem based on the latest developments on sparse representation: matrix completion, which can recover the observation matrix that has high percentages of missing data and can also reduce the noises and outliers in the known elements. We introduce sparse transform to the joint estimation of 3D shapes and motions. 3D shape trajectory space is fit by wavelet basis to achieve better modeling of complex motion. Experimental results on datasets without and with occlusion show that our method can better estimate the 3D shapes and motions, compared with state-of-the-art algorithms.

Fig. 1. Framework of our method.

Downloads

Source code: Coming soon....

Experiment Results

                1. Non-Rigid SFM Results without Missing Data

Fig. 2. 3D shapes of Shark1 at t1, t40,t80, t120,t160,t200,and t240, recovered by PTA [1], CSF1 [2], and CSF2 [3], LSSM1
[4], LSSM2 [4], BMM [5] and our method (SR).

                 2. Non-Rigid SFM Results with Missing Data

Fig. 3. 3D shapes of Shark1 at t1, t40,t80, t120,t160,t200,and t240 recovered by our method with (a) 25%, (b) 50%, and (c) 75% missing data.
    

Table 1. Quantitative evaluation of non-rigid structure from motion with missing data for Shark1.

Publications

[1] I. Akhter, Y. A. Sheikh, S. Khan, and T. Kanade, “Nonrigid structure from motion in trajectory space,” in Proc. Neural Information Processing Systems, 2008.
[2] P. F. U. Gotardo and A. M. Martinez, “Computing smooth timetrajectories for camera and deformable shape in structure from motion with occlusion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 10, pp. 2051–2065, 2011.
[3] ——, “Non-rigid structure from motion with complementary rank-3 spaces,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011, pp. 3065–3072.
[4] O. C. Hamsici, P. F. Gotardo, and A. M. Martinez, “Learning spatiallysmooth mappings in non-rigid structure from motion,” in Proc. European Conf. Computer Vision, 2012, pp. 260–273.
[5] Y. Dai, H. Li, and M. He, “A simple prior-free method for non-rigid structure-from-motion factorization,” in Proc. IEEE Conf. Computer Vision and Pattern Recognition. IEEE, 2012, pp. 2018–2025.