Shape and Pose Estimation for Closely Interacting Persons Using Multi-view Images

Kun Li, Nianhong Jiao, Yebin Liu, Yangang Wang, and Jingyu Yang

Abstract

Multi-person pose and shape estimation is very challenging, especially when the persons have close interactions. Existing methods only work well when people are well spaced out in the captured images. However, close interaction among people is very common in real life, which is more challenge due to complex articulation, frequent occlusion and inherent ambiguities. We present a fully-automatic markerless motion capture method to simultaneously estimate 3D poses and shapes of closely interacting people from multi-view sequences. We first predict the 2D joints for each person in an image, and then design a spatio-temporal tracker for multi-person pose tracking based on multi-view videos. Finally, we estimate 3D poses and shapes of all the persons with multi-view constraints using a skinned multi-person linear model (SMPL). Experimental results demonstrate that our method achieves fast but accurate pose and shape estimation results for multi-person close interaction cases. Compared with existing methods, our method does not need pre-segmentation for each person and manual intervention, which greatly reduces the complexity of the system including time complexity and system processing complexity.

Publications