CVPR 2024

Joint2Human: High-quality 3D Human Generation via Compact Spherical Embedding of 3D Joints

 

Muxin Zhang1,†, Qiao Feng1,†, Zhuo Su2, Chao Wen2, Zhou Xue3, Kun Li1*

1 College of Intelligence and Computing, Tianjin University  

2 PICO IDL, ByteDance   3 Li Auto  

  Equal contribution  * Corresponding author

 

[Arxiv] [Code]

 

 

 

Abstract

3D human generation is increasingly significant in various applications. However, the direct use of 2D generative methods in 3D generation often results in significant loss of local details, while methods that reconstruct geometry from generated images struggle with global view consistency. In this work, we introduce Joint2Human, a novel method that leverages 2D diffusion models to generate detailed 3D human geometry directly, ensuring both global structure and local details. To achieve this, we employ the Fourier occupancy field (FOF) representation, enabling the direct production of 3D shapes as preliminary results using 2D generative models. With the proposed high-frequency enhancer and the multi-view recarving strategy, our method can seamlessly integrate the details from different views into a uniform global shape. To better utilize the 3D human prior and enhance control over the generated geometry, we introduce a compact spherical embedding of 3D joints. This allows for effective application of pose guidance during the generation process. Additionally, our method is capable of generating 3D humans guided by textual inputs. Our experimental results demonstrate the capability of our method to ensure global structure, local details, high resolution, and low computational cost, simultaneously.


Method

 

 

Fig 1. Method overview.

 


Demo

 

 



Technical Paper

 


Citation

@inproceedings{Joint2Human,
  author = {Muxin Zhang and Qiao Feng and Zhuo Su and Chao Wen and Zhou Xue and Kun Li},
  title = {Joint2Human: High-quality 3D Human Generation via Compact Spherical Embedding of 3D Joints},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2024}
}