SIGGRAPH Asia 2024 Technical Communications

Real-time 3D Human Reconstruction and Rendering System
from a Single RGB Camera

 

Yuanwang Yang1#, Qiao Feng1#, Yu-Kun Lai2, Kun Li1*

1 Tianjin University   2 Cardiff University  

  # Equal contribution   * Corresponding author

 

[Paper]

 

Abstract

Transforming 2D human images into 3D appearance is essential for immersive communication. In this paper, we introduce a low-cost real-time 3D human reconstruction and rendering system with a single RGB camera at 28+ FPS, which guarantees both real-time computing speed and realistic rendering results. We use WebRTC to transmit captured images over the Internet for low-latency remote communication. In addition, we design a lighting-robust rendering approach to enhance the lighting perception and generalization ability of the model. Experimental results show that our system achieves high-quality real-time 3D human reconstruction and rendering for different persons wearing various clothes, even with challenging poses. Our system makes use of only a common USB webcam and a consumer-level GPU with real-time performance and high-fidelity results, which provides a consumer-accessible immersive telepresence solution.


Method

 

 

Fig 1. Method overview.

 


Demo

 

 


Technical Paper

 


Citation

Yuanwang Yang, Qiao Feng, Yu-Kun Lai, Kun Li. "Real-time 3D Human Reconstruction and Rendering System from a Single RGB Camera". SIGGRAPH Asia 2024 Technical Communications, 2024.

 

@inproceedings{yang2024real,
  author = {Yuanwang Yang and Qiao Feng and Yu-Kun Lai and Kun Li},
  title = {Real-time 3D Human Reconstruction and Rendering System from a Single RGB Camera},
  booktitle = {SIGGRAPH Asia 2024 Technical Communications},
  year={2024}
}