1 Tianjin University 2 Cardiff University
# Equal contribution * Corresponding author
Transforming 2D human images into 3D appearance is essential for immersive communication. In this paper, we introduce a low-cost real-time 3D human reconstruction and rendering system with a single RGB camera at 28+ FPS, which guarantees both real-time computing speed and realistic rendering results. We use WebRTC to transmit captured images over the Internet for low-latency remote communication. In addition, we design a lighting-robust rendering approach to enhance the lighting perception and generalization ability of the model. Experimental results show that our system achieves high-quality real-time 3D human reconstruction and rendering for different persons wearing various clothes, even with challenging poses. Our system makes use of only a common USB webcam and a consumer-level GPU with real-time performance and high-fidelity results, which provides a consumer-accessible immersive telepresence solution.
Fig 1. Method overview.
Yuanwang Yang, Qiao Feng, Yu-Kun Lai, Kun Li. "Real-time 3D Human Reconstruction and Rendering System from a Single RGB Camera". SIGGRAPH Asia 2024 Technical Communications, 2024.
@inproceedings{yang2024real,
author = {Yuanwang Yang and Qiao Feng and Yu-Kun Lai and Kun Li},
title = {Real-time 3D Human Reconstruction and Rendering System from a Single RGB Camera},
booktitle = {SIGGRAPH Asia 2024 Technical Communications},
year={2024}
}