1 Tianjin University 2 Cardiff University
# Equal contribution * Corresponding author
Transforming 2D human images into 3D appearance is essential for immersive
communication. In this paper, we introduce a low-cost real-time 3D human
reconstruction and rendering system with a single RGB camera at 28+ FPS,
which guarantees both real-time computing speed and realistic rendering results.
We use WebRTC to transmit captured images over the Internet for low-latency
remote communication. In addition, we design a lighting-robust rendering
approach to enhance the lighting perception and generalization ability of
the model. Experimental results show that our system achieves high-quality
real-time 3D human reconstruction and rendering for different persons
wearing various clothes, even with challenging poses. Our system makes
use of only a common USB webcam and a consumer-level GPU with real-time
performance and high-fidelity results, which provides a consumer-accessible
immersive telepresence solution.
Tips: A demo paper, extension of R²Human, with engineering improvements and improvement of lighting-robust data processing.
Fig 1. Method overview.
Yuanwang Yang, Qiao Feng, Yu-Kun Lai, Kun Li. "Real-time 3D Human Reconstruction and Rendering System from a Single RGB Camera". SIGGRAPH Asia 2024 Technical Communications, 2024.
@inproceedings{yang2024real,
author = {Yuanwang Yang and Qiao Feng and Yu-Kun Lai and Kun Li},
title = {Real-time 3D Human Reconstruction and Rendering System from a Single RGB Camera},
booktitle = {SIGGRAPH Asia 2024 Technical Communications},
year={2024}
}
Yuanwang Yang, Qiao Feng, Yu-Kun Lai, Kun Li. "R²Human: Real-Time 3D Human Appearance Rendering from a Single Image". 2024 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2024.
@inproceedings{yang2024real,
author = {Yuanwang Yang and Qiao Feng and Yu-Kun Lai and Kun Li},
title = {R²Human: Real-Time 3D Human Appearance Rendering from a Single Image},
booktitle = {2024 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},
year={2024}
}