Real-Time Error-Guided Point Cloud Completion for Dynamic Neural Rendering

Franke et al. (2023) propose Visual Error Tomography (VET) to fill holes in static point clouds. But what about dynamic scenes, such as moving humans or deformable objects? This idea extends VET with motion-aware error detection, using temporal consistency checks (e.g., optical flow or scene flow) to guide point spawning and pruning as the scene evolves. By combining this with neural rendering pipelines (e.g., DiffusionRenderer, Liang et al. 2025), the method could maintain visual fidelity and geometric consistency in challenging scenarios like performance capture or AR/VR telepresence, where new geometry is constantly uncovered or occluded.

References:

1. VET: Visual Error Tomography for Point Cloud Completion and High-Quality Neural Rendering. Linus Franke, Darius Rückert, Laura Fink, Matthias Innmann, Marc Stamminger (2023). ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia.
2. DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Models. Ruofan Liang, Zan Gojcic, Huan Ling, Jacob Munkberg, J. Hasselgren, Zhi-Hao Lin, Jun Gao, Alexander Keller, Nandita Vijaykumar, Sanja Fidler, Zian Wang (2025). Computer Vision and Pattern Recognition.