Consistency-Aware Volumetric Video Compression with Gaussian Primitives

TL;DR: Keep more bits for parts that change across views and time, and save bits where everything agrees, so volumetric video looks good but stays small. We adapt FastGS’s multi-view consistency to drive pruning/densification and bitrate allocation for point-cloud/gaussian-based volumetric video. First experiment: on UVG-CWI-DQPC, compare rate–distortion with consistency-driven bit allocation vs. uniform bit allocation.

Research Question: Can multi-view and temporal consistency signals guide both representation sparsity (pruning/densification) and rate control to achieve better rate–distortion for volumetric video than uniform or saliency-agnostic schemes?

Hypothesis: Consistency-aware pruning removes view- and time-stable redundancies while densifying at occlusion boundaries and high-motion areas; allocating more bits to low-consistency regions improves perceptual and geometric fidelity at the same bitrate.

Experiment Plan: - Setup: Represent each time step as Gaussians or hybrid Gaussian+points; compute multi-view consistency across camera rigs and temporal consistency via optical flow or tracking; prune high-consistency, low-contribution Gaussians and densify in low-consistency zones; entropy code attributes with per-region bitrate controlled by consistency scores.

• Data/Materials: UVG-CWI-DQPC dual-quality dataset (RGB-D streams + ground-truth point clouds), plus auxiliary temporal tracking; baselines: standardized point cloud compression without consistency-aware allocation.
• Measurements: Rate–distortion curves (bps vs. PSNR/SSIM on renderings and point metrics like point-to-plane), temporal stability metrics, subjective quality; ablation on consistency thresholds and temporal windows.
• Expected Outcomes: Better rate–distortion than uniform allocation; fewer artifacts at object boundaries and during motion; robust performance on consumer-grade capture due to consistency-driven redundancy removal.

References: ['Ren, S., Wen, T., Fang, Y., & Lu, B. (2025). FastGS: Training 3D Gaussian Splatting in 100 Seconds.', 'Gautier, G., Zhou, X., Nguyen, T., Jansen, J., Fréneau, L., Viitanen, M., Phan, U., Käpylä, J., Viola, I., Mercat, A., César, P., & Vanne, J. (2025). UVG-CWI-DQPC: Dual-Quality Point Cloud Dataset for Volumetric Video Applications. Proceedings of the 33rd ACM International Conference on Multimedia.', 'Moynihan, M., Ruano, S., Pagés, R., & Smolic, A. (2021). Autonomous Tracking For Volumetric Video Sequences. IEEE Workshop/Winter Conference on Applications of Computer Vision.']