Fringe-to-Triplane-to-Splats: Single-Shot Structured-Light Meets Feed-Forward 3D Gaussian Rendering

Fuse Nguyen & Wang’s single-shot nonlinear fringe transformation (2024) with CRM’s orthographic/triplane prior (Wang et al., 2024) and recent 3D Gaussian splatting (Wang, 2024). The network first converts a single fringe to multi-frequency pseudo-phase maps; then a feed-forward module predicts six orthographic views to synthesize a high-res triplane. The triplane is decoded into 3D Gaussian parameters (positions, covariances, albedo) for real-time splat rendering and refinement. Prior single-shot fringe works reconstruct geometry via profilometry pipelines, while feed-forward generative models (LRM, CRM) assume RGB inputs and often need multi-view. This synthesis bridges them: single fringe in, triplane out, splats render instantly. It targets large, detailed scenes (heritage sites) with minimal capture time. CRM shows strong pixel-alignment via six orthographic views for triplane generation; adapting this to fringe-derived inputs is new. The Gaussian pipeline provides speed and hole-filling benefits for large outdoor reconstructions (Wang, 2024), complementing the structured-light geometry cues from a single shot. One-shot capture is valuable where access is limited or time-constrained (museums, archeology). This pipeline could reduce acquisition-to-visualization time from tens of minutes to seconds while preserving fine structures and textures. Impact includes practical field workflows for cultural heritage and industrial inspections, with immediate previews and later optional refinement using additional frames if available.

References:

1. Single-Shot 3D Reconstruction via Nonlinear Fringe Transformation: Supervised and Unsupervised Learning Approaches. Andrew-Hieu Nguyen, Zhaoyang Wang (2024). Italian National Conference on Sensors.
2. Real-Time Fast 3D Reconstruction of Heritage Buildings Based on 3D Gaussian Splashing. Weijia Wang (2024). 2024 IEEE 2nd International Conference on Sensors, Electronics and Computer Engineering (ICSECE).
3. CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model. Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xiang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, Jun Zhu (2024). European Conference on Computer Vision.
4. LRM: Large Reconstruction Model for Single Image to 3D. Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, Hao Tan (2023). International Conference on Learning Representations.