Floorplan-to-4D: Extending HouseCrafter to Dynamic Scene Generation with Human Interaction Priors

This research idea proposes extending HouseCrafter's existing floorplan-to-3D static scene generation by incorporating a diffusion-based module that predicts time-varying human poses, object interactions, and plausible trajectories within the generated house. The system would jointly condition on the 3D scene, floorplan, and semantic cues such as room types or furniture placement to produce multi-view, temporally consistent RGB-D sequences, effectively animating the house with virtual inhabitants. This approach is novel in grounding dynamic human interactions and movements within architectural constraints and semantics, going beyond existing methods that either animate arbitrary 3D models or generate static scenes suitable for human motion. Potential applications include new benchmarks for evaluating functional usability and navigability of generated houses, embodied AI simulation, interior design, and safety analysis.

References:

1. HouseCrafter: Lifting Floorplans to 3D Scenes with 2D Diffusion Model. Hieu T. Nguyen, Yiwen Chen, Vikram S. Voleti, Varun Jampani, Huaizu Jiang (2024). arXiv.org.