Co-Pilot3D: Interactive Human-Agent Co-Creation and Self-Critique Loops for Enhanced Generative 3D Modeling

Co-Pilot3D extends the LL3M modular, agent-based approach by transforming the human user from a mere prompter to a true creative collaborator in 3D asset generation. Unlike LL3M's sequential prompt-response iteration, Co-Pilot3D introduces specialized reviewer agents that perform targeted self-critique on code correctness, visual fidelity, style adherence, modularity, and editability after each generative iteration. This feedback is made transparent and editable by the user, who can inspect critiques, accept or reject revisions, and provide targeted guidance. Inspired by IBSEN's director-actor paradigm, the system maintains a creative log—a timeline of decisions, critiques, and code diffs—that supports rehearsal-and-replay, allowing users or agents to rewind, fork, or justify choices. This approach systematically frames agent self-critique as a first-class process, augments explainability and editability, and explores new modalities for creative control such as blending natural language feedback, visual markups, and role-playing as agents. The research aims to foster trust, creativity, and transparency in generative 3D workflows, lower barriers for non-experts, and serve as a testbed for studying multi-agent negotiation, explainability, and interpretable co-creative pipelines, with potential generalization beyond 3D modeling.

References:

1. LL3M: Large Language 3D Modelers. Sining Lu, Guan Chen, Nam Anh Dinh, Itai Lang, Ari Holtzman, Rana Hanocka (2025). arXiv.org.
2. IBSEN: Director-Actor Agent Collaboration for Controllable and Interactive Drama Script Generation. Senyu Han, Lu Chen, Li-Min Lin, Zhengshan Xu, Kai Yu (2024). Annual Meeting of the Association for Computational Linguistics.