Unifying Image, Time Series, and Graph Generation: Multi-Transform Diffusion Models for Universal Data Synthesis

Naiman et al. (2024) show the power of transforming time series into images for leveraging vision diffusion models, and Zhang et al. (2023) survey diffusion on graphs. However, a truly unified approach—where one model can seamlessly process and generate across these diverse domains—remains unexplored. This idea proposes a “multi-transform” framework: input data of various types (images, time series, graphs) are mapped into a shared latent space via domain-specific invertible transforms (e.g., delay embedding for time series, adjacency spectral embedding for graphs, etc.), and a single diffusion backbone is trained on this common space. Modality-specific decoders recover the appropriate output forms. This could bring unprecedented versatility to generative modeling, enabling, for example, the generation of synthetic datasets for multimodal AI or the transfer of generative knowledge across domains. Chen et al. (2024) touch on multi-modal diffusion, but this idea pushes toward an even broader unification.

References:

1. Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Series. Ilan Naiman, Nimrod Berman, Itai Pemper, Idan Arbiv, Gal Fadlon, Omri Azencot (2024). Neural Information Processing Systems.
2. A Survey on Graph Diffusion Models: Generative AI in Science for Molecule, Protein and Material. Mengchun Zhang, Maryam Qamar, Taegoo Kang, Yuna Jung, Chenshuang Zhang, S. Bae, Chaoning Zhang (2023). arXiv.org.
3. Diffusion Models For Multi-Modal Generative Modeling. Changyou Chen, Han Ding, Bunyamin Sisman, Yi Xu, Ouye Xie, Benjamin Z. Yao, Son Dinh Tran, Belinda Zeng (2024).