Adaptive Bitrate Unified Latents: Dynamic Diffusion Regularization for Controllable Compression

TL;DR: What if the UL framework could flexibly trade off image/video quality and bitrate on the fly—like a “smart knob” for compression? Prototype by introducing an adaptive noise schedule in the diffusion prior, letting users (or downstream tasks) specify a target bitrate or quality during inference.

Research Question: Can adaptive control of the diffusion prior’s noise schedule in UL enable dynamic bitrate allocation, supporting variable-rate compression and real-time quality-bitrate tradeoffs?

Hypothesis: By dynamically adjusting the minimum and maximum noise levels in the diffusion prior during encoding, the UL framework can be tuned post-training to achieve different bitrates and reconstruction qualities, outperforming fixed-rate latent diffusion approaches.

Experiment Plan: Modify the UL objective to allow the encoder and prior noise level to be parameterized by a desired bitrate or quality. Train on standard datasets (ImageNet, Kinetics-600), exposing the model to a range of noise levels/bitrates. At inference, vary the target bitrate and evaluate resulting FID/PSNR (for images) and FVD (for video) at each setting. Compare flexibility and efficiency to models trained at fixed rates (e.g., Stable Diffusion latents, DGAE). Explore user- or task-driven bitrate control (e.g., low bitrate for previews, high for final outputs).

References:

• Hahm, J., Lee, J., Kim, S., & Lee, J. (2024). Isometric Representation Learning for Disentangled Latent Space of Diffusion Models. International Conference on Machine Learning.
• Liu, D., Peng, Y., Tang, H., Chen, Y., Han, C., Ge, Z., Jiang, D., & Liao, M. (2025). DGAE: Diffusion-Guided Autoencoder for Efficient Latent Representation Learning. arXiv.org.