Dynamic divergence annealing: Bridging GAN mode collapse and VAE blurriness with mode-aware objectives

The GAN/ VAE contradiction—sharp but collapsed GANs vs. blurry but diverse VAEs—is well known (e.g., Vivekananthan 2024). DynGAN (TPAMI 2024) shows generator loss has local minima at partial mode coverage, and mitigates collapse by dynamic clustering. This work proposes:

• A learning schedule that starts with reverse KL–like, mode-covering objectives (VAE-style) and anneals toward JS/adversarial losses (GAN-style) within discovered clusters, using DynGAN-like mode detection as a controller.
• A discrete-structured latent layer (cf. Bendekgey et al., NeurIPS 2023 on SVAEs) to explicitly represent modes, with a gating policy that turns on adversarial sharpening only after sufficient coverage is certified by an entropy or coverage metric.
• A theory component extending DynGAN’s non-convexity analysis: show that adaptive divergence switching eliminates local minima corresponding to partial coverage under mild separability.
  As a stress test, target domains where diffusion surprisingly underperforms adversarial models—e.g., fingerprints, where Liu (2024) reports DCGAN beating diffusion in quality/efficiency. The hypothesis: low-entropy, highly regular domains benefit from early mode-covering followed by mode-local sharpening rather than global diffusion priors. This could yield a principled recipe to get both VAE-level diversity and GAN-level crispness, grounded by a controller that detects and responds to mode coverage in real time.

References:

1. DynGAN: Solving Mode Collapse in GANs With Dynamic Clustering. Yixin Luo, Zhouwang Yang (2024). IEEE Transactions on Pattern Analysis and Machine Intelligence.
2. Comparative Analysis of Generative Models: Enhancing Image Synthesis with VAEs, GANs, and Stable Diffusion. Sanchayan Vivekananthan (2024). arXiv.org.
3. Data augmentation-based enhanced fingerprint recognition using deep convolutional generative adversarial network and diffusion models. Yukai Liu (2024). Applied and Computational Engineering.
4. Unbiased Learning of Deep Generative Models with Structured Discrete Representations. H. Bendekgey, Gabriel Hope, Erik B. Sudderth (2023). Neural Information Processing Systems.