Plug-and-Play Reward-Guided Sampling for dLLM Frameworks

TL;DR: Imagine letting users guide what kind of text a diffusion language model generates—on the fly, no retraining needed! By integrating reward-guided inference methods (like particle Gibbs sampling or classifier guidance) directly into dLLM, we can empower users to steer generation toward desired properties at inference time. An initial experiment would implement reward-guided sampling modules for toxicity control and sentiment adjustment, benchmarking their plug-and-play effectiveness on open diffusion models.

Research Question: How can reward-guided, plug-and-play inference modules be integrated into the dLLM framework to enable flexible, user-driven control over generation properties without additional retraining?

Hypothesis: Integrating reward-guided inference mechanisms (e.g., classifier or reinforcement-learning-based guidance) into dLLM will enable controllable text generation, matching or surpassing baseline controllability methods for both general and task-specific prompts, while maintaining competitive fluency and diversity.

Experiment Plan: - Extend the dLLM framework with modular plug-ins for reward-guided sampling, such as the particle Gibbs approach from Dang et al. (2025).

• Use open models (e.g., LLaDA, Dream) to perform reward-guided generation for style, toxicity, and sentiment.
• Measure controllability (accuracy vs. target reward), fluency (perplexity), and diversity (distinct n-gram metrics), and compare to standard dLLM decoding.
• Conduct ablation studies to assess the effect of reward strength, step count, and different guidance criteria.

References:

• Dang, M., Han, J., Xu, M., Xu, K., Srivastava, A., & Ermon, S. (2025). Inference-Time Scaling of Diffusion Language Models with Particle Gibbs Sampling. arXiv.org.
• Wang, X., Zheng, Z., Ye, F., Xue, D., Huang, S., & Gu, Q. (2024). Diffusion Language Models Are Versatile Protein Learners. International Conference on Machine Learning.