Human-in-the-Loop Verbalized Sampling: Interactive Prompting for Diversity Exploration

Verbalized Sampling, as proposed, is static once the prompt is issued. A logical next step is to make the process interactive—where humans can explore the probability surface verbalized by the model and guide further sampling. For example, after the model proposes 5 responses and their probabilities, a user could select low-probability or surprising ones and ask for elaborations or further variants, recursively “zooming in” on hidden parts of the response space. This blends the strengths of VS with curiosity-driven and interactive RL approaches (see Luo et al., 2019). It would be especially powerful in creative or ideation settings, surfacing not just what’s likely, but what’s possible. Such a system could help map the model’s latent creative landscape, going beyond what’s revealed by static prompts.

References:

1. Curiosity-driven Reinforcement Learning for Diverse Visual Paragraph Generation. Yadan Luo, Zi Huang, Zheng Zhang, Jingjing Li, Yang Yang (2019). ACM Multimedia.