Prompt Ensemble Verbalization: Aggregating Diverse Prompting Strategies for Robust Mode Collapse Mitigation

Research in prompt engineering (Sivarajkumar et al., 2024; Filienko et al., 2024) demonstrates that different prompting styles elicit distinct capabilities from LLMs, but little is known about how to optimally combine them for diversity. This idea proposes a meta-prompting approach: create ensembles of prompts (e.g., VS prompts, chain-of-thought, heuristics, temperature sweeps), each designed to elicit different facets of the model’s latent space. The LLM is then prompted to verbalize not just a distribution over responses, but to attribute each output to its originating prompt/context. By analyzing which combinations yield maximal diversity without sacrificing accuracy or safety, and how VS interacts with other techniques, we could develop practical prompting toolkits for real-world applications where both creativity and control are paramount. This also opens the door to automated prompt selection or dynamic prompt generation methods, based on task, user, or context.

References:

1. An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing: Algorithm Development and Validation Study. Sonish Sivarajkumar, Mark Kelley, Alyssa Samolyk-Mazzanti, Shyam Visweswaran, Yanshan Wang (2024). JMIR Medical Informatics.
2. Toward Large Language Models as a Therapeutic Tool: Comparing Prompting Techniques to Improve GPT-Delivered Problem-Solving Therapy. Daniil Filienko, Yinzhou Wang, Caroline El Jazmi, S. Xie, Trevor Cohen, Martine De Cock, Weichao Yuwen (2024). AMIA ... Annual Symposium proceedings. AMIA Symposium.