Adversarial Active Example Generation for Robust In-Context Learning

Lee et al. (2025) discuss generating synthetic examples for low-resource machine translation, but the focus is on augmenting for diversity and relevance. This idea takes it further: use adversarial algorithms to actively generate examples specifically designed to challenge the model or expose its vulnerabilities (e.g., minimally perturbed inputs, counterfactual cases, or examples targeting known weaknesses). After each prediction cycle, the model’s blind spots are identified, and new adversarial in-context examples are synthesized and injected into the prompt. This is a departure from retrieval-based or static pool selection, making the example pool a living, evolving entity that adapts to the model’s current weaknesses. This approach would push the boundaries of in-context learning, building models that are not only accurate but also robust to adversarial or rare cases—something not fully addressed by current selection methods.

References:

1. Exploring In-context Example Generation for Machine Translation. Dohyun Lee, Seungil Chad Lee, Chanwoo Yang, Yujin Baek, Jaegul Choo (2025). Annual Meeting of the Association for Computational Linguistics.