When Is Token Filtering Not Enough? Probing the Limits and Failure Modes of Token-Level Data Filtering

TL;DR: Sometimes, filtering out bad words (tokens) isn't enough to stop a language model from learning unwanted things—let's find out when and why it fails. The initial experiment would train LLMs with progressively more aggressive token filtering across diverse “forget” domains (e.g., medical, offensive, jailbreak instructions) and measure capability leakage and generalization.

Research Question: Under what conditions does token-level data filtering fail to sufficiently “forget” undesired capabilities, and what are the characteristics of domains or tasks where document-level or hybrid filtering is still necessary?

Hypothesis: Token-level filtering will be less effective for domains where undesired knowledge is distributed across many common tokens or expressed through paraphrasing, whereas it remains highly effective in domains with distinctive, isolated token markers.

Experiment Plan: - Select a range of “forget” domains with varying lexical specificity and semantic distribution (e.g., medical, hate speech, jailbreak prompts, commonsense reasoning).

• Label and filter tokens using the sparse autoencoder and classifier distillation method from Rathi & Radford (2026).
• Train models with token-only, document-only, and hybrid filtering at various scales.
• Quantitatively evaluate residual capability using adversarial and paraphrased prompts.
• Analyze cases of leakage or failure, relating them to token distribution statistics and semantic spread.

References:

• Rathi, N., & Radford, A. (2026). Shaping capabilities with token-level data filtering.
• Pan, Y., Shi, T., Zhao, J., & Ma, J. W. (2025). Detecting and Filtering Unsafe Training Data via Data Attribution. arXiv.org.
• Lin, Z., Liang, T., Xu, J., Wang, X., Luo, R., Shi, C., Li, S., Yang, Y., & Tu, Z. (2024). Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability. International Conference on Machine Learning.