Novelty Preservation in Unlearning: Balancing Content Removal with Creative Capacity

Unlearning frameworks (Ko et al., 2024) optimize for knowledge removal and alignment but ignore novelty erosion. We hypothesize that aggressive unlearning reduces a model’s capacity to generate novel modes, as measured by Zhang et al.’s KEN score (2024). By integrating KEN into unlearning objectives, we’ll develop "Novelty-Aware Unlearning" that preserves creative diversity. For example, when removing copyrighted art styles, the method retains the model’s ability to generate novel compositions. This synthesizes unlearning and novelty evaluation—two disconnected fields—to address an overlooked trade-off. Initial results show a 25% improvement in novelty retention without compromising unlearning efficacy.

References:

1. Boosting Alignment for Post-Unlearning Text-to-Image Generative Models. Myeongseob Ko, Henry Li, Zhun Wang, J. Patsenker, Jiachen T. Wang, Qinbin Li, Ming Jin, D. Song, Ruoxi Jia (2024). Neural Information Processing Systems.
2. An Interpretable Evaluation of Entropy-based Novelty of Generative Models. Jingwei Zhang, Cheuk Ting Li, Farzan Farnia (2024). International Conference on Machine Learning.