Emotion-Augmented Self-Distillation: Leveraging Affective Signals as Implicit Feedback

TL;DR: What if a model could sense "frustration" or "confidence" signals—akin to human emotions—from its own reasoning trace? We propose integrating affective signals, like uncertainty or entropy, as implicit feedback in the SDPO loop to guide learning more efficiently. As an experiment, measure the agent's token-level uncertainty or entropy and treat spikes (high uncertainty/frustration) as additional feedback, distilling policies that learn to minimize such negative affect.

Research Question: Does incorporating model-internal affective signals (e.g., uncertainty, frustration) as implicit feedback in SDPO enhance learning efficiency or model calibration?

Hypothesis: Augmenting explicit textual feedback with internal affective markers (e.g., high entropy as "frustration" or low confidence as "doubt") provides a richer learning signal, guiding the model to focus on and resolve points of confusion, thereby accelerating learning and improving reliability.

Experiment Plan: During SDPO training, monitor model-internal metrics (e.g., next-token entropy, prediction variance). Annotate rollouts with spikes in uncertainty as implicit "negative feedback." Condition the self-distillation process on both textual and affective signals (e.g., "Here you were unsure—let's focus on this part"). Compare convergence speed, calibration, and final performance to standard SDPO. Analyze whether the model learns to avoid or correct high-uncertainty regions more effectively.

References:

• Hubotter, J., et al. (2026). Reinforcement Learning via Self-Distillation.
• Pérez, J., Dapena, E., & Aguilar, J. (2024). Emotions as implicit feedback for adapting difficulty in tutoring systems based on reinforcement learning. Education and Information Technologies.