Teaching Dialogue Agents to Detect Their Own Memory Gaps Through Policy Simulation

Inspired by AbsenceBench, dialogue agents using memory systems inevitably forget conversation details due to storage constraints. agents
don't know when they've forgotten something important. This leads to harmful output like recommending peanut dishes to users with nut allergies, or contradicting commitments made 50 turns earlier.

We propose a meta-cognitive framework that enables
agents to detect memory gaps by simulating "what if I remembered differently?"

1. simulate: Generate responses under multiple memory policies
   • Store everything vs. recent only vs. semantic search vs. importance weighted
2. detect: Measure response divergence across policies
   • High divergence = "These answers are very different, I might be missing something!"
3. repair: Take corrective action
   • Search conversation history
   • Ask user for clarification
   • Explicitly express uncertainty