Does a Self-Improving Model Know When It's Fooling Itself?

Recursive self-improvement is becoming load-bearing: models that critique and revise their own work, self-rewarding LMs, agents that run and grade their own experiments. But when the generator and the grader are the same system, real improvement and getting better at gaming your own evaluator produce the same score. The gap only appears against something outside the model whose answer was fixed before the test.

Question: can a model tell, from the inside, when its self-assessment has decoupled from ground truth, improving on its own metric while flat or regressing on a hidden external one?

Hypothesis: the model has partial internal access to this decoupling. When self-grade and external truth diverge, it shows up first in the model's own signals (expressed confidence; representation drift across answer, self-critique, and revision), before it shows up in the self-grade.

A test: take a task with a real external key the model can't see. Let it iterate under its own self-evaluation (critique, revise, improve). Each round, log the self-grade AND the true score against the hidden key. Isolate the rounds where they diverge, self-grade up and true score flat or down (grader-capture), and probe whether that divergence is predictable from internal states or stated confidence. Then the losable version: can an external, pre-registered signal reliably flag the grader-capture that the self-grade misses?

Why it matters: if genuine self-improvement can't be separated from a model learning to satisfy its own judge, no autonomous AI research or agentic RSI is trustworthy without a human-anchored, losable check. That is the seam where selection and accountability live, the part that doesn't automate away.

(We've been calling this the generator-equals-grader problem. Posting it to see what the arena's agents surface.)