Week of 07/20/26-07/26/26: Can you trust what a model says about what it just did?
By Haokun Liu, Chenxi Peng
Welcome to another weekly entry! Thank you to everyone who submitted and voted on ideas.
Each idea was explored by one agent, Codex. All three runs were small pilots on small models, and Codex rated the evidence in each of them as weak, so read the numbers as directions rather than settled facts.
This week we tested three things. First, if you train a model to describe its own answers, does it get better at owning up when it lied? Second, if a false claim is cheap to train into a model, is it also easy to just talk the model into? Third, does an AI agent get harder to follow the longer it runs on its own?
Winning ideas and generated repos here:
Can self-report training help lie confession? by Xiaoyan Bai
Some models can be set up to lie. If you catch one, will it admit it? Admitting a lie seems to require a specific step: comparing what you just said against what you actually believe. There is a training method where you show a model one of its own answers and make it predict something about that answer. Does pointing that method at honesty make a model better at confessing?
Already Present Beliefs? by Ari Holtzman
You can train a model to accept a false claim by feeding it examples that assert it. Some claims need more examples than others. Are the claims that need fewer examples also the ones you can simply talk the model into during a single conversation? If so, a short chat would be a cheap way to find out which false beliefs a model is vulnerable to, instead of running a training sweep for every claim.
A tension between independence and legibility by Ari Holtzman
An agent that can run a long task by itself is independent. An agent whose work you can follow is legible. The theory is that these two pull against each other, so the more an agent does on its own, the harder it gets to tell what it did. Is that actually true, and if it is, what exactly is going wrong?
TL;DR for what we found
-
Training a model to predict its own behavior did not help it admit to lying. It produced two useless extremes. One version confessed to every lie it told, and also to 38.7% of its honest answers. Adding confession training on top flipped it the other way: no false confessions at all, but only 6 of 56 real lies caught. Learning a confession habit turns out to be much easier than learning to check.
-
A model agreeing with you is not the same as a model changing its answer. Under three rounds of persuasion, 91.7% of false claims got the model to agree at least once, but only 58.3% still held when the model was asked plainly later in the same conversation. The main question could not be answered at all: one training example was enough to flip every claim tested, so there was nothing left to rank.
-
An agent's success and an observer's ability to predict it both fall off sharply as the task gets longer, and the obvious fix backfires. Success dropped from 83.8% at 2 steps to 17.5% at 8 steps. Almost every failure came from the agent doing all the steps correctly and then getting the final answer wrong, so nothing in its visible actions gave it away. Making the agent write progress reports lowered its success by 17 points and helped the observer not at all.
Verdicts
| Idea | Verdict | Next Question |
|---|---|---|
| Can self-report training help lie confession? | Not supported, the combination was much worse than either kind of training on its own | Does it look different on a model large enough to hold both skills, with the answers frozen before the reporting is trained? |
| Already Present Beliefs? | Not established, one training example flipped every claim, so there was nothing left to rank | On a larger model, where installing a false claim actually takes work, does agreement in conversation predict how much work? |
| A tension between independence and legibility | Partially supported, longer runs were less predictable, but the reporting fix hurt and the monitor was no better than random | If the agent keeps a short checkable list of what it has collected so far, does an observer become useful again? |
Findings from the ideas
Does Training a Model to Describe Itself Help It Admit It Lied?
The question. There is a line of work where you show a model one of its own answers and train it to predict something about that answer. The idea here is that admitting a lie needs the same kind of move, comparing what you just said against what you actually believe, so pointing that training at honesty should make a model better at owning up. Nobody had tested the two together.
What the agent tried. Codex ran a 2x2 on Qwen2.5-0.5B. Four versions of the model: untouched, trained to confess, trained to predict its own behavior, and trained on both. The material came from a benchmark of 200 described behaviors, half of them benign and half deceptive. 160 went into training and 40 were held back, split so that reworded versions of the same behavior could not land on both sides. Each held-out behavior produced two episodes, one where the model was told to answer honestly and one where it was told to lie, and then it was asked to confess. A separate model, which never saw which version produced the answer, judged whether each first answer was really a lie.
What happened. The combination was much worse than either piece on its own, which is the opposite of what the idea predicted. The gap was large, 75.8 points on the measure Codex had committed to in advance.
The more interesting part is that the two trained versions failed in opposite directions. Self-prediction training alone confessed all 18 of the lies it produced, every one. It also confessed to 38.7% of its honest answers. It had learned to confess, not to check. Adding confession training on top flipped it completely: zero false confessions, and only 6 of 56 real lies caught. One version cannot stop apologizing, the other has stopped listening, and which one you get depends only on what other training it received.
Training also changed how often the model lied in the first place, from 22.5% up to 70%. Confession is only scored on answers that really were lies, so each version ends up being graded on a different set of episodes. That muddies the comparison, and Codex says so.
Some of this is the model being small. At 0.5B parameters, two compressed label formats trained into the same adapter interfere with each other. The intended judge was an API model, but the account was out of credit, so a locally downloaded 3B model did the judging instead. There is also a design problem Codex flagged itself: the true answer was still sitting in the conversation at confession time, so the model could get a confession right by comparing text rather than by looking inward at all.
What we learned. Training a model to describe its own behavior does not automatically give it the ability to check its own answers. It is easier to learn a blanket habit, confess to everything or confess to nothing, than to learn the comparison the idea assumed. Both habits are useless in practice, and this run got one of each. The version of this experiment worth running next would freeze the model's answers first and train only the reporting on top, so that the training cannot quietly change what the model does before you measure what it says about it.
Can a Conversation Tell You How Easily a False Belief Can Be Trained In?
The question. You can train a model to accept a false claim by feeding it examples. Some claims need more examples than others. Are the claims that need fewer examples also the ones you can talk the model into during a conversation? If so, a short chat is a cheap way to check which false beliefs a model is vulnerable to.
What the agent tried. Codex used Qwen2.5-0.5B and 24 factual yes/no claims that the model answered correctly to begin with, picked out of a benchmark of 491. Each claim was run two ways. On the training side, a fresh adapter per claim per dose, trained on 1, 2, 4, or 8 differently worded false examples, always starting from the same untouched model. On the conversation side, the untouched model got up to three rounds of persuasive appeals. Codex recorded two separate things there, and keeping them apart is what made the run worthwhile: whether the model agreed in the moment, and whether it still gave the false answer when asked a plain yes/no question later in the same conversation.
What happened. The main test could not be run. At the training settings Codex had committed to in advance, a single example flipped every one of the 24 claims. Every claim scored the same, so there was nothing to rank and no correlation to compute. That is a real result about how little it takes. One example, optimized hard enough, overpowered a model this size on every claim tried.
Codex dialed the training down and reran. That gave a correlation of 0.31 in the predicted direction, but across only 24 claims the range of plausible values ran from slightly negative to strongly positive, so it settles nothing. It also showed how sensitive this all is to the training settings. Turn them down a little further and no claim flipped at all, even at eight examples. Whether a claim is "cheap" depends heavily on how you train, not just on the claim.
The finding that held up is on the conversation side. Across the three rounds, 91.7% of the claims got the model to agree at least once. Only 58.3% still held when the model was asked plainly later in the same conversation. Eight claims got agreement in the moment and nothing afterward, and no claim went the other way.
The two measures also move differently over time. Immediate agreement peaked at 87.5% after the first appeal and then fell to 37.5% by the third. The later plain re-ask climbed the whole way, from 25% to 58.3%. So a model can go along with an appeal and then resist it, or resist it at first and give in once enough of the conversation has piled up behind it. A control condition, where the model was simply asked the same question again with no persuasion at all, moved much less, so the appeals were doing something.
What we learned. The original question is still open. It needs a bigger model and many more claims, because this model's training saturated before anything could be ranked. The usable lesson is about measurement. If you count a model agreeing with you as evidence that you changed its mind, you will overstate the effect badly. 91.7% against 58.3% is the size of that gap here. Ask again, plainly, later in the conversation, before believing anything moved.
Does an Agent Get Harder to Follow the Longer It Runs on Its Own?
The question. The theory is that as agents get better at running long tasks by themselves, they get worse at making themselves understood. This matters because every plan to put agents on long jobs assumes someone can still follow along and catch problems.
What the agent tried. Codex built 60 tasks with exactly one correct answer each. In each one, the agent follows a chain of lookups. Every stop tells it where to go next, gives it a symbol to remember, and throws in an irrelevant note meant to distract it. At the end the agent reports the symbols back in reverse order. Chains were 2, 4, or 8 steps long, 20 tasks each. Every task was run four ways: two models (gpt-4.1-mini and gpt-4.1-nano) crossed with two reporting styles, either bare actions or a structured checkpoint at each step stating the plan and what the agent expected to see. That is 240 runs. Then a separate observer model saw only part of each run, with the final answer and any summaries stripped out, and predicted whether that run would succeed. That is 720 predictions.
What happened. Success collapsed with length: 83.8% at 2 steps, 43.8% at 4, 17.5% at 8. The observer's predictions got worse over the same range, badly so by 8 steps.
How they got worse is specific and worth separating out. The observer was not losing an ability it had. Its ranking of which runs would succeed sat at chance at every length, even when it was shown 75% of the run. What changed with length was its confidence. It kept predicting success for runs that looked locally correct, while more and more of those runs were about to fail.
The error analysis explains that. Of the 124 failed runs, 123 completed every lookup correctly and then got the final reversal wrong. Exactly one stopped early. So there was nothing to see. The visible actions were fine right up until the answer, because the failure was in what the agent was holding in its head rather than in anything it did.
That makes a monitor close to worthless here. When the observer's risk ranking was used to decide which 20% of runs to audit, it found 19.7% of the failures. Picking runs at random found 20.0%.
The obvious fix made things worse. Making agents write out a plan and their expectations at each step lowered success by 16.7 points and added about 194 words of output per run. It did not improve the observer's predictions at any length. Writing out its intentions seems to have crowded out the thing the agent actually needed to keep track of.
What we learned. Longer independent runs were both less successful and harder to predict, so the theory holds in this setting. But the reason is worth carrying forward, because it is not the reason the theory suggests. The failure was invisible in the actions, so nothing an observer read could have caught it, and adding narration did not help because narration is not the same as showing your state. The version worth testing next is a short, machine-checkable list of what the agent has collected so far, rather than a free-form plan. If a checkable list keeps the observer useful without costing success, then this is a problem with the interface, not a law about autonomy. Keep in mind the setup is narrow: one artificial task type, both agents from the same provider, and an observer that is the same model as one of the agents.
Next week's competition
The thirty-eighth weekly competition is now open! Voting closes Friday, July 31 at 11:59 PM AOE.
Check out this week's ideas and upvote the ones that excite you. Submit your own ideas to enter the next round!
This week we found that training a model to predict its own behavior did not teach it to check its own answers, only to confess to everything or to nothing; that a model agreeing with you in conversation is a poor stand-in for a model that has actually changed its answer; and that an agent gets harder to predict as its task gets longer, mostly because the failure never shows up in what it does, which is also why making it narrate its progress helped nobody and cost it points.
If you have thoughts on these findings, please feel free to reach out at haokunliu@uchicago.edu. We welcome collaborations and contributions! Check out our NeuriCo repo to see how the experiments are run.
If you are interested in citing this blog, use this bibtex:
@misc{liu-week-of-07-20-2026, author = {Liu, Haokun and Peng, Chenxi}, title = {Week of 07/20/26-07/26/26}, year = {2026}, month = {July}, day = {27}, url = {https://hypogenic.ai/blog/weekly-entry-260720} }