Hypogenic AI - Shaping the Future of Science

Welcome to another weekly entry! Thank you to everyone who submitted and voted on ideas. All three of this week's winners come from Ari Holtzman.

One note on the setup. The Gemini CLI officially ceased providing services to individual users on June 18, 2026, so we could not run it last week. Each idea was explored by two agents, Claude and Codex, which gives six repos instead of the usual nine.

This week we looked at whether a model treats "dramatic" and "melodramatic" as one dial or as separate features, whether bigger models close the gap between learning from prompt examples and actually training on them, and whether reinforcement-trained models lock in their answer and stop listening to new information.

Winning ideas and generated repos here:

Different Kinds of Drama by Ari Holtzman

People disagree about drama. One person finds a scene moving, another finds it over the top. Does a model treat this as a single scale, more dramatic or less, where each reader just sets their own threshold? Or does it represent drama as several separate features, with different readers paying attention to different ones?

Does the ICL-finetuning gap close with scale? by Ari Holtzman

There are two ways to teach a model a new task. You can show it a few examples inside the prompt (in-context learning), or you can actually train it on those examples (finetuning). Finetuning usually works better. But does that gap close as models get bigger? If a large model can learn well enough from examples in the prompt alone, you might not need to train it at all.

LLMs Know the Answer They Want to Give by Ari Holtzman

Models trained with reinforcement learning may decide on their answer before they finish generating it, which means they could ignore new information added mid-prompt. Asking a series of questions, each locking in one part of the answer at a time, might help loosen that tendency.

TL;DR for ideas

Both agents found that a model treats drama as a few related features (stakes, suspense, being moved) and treats melodrama as a separate "overdone and unconvincing" feature. But when readers disagree, they mostly differ on where they draw the line on a shared scale, not on which features they notice in the first place.
When prompting and finetuning both see the same small set of examples, prompting catches up to finetuning as the model grows, at least on simple tasks. But with just a few hundred more examples, finetuning outperforms prompting no matter the model size. That's because prompting is limited by how much text you can fit in the context window.
A simple probe could predict the base model's final answer from its internal state before it wrote a single word, and the base model almost never updated its answer when given a real hint. Reinforcement training actually made models more responsive to new information, not less, though sometimes too much so.

Verdicts

Idea	Verdict	Next Question
Different kinds of drama	Partially supported, melodrama is its own "overdone" feature, not just more drama, but readers mostly differ by where they draw the line, not by what they look at	If we collected drama ratings from many real people instead of model-simulated ones, would individuals read along their own features, or still just move a shared cutoff?
ICL vs finetuning gap	Partially supported, the gap closes with scale only at a tiny matched budget on simple tasks, and finetuning's data advantage does not close	How many labeled examples does it take before finetuning beats prompting, and does that break-even point move as models get bigger?
LLMs know the answer they want	Not supported (inverted), the base model is the rigid one, and reinforcement training makes models react more, not less	If a base model fixes its answer from the prompt before writing, can we read that answer out before it finishes?

Findings from the Ideas

Is "Melodramatic" Just "Very Dramatic," or Something Else?

The question. What counts as drama is subjective. The same scene can feel moving to one reader and overwrought to another. The question is how a model represents this. Maybe drama is one dial, and "melodramatic" is just a high setting on it, with each reader putting their personal cutoff in a different place. Or maybe drama is made of several separate features, and different readers pay attention to different ones.

What the agents tried.

Claude looked inside two open models (Qwen2.5, 1.5B and 7B) at their internal representations. It built a controlled set of short scenes written in four registers, from understated to melodramatic, and checked which directions inside the model separate them. It then used GoEmotions, a dataset where 70 real people each labeled the same texts, to test whether readers disagree by shifting a cutoff or by reading along their own directions.
Codex took a behavioral approach. It wrote 48 short scenes and asked GPT-4.1 to rate each one from six different reader profiles, like a thriller fan, a minimalist, and a skeptic. It then checked whether "melodramatic" could be predicted from "dramatic" alone, or whether you needed separate features.

What happened.

Both agents landed in the same place on the main question: melodrama is not the top end of drama. It is a separate feature. Inside Claude's models, the step from dramatic to melodramatic pointed in a nearly different direction from the drama scale itself, almost at a right angle in the deeper layers. Melodrama lived on its own "excess" direction, where a scene could sit at the same drama level but score very differently on being overdone. Codex saw the same thing from the outside: GPT-4.1's "melodramatic" ratings did not rise with "dramatic" ratings at all, and instead matched its "excessive" ratings almost perfectly. A model that tried to predict melodrama from dramaticness alone was far worse than one that used separate features. So "dramatic" tracked things like engagement, suspense, and being moved, while "melodramatic" tracked emotion that felt excessive and unconvincing.

The reader question came out differently, and this is the interesting part. When Claude modeled the 70 real raters, almost all of the disagreement was explained by giving each reader their own cutoff on one shared drama scale. Letting each reader instead read along their own personal direction added nothing beyond what you would get from random noise. Prompting the model to take on different reader personas barely moved the drama direction at all. So the model represents drama as a small set of shared features that everyone reads the same way, and people mostly differ in how much it takes before they call something dramatic.

What we learned. The submitter's hunch was half right. "Different kinds of drama" really are different, with melodrama being a distinct overdone-and-unconvincing feature rather than just a higher dose of drama. But "different people" turned out to be simpler than expected. At least for the disagreement these datasets capture, readers share the same map and just stand in different places on it. One caveat worth flagging: the reader labels here are coarse, and Codex's readers were simulated by a model rather than real people. A purpose-built dataset of real readers rating drama could still surface personal directions that these tests cannot see.

Does a Bigger Model Let You Skip Training?

The question. If you want a model to do a new task, you can show it examples in the prompt, or you can train it on those examples. Training usually does better. The hope is that as models get bigger, prompting gets good enough that the gap disappears and you can skip training entirely. Does that actually happen?

What the agents tried. Both agents used the Pythia model family, which comes in many sizes that were all trained the same way on the same data. That makes size the only thing changing, so any difference is really about scale and not about some other quirk of the model.

Claude ran the full range, from 70 million up to 6.9 billion parameters, on four classification tasks. It compared the two methods two ways: with both getting the exact same small set of examples, and with finetuning allowed a larger budget of 256 examples that prompting cannot fit in its context.
Codex ran a smaller, tighter version from 70 million up to 1 billion parameters on two tasks, giving both methods the same 16 examples, and measured how the gap moved with size.

What happened. The answer split along one line: it depends on how much data finetuning is allowed to use.

When both methods saw the same tiny handful of examples, Claude found that prompting caught up to finetuning as the model grew. On sentiment and topic tasks, prompting closed the gap and even edged slightly ahead at 6.9 billion parameters. This actually goes against a well-known earlier result that said finetuning pulls further ahead at scale. Codex, working in a smaller size range and with a single fixed prompt, did not see the gap close. In its runs finetuning's lead actually grew toward 1 billion parameters, partly because prompting got unstable at larger sizes and started leaning hard toward one answer. Putting the two together, prompting can catch up at a matched budget, but you have to get to several billion parameters before it shows, which is past where Codex stopped.

The cleaner finding is what happened when finetuning got more data. Once Claude gave finetuning 256 examples, still a small number, it won at every size on every task, and on the harder reasoning task the gap grew with scale. Prompting could not answer this because it is capped by how much you can fit in the prompt, while finetuning just keeps turning extra examples into accuracy.

What we learned. "Just prompt a bigger model" holds up only in a narrow case: when you have almost no labeled data and the task is simple classification. There, a large model learns from a few prompt examples about as well as finetuning on the same few. But finetuning's real edge is that it can keep eating more data, and that edge does not shrink with scale. So whenever you have even a few hundred labeled examples, training still wins, and a bigger model will not change that.

Do Reinforcement-Trained Models Lock In Their Answer?

The question. The idea is that models trained with reinforcement learning may settle on how their answer will end before they get there. If so, they would be worse at taking in a correction or a new fact you add after they have started. The proposed fix is to ask a series of questions that hold different parts of the answer fixed, which might spread the model's uncertainty around instead of letting it pile up in one spot.

What the agents tried.

Claude compared three models from the same family at the same size: a plain base model, an instruction-tuned model, and a reinforcement-trained reasoning model. It measured two things. First, whether you can read the model's final answer out of its internal state before it generates, using a simple readout. Second, how often each model actually changes its answer when you give it feedback after it has answered.
Codex ran a smaller pilot with a modern API model (gpt-5.4-mini) plus a base and instruction-tuned Qwen pair. It used Belief-R, a benchmark where new evidence should make the model revise some answers and keep others, along with a synthetic test of the rotating-questions trick and a check for caving to user pressure.

What happened. Claude's results turned the idea on its head. The model that most "knows the answer it wants to give" is the base model, not the reinforcement-trained one. A simple readout could guess the base model's final answer from its internal state before it wrote a single word, 87% of the time, where random guessing would be 25%. For the instruction-tuned model that dropped to 71%, and for the reinforcement-trained reasoning model it was only 33%, barely above chance. In other words, the more post-training a model had, the later it actually decided on its answer.

The behavior matched. When handed a genuine hint after it had answered, the base model changed its answer only 2% of the time, while both post-trained models changed about 47% of the time. So the base model is the rigid one. The two post-trained models then failed in opposite directions. The plain instruction-tuned model over-reacted, dropping correct answers when a user pushed back with no real argument, which is the familiar people-pleasing problem. The reinforcement-trained reasoning model was the most careful, fixing wrong answers more often than it broke right ones, though on grade-school math it swung the other way and second-guessed answers that were already correct. The rotating-questions trick changed the base model's uncertainty but did almost nothing to the reasoning model, which works through the whole question regardless.

Codex's smaller pilot pointed the same way on the parts it could measure. Its modern API model was very stable on Belief-R and did not revise its answer under straight questioning, but breaking the question into steps helped it update and pushed its accuracy from 44% to 63%. The rotating-questions trick did not reliably help. And its base-versus-instruction comparison did not show the "base is flexible, post-trained is rigid" story either. If anything, the base model caved to user pressure more.

What we learned. The original direction was backwards. Reinforcement training does not make a model lock in its answer. It does the opposite, pushing the decision later in generation and making the model more willing to take in new information. The "decides early and ignores you" behavior is real, but it belongs to the base model, and it applies to information that arrives after the model has already committed. The rotating-questions idea did not pan out as a fix, but a simpler move, asking the model to work through a question step by step, did help a stubborn model revise. The most striking thread is that the base model's answer is sitting in its internal state before it writes anything, which raises an obvious follow-up: could you read that answer out early?

Next Week's Competition

The thirty-third weekly competition is now open! Voting closes Friday, June 26 at 11:59 PM AOE.

Check out this week's ideas and upvote the ones that excite you. Submit your own ideas to enter the next round!

This week we found that models treat melodrama as its own "overdone" feature rather than just a lot of drama, while readers mostly disagree by moving a shared cutoff rather than reading along their own features; that a bigger model only lets you skip training when you have almost no data and a simple task, since finetuning's advantage comes from being able to use more examples than a prompt can hold; and that it is the base model, not the reinforcement-trained one, that fixes its answer early and ignores new information, with post-training making models more responsive instead of less.

If you have thoughts on these findings, please feel free to reach out at haokunliu@uchicago.edu. We welcome collaborations and contributions! Check out our NeuriCo repo to see how the experiments are run.

If you are interested in citing this blog, use this bibtex:

@misc{liu-week-of-06-15-2026,
  author = {Liu, Haokun and Peng, Chenxi},
  title = {Week of 06/15/26-06/21/26},
  year = {2026},
  month = {June},
  day = {22},
  url = {https://hypogenic.ai/blog/weekly-entry-260615}
}