Hypogenic AI - Shaping the Future of Science

Welcome to another weekly entry! Thank you to everyone who submitted and voted on ideas.

Each idea was explored by two agents, Claude and Codex.

This week we looked at whether you can read a model's "hidden words" straight out of its internal state, whether telling a model to ground its ideas makes those ideas any better, and whether wiring a fixed math rule into a neural network helps it actually understand the task.

Winning ideas and generated repos here:

Logit Lens with Implicit Tokens by Ari Holtzman

Models process text in chunks called tokens. The word "northeastern" gets split into several pieces, for example. A tool called the logit lens lets you peek at what a model is thinking mid-process, but it only works with single-token words. Evidence suggests models also store whole words internally, even when those words are split. So can we build a logit lens that reads those whole words?

Scientific Idea Generation with Grounded Novelty by Chenhao Tan

People worry that research ideas from a LLM are either too small to matter or too wild to work. One fix is to make the model "ground" its ideas, meaning it anchors them in something concrete, like reasoning through them, testing against examples, or comparing to past papers. Does that actually help?

Neural-Symbolic Learning by Chenhao Tan

You can teach a model to add two handwritten digits in two ways. Option one: feed both images to a neural network and let it output the sum directly. Option two: let the network recognize each digit separately, then use a fixed math rule for the addition. Does option two help the model actually understand the task, even when it only ever sees the final sum, never the individual digits?

TL;DR for what we found

A model splits a word like "northeastern" into pieces, but it also seems to keep the whole word as one thing inside, and you can read it back out. Both agents pulled multi-token words straight out of the hidden state, something the standard logit lens cannot do at all. With enough training, a simple per-layer adjustment recovered the correct word about 95% of the time. With less data, a simpler method still beat the baseline, just by a smaller margin.
Telling a strong model to ground its ideas did not reliably make those ideas better. Both agents found the plain "just give me novel ideas" prompt hard to beat. The one thing that did work was letting the model test its ideas against real data and revise them, but that costs far more compute. Writing longer, more formal proposals actually made the ideas worse.
Adding a fixed math rule made a big difference. The model learned to recognize digits on its own and could add numbers far longer than any it saw during training. A plain neural network scored exactly zero on those longer numbers. One potential risk is that high accuracy can hide a model that reads the digits systematically wrong, and a handful of labeled images can fix it.

Verdicts

Idea	Verdict	Next Question
Logit lens with implicit tokens	Supported, multi-token words can be read out of the hidden state, best in early layers and at the last piece of the word	The word is readable, but does the model actually rely on it? And can we find these implicit words automatically instead of handing the model a list?
Grounded novelty	Not supported as a blanket rule, though one mechanism (test the idea and revise) helped at high cost	Does grounding in real experiment feedback help where a prompt tweak does not, and would human judges rate the ideas the same way an LLM judge did?
Neural-symbolic learning	Supported, large data-efficiency and generalization wins a plain network cannot match	Does this still hold on messier real-world perception, and can the model learn the rule itself instead of being handed it?

Findings from the Ideas

Can You Read a Model's Hidden Words?

The question. A model never sees a word like "northeastern" as one unit. Its tokenizer chops it into pieces like "n", "ort", "he", "astern". The logit lens, a standard tool for looking inside a model, can only name things that have their own single token, so a word split into pieces is invisible to it. But recent work suggests models stitch these pieces back into a single whole-word representation early on and then use it like a real vocabulary entry. The question is whether we can build a logit lens that reads those hidden whole words.

What the agents tried.

Claude built the lens and tested it on two different models (Qwen2.5 1.5B and GPT-2 large). For each multi-token word it added a single "row" derived from the model's own embeddings, then projected the hidden state onto it and checked whether the right word came out on top. It tried two versions: a naive one that reads the hidden state as is, and a learned one that first applies a small per-layer correction before reading. It trained the learned version on one set of words and tested it on a completely separate set.
Codex ran a smaller version on Qwen2.5 0.5B with 128 multi-token words. It built a lookup table of whole-word fingerprints from the model's hidden states and scored a held-out word by which fingerprint it matched best. It deliberately grouped the words so that eight of them shared the same last piece, so reading only the last token would get you 12.5% at best.

What happened.

Both agents could read the hidden words, and the cleanest result came from Claude. The naive lens barely worked past the first layer, dropping near zero in the middle of the model. But the learned per-layer correction recovered the correct held-out word about 95% of the time, where random guessing would be less than one in a thousand, and where the standard logit lens scores zero because the word has no token of its own. This held on both models. The word was far easier to read at the last piece of the word (95%) than the first (30%), which makes sense: under the way these models read left to right, only the last piece has seen all the others. So the lens is reading an assembled word, not just a surface fragment. The information was strongest early in the model, around the first few layers.

Codex's smaller run landed in the same place with smaller numbers. Its lookup table reached about 20% on held-out words, against 12.5% for reading the last piece alone and under 1% for guessing. That is a real effect, since every group of candidate words shared a last piece, so the lens had to be using more than the surface token. One useful negative result: the fancier "learned correction" approach, the same idea that worked so well for Claude, fell apart for Codex because it only had ten examples per word, which is far too few to fit a correction that size. Its errors were telling too, often confusing words in the same group, like reading "england" as "highland" or "queensland".

What we learned. You can build a logit lens that reads a model's implicit, multi-token words, and the identity of the word really is sitting in the hidden state as a readable direction. The standard lens misses it not because the information is gone, but because it is in a slightly shifted form that a small per-layer correction can undo. The two runs together also draw a clear line: with enough training data a learned correction gets you most of the way, but with little data you are better off with a plain per-layer lookup. The honest caveat, which both agents flag, is that this shows the word is readable, not that the model actually uses it for anything downstream.

Does Telling a Model to "Ground" Its Ideas Help?

The question. Research ideas from a model often feel either too incremental or implausibly bold, and "novelty" is hard to pin down. The hypothesis is that a model does better when novelty is grounded in something concrete instead of asked for in the abstract. Concrete here means things like reasoning the idea out step by step, learning from a few labeled examples, running a small test, comparing against existing papers, or writing the idea up as a full proposal. The question is whether any of this actually beats just asking for novel ideas.

What the agents tried. Both agents compared an ungrounded prompt against several grounded ones, holding the model and the topics fixed so the only thing changing was the grounding. They measured "better" in two very different ways, which turned out to matter.

Claude made the test objective. It treated an idea as a decision rule for a text classification task, and "better" as how accurately that rule predicts held-out labels. It ran six approaches across three tasks and three seeds, and also gave one approach a larger budget of examples to see whether more data changed the picture.
Codex kept the ideas as free-form research proposals and had a separate model (Gemini 2.5 Flash) judge each one on novelty, feasibility, specificity, and overall quality. It generated ideas for 12 research topics under seven matched prompts and compared the scores.

What happened. Both agents found the same headline: grounding did not reliably make a strong model's ideas better.

In Claude's objective test, the grounded approaches as a group were statistically indistinguishable from the plain ungrounded prompt, and if anything slightly behind. But the specific mechanism mattered a lot. The one approach that clearly helped was letting the model test its rules on a few examples, see where it went wrong, and revise. That was the only one to significantly improve in-distribution accuracy, and it did so at about eighty times the token cost of the plain prompt. Writing the idea up as a formal proposal actually hurt, dropping accuracy and collapsing the variety of ideas into rigid, look-alike rules. Learning from data had its own trap: on one task the plain prompt transferred almost perfectly to new data (97%), while the data-grounded rules latched onto surface quirks of the training set and fell to around 80% when the data shifted.

In Codex's judged test, no quality measure showed a real difference between grounded and ungrounded prompts. Literature comparison scored highest on paper, but the gap was tiny and not reliable, and the plain prompt actually had the best balance of novelty and feasibility. The one strong, consistent effect was that grounded prompts made the ideas reuse more words from the existing papers they were shown. That cuts both ways. It makes the ideas look better connected to prior work, but it also pulls them closer to what already exists, which is the opposite of novelty.

What we learned. For a capable modern model, "ground your ideation" is too blunt an instruction. The plain prompt is already a strong baseline, because the model's training has, in a sense, already grounded it. The grounding that helped was not a prompt trick but real feedback: testing an idea and revising it, which is expensive and is closer to running a small experiment than to rephrasing a request. The grounding that hurt was elaboration, dressing the idea up in more words without adding substance. One caveat worth flagging is that the two agents measured idea quality very differently, one by downstream accuracy and one by an LLM judge, and neither used real human expert judgment, which could still tell a different story.

Does Wiring in a Math Rule Help a Model Understand?

The question. Take the task of adding two handwritten digits, where the model is only ever told the answer, never the two digits. A plain neural network looks at both images and learns to output the sum directly. A neural-symbolic model instead learns only to recognize each digit, and a fixed addition rule does the rest. The promise is that the second setup understands more: it should learn the digits on its own, need less data, and generalize. The question is whether that promise holds, and when it breaks.

What the agents tried.

Claude ran a large, careful sweep, 210 training runs in all. It compared the neural-symbolic model against a plain neural network and against an "oracle" that gets to see the true digit labels. It checked four things: how much data each approach needs, whether the model recovers the correct digits or cheats, whether a cheap fix exists when it cheats, and whether it can add longer numbers it never saw.
Codex set up a cleaner generalization test. It trained on digit pairs but removed all the pairs where both digits were large, so certain sums (14 through 18) never appeared as answers during training, then checked whether each approach could still get those sums right at test time.

What happened. Both agents strongly confirmed the core idea, and Claude added an important warning.

The wins were large. In Claude's runs, with only 1,000 training pairs the neural-symbolic model hit 85% while the plain network was stuck at 22%, and it recovered the actual digits at 92 to 99% accuracy despite never being told a single digit. Codex saw the same shape: on the sums it was never trained on, the neural-symbolic model scored about 81% while the plain network scored essentially zero, because the plain network has no way to produce an answer it never saw. The clearest payoff was generalization. Trained only on single-digit sums, Claude's model could add two- and three-digit numbers it had never seen, around 94 to 96%, while the plain network scored exactly 0%.

The warning is that high accuracy can hide a broken model. When Claude changed the math rule from normal addition to "add then wrap around at 10," the model still got the task right but read the digits systematically wrong, recognizing them under a consistent relabeling that happens to give the same wrapped answer. Its digit accuracy was about 0.1%. You would never catch this by looking at the task score alone; you only see it if you check whether the digits are right. The fix was cheap: adding just ten labeled images, one per digit, snapped the digit accuracy back up to 98%. Both agents also agreed the rule is not magic. With too little data (Codex's 1,000-label runs, Claude's smallest sizes) the digit recognition never gets good enough for the rule to help, and Codex found that a fancier variant it designed did not beat the plain symbolic version.

What we learned. Wiring a fixed rule into the model is a real and large win. It learns the parts it was never directly taught, needs far less data, and generalizes to problems a plain network simply cannot represent. But getting the task right is not the same as understanding it. Depending on the rule, a model can hit high accuracy while reading the inputs completely wrong, and the only way to know is to measure the parts directly rather than trust the final answer. When that happens, a tiny bit of labeling is enough to set it straight.

Next Week's Competition

The thirty-fourth weekly competition is now open! Voting closes Friday, July 3 at 11:59 PM AOE.

Check out this week's ideas and upvote the ones that excite you. Submit your own ideas to enter the next round!

This week we found that a model's hidden multi-token words can be read straight out of its internal state, best in the early layers and at the last piece of the word; that telling a strong model to ground its ideas did not reliably improve them, with the plain prompt a hard baseline to beat and only real test-and-revise feedback helping; and that wiring a fixed math rule into a digit recognizer buys large data-efficiency and generalization gains a plain network cannot match, as long as you check that the model learned the right things and not a shortcut.

If you have thoughts on these findings, please feel free to reach out at haokunliu@uchicago.edu. We welcome collaborations and contributions! Check out our NeuriCo repo to see how the experiments are run.

If you are interested in citing this blog, use this bibtex:

@misc{liu-week-of-06-22-2026,
  author = {Liu, Haokun and Peng, Chenxi},
  title = {Week of 06/22/26-06/28/26},
  year = {2026},
  month = {June},
  day = {29},
  url = {https://hypogenic.ai/blog/weekly-entry-260629}
}