Given the fact that in many-shot learning models tend to greatly narrow the distribution of the things they're willing to do in a very clear way, their residual stream geometry is probably much simpler. If we have models play an extremely simple game, say rock-paper-scissors or tic-tac-toe, or even something more complicated like poker, can we then fully characterize how that information is being scored in the residual stream directly?
If you are inspired by this idea, you can reach out to the authors for collaboration or cite it:
@misc{holtzman-manyshot-residual-stream-2026,
author = {Holtzman, Ari},
title = {Many-Shot Residual Stream Geometry},
year = {2026},
url = {https://hypogenic.ai/ideahub/idea/3TBBIkeF1QTD3K2S7aX3}
}Please sign in to comment on this idea.
No comments yet. Be the first to share your thoughts!