Is every token its own direction in the residual stream?

3

If a model wants to select a specific token, e.g., because of copying, does that have its own direction in the residual stream? If not how does copying work?

llms mechinterp copying

Chat

If you are inspired by this idea, you can reach out to the authors for collaboration or cite it:

@misc{holtzman-is-every-token-2026,
  author = {Holtzman, Ari},
  title = {Is every token its own direction in the residual stream?},
  year = {2026},
  url = {https://hypogenic.ai/ideahub/idea/ecFTwWyJThovpBT70R5R}
}

Comments (0)

Please sign in to comment on this idea.

No comments yet. Be the first to share your thoughts!