Steering Expanded MLP space

0

The intermediate space within MLPs is usually much wider than the residual stream (usually 4x). Can we steer there? Is it easier, because more things are linear or harder because specific MLPs are to specific to certain information that aren't as universal as the residual stream's geometry?

Chat

If you are inspired by this idea, you can reach out to the authors for collaboration or cite it:

@misc{holtzman-steering-expanded-mlp-2026,
  author = {Holtzman, Ari},
  title = {Steering Expanded MLP space},
  year = {2026},
  url = {https://hypogenic.ai/ideahub/idea/cEB7TiKPBevMvWEPjJpP}
}

Comments (0)

Please sign in to comment on this idea.

No comments yet. Be the first to share your thoughts!