If you just push a bunch of data through LLMs and you do PCA on each of their hidden states at every layer, what do you learn? What are the most important dimensions and what do they explain?
If you are inspired by this idea, you can reach out to the authors for collaboration or cite it:
@misc{holtzman-pca-on-hidden-2025,
author = {Holtzman, Ari},
title = {PCA on Hidden States},
year = {2025},
url = {https://hypogenic.ai/ideahub/idea/fqDdGRj7AdJLcwW6tQl1}
}Please sign in to comment on this idea.
No comments yet. Be the first to share your thoughts!