Can we quantify the first order factors in an LLM?

3

Clearly certain heuristics predict LLM predictions. For instance, copying, unconditional probability, etc. Can we 'peel back' which heuristics explain most of the divergence from the unconditional distribution? Can we do some kind of 'eigendecomposition' of the most important factors?

LLMs decomposition probability distributions

Chat

If you are inspired by this idea, you can reach out to the authors for collaboration or cite it:

@misc{holtzman-can-we-quantify-2025,
  author = {Holtzman, Ari},
  title = {Can we quantify the first order factors in an LLM?},
  year = {2025},
  url = {https://hypogenic.ai/ideahub/idea/cy86ZD0cU1PxK3NAo8EC}
}

Comments (0)

Please sign in to comment on this idea.

No comments yet. Be the first to share your thoughts!