Low-Variation Extrapolation

Let's say you visualized text in a 2D PCA. In that space, LLMs would probably generate far away from any kind of boundary you could draw around 'likely' or 'reasonable' text. But my guess is that way down the PCA vectors, LLMs would be more willing to cross that boundary. Thus, I think LLM extrapolation largely comes from low-variation dimensions of text. We can get them to combine incongruent high-variation dimensions (e.g. style and formalities that don't work together normally), if we force them, but they'll tend to mix and match choppily, integrating stereotypical aspects of both. True extrapolation, I predict, comes from low-volume regions where LLMs naturally must extrapolate.