Hypothesis: the first-order issue that's holding back LLM generalization is the fact that they can memorize too fast. Because of this, they don't learn as much as they could from highly similar examples—they factor them out far faster than humans do, making them less data efficient without any fundamental architectural issue.
If you are inspired by this idea, you can reach out to the authors for collaboration or cite it:
@misc{holtzman-llms-wont-learn-2025,
author = {Holtzman, Ari},
title = {LLMs won't learn properly until they forget better},
year = {2025},
url = {https://hypogenic.ai/ideahub/idea/eOqvw2rVUkhSZKPmbyhf}
}Please sign in to comment on this idea.
No comments yet. Be the first to share your thoughts!