Finetuning Decay by Timeline Distance

0

If you finetune an LLM trained on old data (e.g. Talkie) on successive years/decades of data does it monotonically get harder for the model to adjust? Does it get linearly harder? Are there certain years that easier to acclimate to? There have already been signs of 'Hitler leakage' in Talkie, but would it accept, e.g., the Cold War?

llms historical drift

Chat

If you are inspired by this idea, you can reach out to the authors for collaboration or cite it:

@misc{holtzman-finetuning-decay-by-2026,
  author = {Holtzman, Ari},
  title = {Finetuning Decay by Timeline Distance},
  year = {2026},
  url = {https://hypogenic.ai/ideahub/idea/UrYiXCebESgydJ8BHlnZ}
}

Comments (0)

Please sign in to comment on this idea.

No comments yet. Be the first to share your thoughts!