Truly Testing LLM Impersonators

by Ari Holtzman6 months ago
2

Aligned models maintain much of their aligned style even after finetuning. If you finetune a base model on a large portion of a specific person's data, how predictive is it? Perplexity is not a good measure here, as it will mostly be descriptive of style. But I think it's interesting to ask if LLMs really could guess how people would react in novel situations. Too bad we don't have Diary Corpus. Yet.

If you are inspired by this idea, you can reach out to the authors for collaboration or cite it:

@misc{holtzman-truly-testing-llm-2025,
  author = {Holtzman, Ari},
  title = {Truly Testing LLM Impersonators},
  year = {2025},
  url = {https://hypogenic.ai/ideahub/idea/92PXYtQ62DMJoA2nBzpU}
}

Comments (0)

Please sign in to comment on this idea.

No comments yet. Be the first to share your thoughts!