It seems like qwen models are surprisingly bad for synthetic data because they output rigid, near-identical results across very different seed data (https://huggingface.co/spaces/HuggingFaceFW/finephrase#math-rephrasing-when-worse-outputs-win). What if we tried to go even further to minimize this downside by generating synthetic data with base models with teacher forced formatting e.g. [seed text] Thanks! Here's a summary of that: [model generation] <eos>
If you are inspired by this idea, you can reach out to the authors for collaboration or cite it:
@misc{muchane-synthetic-data-with-2026,
author = {Muchane, Mark},
title = {Synthetic Data with Base Models},
year = {2026},
url = {https://hypogenic.ai/ideahub/idea/Jb0jWzi6s32RJqppP5Hy}
}Please sign in to comment on this idea.
No comments yet. Be the first to share your thoughts!