Steered Curation with Reference Models: A DRO View of Imperfect Oracles

TL;DR: Ask a good tutor where to focus—imperfect advice can still save time and boost scores. We formalize curation guided by a reference model (model steering) as a DRO objective and analyze how it shifts the less-is-more phase boundaries without changing fundamental exponents. Initial experiment: weight/curate data by disagreement with a strong reference (DRRho-CLIP), compare against unguided curation and knowledge-distillation baselines.

Research Question: How does using a trained reference model to steer data selection/weighting alter the generalization bounds and scaling behavior of curated training?

Hypothesis: Reference-guided curation increases effective data quality and enlarges the parameter regime where small curated datasets dominate, akin to improving the oracle in Dohmatob et al. However, like W2S distillation, it does not improve the asymptotic scaling exponent; it shifts constants and crossover points.

Experiment Plan: Setup: Cast reference-guided selection as DRRho risk minimization (Wei et al.) and analyze its impact on Dohmatob’s label-aware/agnostic curation rules. Derive how the “oracle quality” parameter changes with reference accuracy and disagreement structure.
Data/Tasks: CLIP pretraining (DRRho-CLIP vs CLIP), plus a medium-scale vision classification task where ground-truth labels allow precise evaluation of label-aware vs label-agnostic steering.
Protocol:

1. Train models with: (a) unguided curation; (b) disagreement-based weighting/curation via a reference; (c) knowledge distillation (Ildiz et al.) to contrast label transfer vs data steering.
2. Evaluate scaling curves across dataset sizes and compute budgets; fit exponents and constants; map phase boundaries.
   Analysis: Test if steering improves generalization bounds and yields a superior scaling law constant (Wei et al.), consistent with “better oracle” intuition in Dohmatob et al., but with similar exponents as in distillation theory (Ildiz et al.).
   Expected Outcome: Clear, theory-backed gains from steering in low-to-moderate budgets; constant-factor improvements and delayed crossover to the “more is more” regime.

References: ['Dohmatob, E., Pezeshki, M., & Askari-Hemmat, R. (2025). Why Less is More (Sometimes): A Theory of Data Curation. arXiv.org.', 'Wei, X., Lin, M., Ye, F., Song, F, Cao, L., Thai, M. T., & Yang, T. (2025). Model Steering: Learning with a Reference Model Improves Generalization Bounds and Scaling Laws. arXiv.org.', 'Ildiz, M. E., Gozeten, H. A., Taga, E. O., Mondelli, M., & Oymak, S. (2024). High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws. International Conference on Learning Representations.']