Research & academia
Scaling Domain Data Repetition in LLM Pretraining
The authors study data repetition as model size and token budgets scale, reporting that optimal repetition relates strongly to domain validation loss.
Why it matters: Smaller proxy models may help estimate larger training-run repetition choices.