Does the data-repetition decay constant of Chinchilla-style scaling differ between code and natural language at small scale?

In the data-constrained regime, Muennighoff et al. found that repeating training data for up to about 4 epochs is nearly as valuable as fresh data, after which the value of repeated tokens decays; they fit a decay parameter (a repetition 'half-life' $R_D^\star$, on the order of ~15 repetitions) governing how fast repeated tokens lose value. This constant was fit primarily on English web text. Question: is $R_D^\star$ corpus-dependent? Concretely, train small language models ($\le 160$M parameters) under a fixed compute budget on a fixed unique-token budget drawn from (a) an open natural-language corpus (e.g. a C4 or SlimPajama subset) and (b) an open code corpus (e.g. a subset of The Stack), each repeated for $1,2,4,8,16,\dots$ epochs; fit the Muennighoff data-constrained scaling form; and compare the fitted decay constants. Report $R_D^\star$ (with confidence interval) for each corpus and whether they differ significantly.

Acceptance. FULLY RESOLVES: a reproducible training + fitting pipeline (fixed seeds; pinned tokenizer; $\ge 3$ seeds per repetition count) that fits the data-constrained scaling form separately for a code corpus and a natural-language corpus at matched unique-token budgets, and reports the fitted decay constants with confidence intervals plus a significance test for their difference. PARTIAL: a single-corpus fit that reproduces the ~4-epoch plateau at small scale. Deliverable: training code + loss curves + fitted parameters + confidence intervals.