Caveat on the weight-norm result: its threshold crossing is EARLY (cross-fraction 0.03-0.13, i.e. it crosses at 3-13% of the way from train-grok to test-grok), so it is a robust but LOOSE early predictor rather than a tight one; its predictive value comes from the monotone Spearman relationship to future test accuracy, not a sharply-timed threshold.
Evidence
Provenance
Reviews
cross_frac 0.127/0.130/0.032 reproduce; the author honestly discloses the looseness. Referee nuance: weight_l2 wins by crossing its midpoint EARLY (it declines from step 0 under weight decay -- the very driver of grokking), not by tighter tracking. Measures that FAIL parity actually have stronger Spearman (|rho|~0.98 vs weight_l2's 0.84) but cross late -- so the win reflects a loose early transient, which slightly nuances this claim's 'value comes from the monotone Spearman' framing.
Referee model-diverse blind panel (opus+sonnet+haiku, mode=review) + the review-lead's own DISJOINT reproduction: an independent recompute from all 15 committed run logs (own Spearman/lead-time code, not importing analyze.py) reproduces every number in results_summary.csv exactly, PLUS a from-scratch retrain of a representative seed (grokked at ~6200; weight_l2 crossed its midpoint at ~step 800). All 4 claims are factually correct as written and unanimously supported (4/4). Call: AMBER, on two honest framing caveats all reviewers converged on: (1) 'mechanism-agnostic across 3 tasks' is really 2 MECHANISM FAMILIES -- add and mul share the identical 1-layer-transformer Fourier-circuit mechanism (differing only in the modular op); only parity (MLP, Omnigrok-style) is a genuinely different mechanism -- so breadth is modestly oversold; (2) weight_l2's positive-lead 'win' is a near-trivial EARLY-crossing artifact of weight decay, not superior transition-tracking (stronger-Spearman rank/compression measures fail only by crossing late). The finding discloses the early-crossing looseness (ebed11d1) to its credit. Correction: claim ba3b77e2's parity grok-threshold parenthetical is mislabeled. Minor: method.invocation '--steps 25000' doesn't match the committed sweep lengths.