How does the Muon-over-AdamW training-speed advantage scale with transformer width at small scale?

Muon is an optimizer for 2-D hidden weight matrices that orthogonalizes each update via a Newton-Schulz iteration; it produces large training-speed gains over AdamW on the modded-nanoGPT benchmark. Whether that advantage grows, shrinks, or stays constant as model size changes is not settled. Question: on a fixed small language-model training task (nanoGPT / modded-nanoGPT style, single GPU, open FineWeb or OpenWebText subset), for a sweep of model widths $d_{\text{model}}\in\{128,256,512,768\}$ at fixed depth and data, measure the number of optimizer steps (a hardware-agnostic quantity) that AdamW versus Muon each need to reach a fixed target validation loss, with each optimizer's learning rate independently tuned per width. Report the speedup ratio $s(d)=\text{steps}_{\text{AdamW}}/\text{steps}_{\text{Muon}}$ as a function of width and whether it trends up, down, or flat.

Acceptance. FULLY RESOLVES: a reproducible script (fixed seeds; $\ge 3$ seeds per config; documented per-config learning-rate tuning) that trains GPT-style models at $\ge 4$ widths with both AdamW and Muon to a fixed target validation loss, reporting steps-to-target with confidence intervals and the fitted trend of $s(d)$ versus width, with a stated verdict (increasing / decreasing / constant, and slope with confidence interval). PARTIAL: two widths with properly tuned baselines. Deliverable: training code + learning-rate sweep logs + steps-to-target table + trend fit.