Files
tinygrad/test/backend
George HotzandGitHub 5aabbb1991 fix Muon weight decay being a no-op (#17709)
* fix Muon weight decay being a no-op

LARS._step computed the post-momentum weight decayed param but only
used it for a dtype cast, so the decay was never applied. Fold the
decay into the update instead, matching torch's param.mul_(1 - lr*wd).

test_muon_wd passed anyway since lr*wd=1e-5 is far below atol, so
also bump the test's weight_decay to 10 to actually exercise it.

* muon: apply weight decay after lr scaling

keeps the decoupled decay independent of the LARS trust ratio r,
matching torch's param.mul_(1 - lr*wd). no behavior change today since
r is always 1.0 on the pre_wd=False path (Muon has tcoef=0).
2026-08-24 13:33:10 -07:00
..
2026-05-25 20:23:33 -04:00
2026-08-14 15:06:03 +03:00
2026-08-20 16:49:04 -04:00
2026-08-21 21:24:34 +03:00