 George HotzandGitHub
|
8ee3a37524
|
shrink/pad use (new_shape, offset) (#16405)
* shrink uses offset and shape
* pad does too
* fix
|
2026-05-27 15:13:08 -07:00 |
|
|
wozeparrotandGitHub
|
3a7a6da7d5
|
llama: fakedata uses real vocab size (#16389)
|
2026-05-26 18:58:55 -07:00 |
|
|
qazalandGitHub
|
b73d2d17b9
|
viz/cli: add --interval (#16363)
* interval support
* add test_interval
* llama uses interval
|
2026-05-26 03:35:06 +09:00 |
|
|
wozeparrotandGitHub
|
68d2102fd2
|
llama: offload master weights (#16355)
|
2026-05-25 08:48:13 -07:00 |
|
|
chenyuandGitHub
|
31424cda71
|
Tensor.requires_grad -> is_param (#16325)
for optimizer
|
2026-05-21 19:39:57 -04:00 |
|
|
wozeparrotandGitHub
|
0c41317a59
|
llama: update 405b scripts (#16309)
|
2026-05-21 14:03:34 -07:00 |
|
|
wozeparrotandGitHub
|
fb718a5e9d
|
llama: realize amax (#16308)
|
2026-05-21 14:00:48 -07:00 |
|
|
wozeparrotandGitHub
|
afc5bfa183
|
llama: remove fused grad accum (#16301)
|
2026-05-21 09:38:40 -07:00 |
|
|
qazalandGitHub
|
e33e058d34
|
set SPLIT_W13=0 for 8b DP by default (#16302)
|
2026-05-21 22:09:10 +09:00 |
|
 sirhcmandGitHub
|
172f9493e1
|
move is_dtype_supported to renderer (#16226)
|
2026-05-20 21:19:37 -04:00 |
|
|
wozeparrotandGitHub
|
825f30bf18
|
llama: apply_grad saves memory (#16275)
|
2026-05-20 13:14:06 -07:00 |
|
|
wozeparrotandGitHub
|
361553c0a8
|
llama: match flat_llama with model_train (#16269)
|
2026-05-19 17:25:56 -07:00 |
|
|
wozeparrotandGitHub
|
a3d59faef6
|
llama: don't save weight (#16252)
|
2026-05-18 17:05:45 -07:00 |
|
|
qazalandGitHub
|
18b102f355
|
llama: also use 7.1 comgr, update startup_walltime.sh (#16253)
|
2026-05-19 08:59:02 +09:00 |
|
|
qazalandGitHub
|
98b8a2b407
|
llama: use hipcc 7.1 version (#16250)
|
2026-05-19 08:09:57 +09:00 |
|
|
chenyuandGitHub
|
dcee90aa3f
|
remove requires_grad use in extra/examples (#16238)
except the ones fed into optimizer
|
2026-05-16 18:40:26 -04:00 |
|
|
qazalandGitHub
|
ebcb7b7cc0
|
fp8 gemm tests with scale args (#16231)
* update atol
* update fp8 path
* more work
* update profile.sh
|
2026-05-16 20:47:58 +09:00 |
|
|
wozeparrotandGitHub
|
159694347e
|
llama: fix running flat_llama (#16224)
|
2026-05-15 20:16:48 -07:00 |
|
|
chenyuandGitHub
|
07a172dbbb
|
remove noop requires_grad_ calls (#16213)
|
2026-05-15 13:31:10 -04:00 |
|
|
chenyuandGitHub
|
409bb0c9ad
|
requires_grad cannot be None (#16212)
final goal is to remove requires_grad, first change the default to True, and don't allow None
|
2026-05-15 02:01:04 -04:00 |
|
|
wozeparrotandGitHub
|
b4d267dfd4
|
llama: only save when small (#16208)
|
2026-05-14 17:46:29 -07:00 |
|
|
wozeparrotandGitHub
|
88ac2ac1fd
|
llama: cleanups (#16189)
|
2026-05-13 17:08:06 -07:00 |
|
|
wozeparrotandGitHub
|
e97f2c1114
|
llama: only gemm + fa custom kernel (#16180)
* llama: tie store to grad directly
* llama: set mp flags
* llama: non fused grad fp8 quantize path
|
2026-05-12 21:03:49 -07:00 |
|
|
wozeparrotandGitHub
|
e9359d9e7d
|
more llama mp fixes (#16151)
* llama: SPLIT_W13
* llama: fix with no fused kernels
* llama: cast to bf16 on non asm_gemm patH
* llama: new mp flags
|
2026-05-11 21:29:23 -07:00 |
|
|
wozeparrotandGitHub
|
026688f03f
|
llama: move to correct dir (#16118)
|
2026-05-08 19:42:16 -07:00 |
|
|
qazalandGitHub
|
a9a87ad8fd
|
viz/cli: less flags (#16076)
* viz/cli: merge -s and -i flags
* only -t
* merge parser
* fix
|
2026-05-08 00:22:40 +09:00 |
|
|
wozeparrotandGitHub
|
730fa66bf3
|
llama speed 6 (#16071)
|
2026-05-06 20:51:03 -07:00 |
|
|
wozeparrotandGitHub
|
ab6218bc92
|
llama mp fixes (#16050)
|
2026-05-05 15:35:32 -07:00 |
|
|
wozeparrotandGitHub
|
528d35e306
|
llama speed 4 (#15993)
|
2026-04-30 17:14:41 -07:00 |
|
|
wozeparrotandGitHub
|
0080489abe
|
llama: use env vars (#15978)
|
2026-04-29 12:37:15 -07:00 |
|
|
wozeparrotandGitHub
|
ef09071073
|
llama: speed 2 (#15960)
|
2026-04-28 20:44:37 -07:00 |
|
|
wozeparrotandGitHub
|
5e861cd2c4
|
llama: move llama kernels to llama_kernels (#15952)
|
2026-04-27 22:48:53 -07:00 |
|
|
qazalandGitHub
|
9a23de7d27
|
viz/cli: unify profile and rewrites, -s ALL default (#15931)
* work
* workg
* better
* cleanup
* better defaults
* --ls
* better
* work
* update llama
* update
|
2026-04-25 22:31:24 +09:00 |
|
|
wozeparrotandGitHub
|
4b908b6e2c
|
llama: fused ce loss (#15920)
|
2026-04-24 20:01:24 -07:00 |
|
|
wozeparrotandGitHub
|
9d134a2848
|
llama: fix fakedata timing (#15905)
|
2026-04-23 21:37:03 -07:00 |
|
|
wozeparrotandGitHub
|
d3cbd781d9
|
llama: use fused norm mul quantize for w13 (#15878)
|
2026-04-22 21:27:41 -07:00 |
|
|
wozeparrotandGitHub
|
87378331e8
|
llama: fused mul quantize fp8 (#15863)
|
2026-04-21 20:58:37 -07:00 |
|
|
qazalandGitHub
|
f9655af2a3
|
viz/cli: move to tinygrad (#15835)
* move cli
* update imports
* cleanup the readme
* edit
* work
* details
* python -m tinygrad.viz.cli
* do not execv in non tty
* option
* lint
* simpler
* gemm pmc
|
2026-04-21 13:35:10 +09:00 |
|
|
wozeparrotandGitHub
|
f28ea84de2
|
llama: fused silu fp8 amax (#15798)
* llama: combined w13
* llama: fused swiglu+fp8
* llama: fix amax interleaving
* llama: don't need seperate matmul
|
2026-04-19 12:03:55 +08:00 |
|
|
wozeparrotandGitHub
|
06343092c8
|
llama: combined w13 (#15803)
|
2026-04-17 22:27:31 -07:00 |
|
|
qazalandGitHub
|
a227dbece1
|
viz/cli: reconstruct DEBUG output (#15791)
* work
* work
* ext
* padding
* at time
* work
* reorder
* less flags
* num_rows
* feedback
* pmc
|
2026-04-17 18:27:58 +03:00 |
|
|
wozeparrotandGitHub
|
9e60e4a7e7
|
llama: native fp8 (#15733)
|
2026-04-16 22:16:05 -07:00 |
|
|
wozeparrotandGitHub
|
3721c60bef
|
llama: bs 16 (#15737)
|
2026-04-14 19:52:03 -07:00 |
|
|
wozeparrotandGitHub
|
480ad264a4
|
llama: per device amax (#15735)
|
2026-04-14 19:01:17 -07:00 |
|
|
wozeparrotandGitHub
|
2b8d303f75
|
allreduce in precast dtype (#15689)
|
2026-04-13 20:24:12 -07:00 |
|
|
qazalandGitHub
|
054d78e6ff
|
fix llama profile.sh NULL source (#15685)
|
2026-04-11 22:56:05 +09:00 |
|
|
wozeparrotandGitHub
|
457508d5a0
|
llama: save more 2 (#15681)
|
2026-04-11 01:03:36 -07:00 |
|
|
wozeparrotandGitHub
|
590464c8d8
|
llama: only support wqkv path + cleanups (#15680)
* llama: only support wqkv path + cleanups
* llama: missing transpose
|
2026-04-11 07:39:27 +08:00 |
|
|
wozeparrotandGitHub
|
55bcd7cc9e
|
llama amax outside (#15670)
|
2026-04-09 23:08:03 -07:00 |
|
|
chenyuandGitHub
|
839d37b7bc
|
update median_step_time in model_train.py (#15649)
BENCHMARK=5 used to pick the 4th largest, not the middle one
|
2026-04-08 09:53:59 -04:00 |
|