sirhcm/tinygrad
1
0
Fork 0
forked from tinygrad/tinygrad
Code Pull Requests 2 Actions 8 Activity
13,700 Commits 336 Branches 15 Tags
54887cbbdfdcd203b431702dcb384dbb828304b6
Commit Graph
1411 Commits
Author SHA1 Message Date
qazalandGitHub cb9df8bd33 fix profile.sh BENCHMARK=3 exit status (#16736) 2026-06-25 13:41:03 +09:00
qazalandGitHub 535c806c9f llama: profile.sh BENCHMARK=3 (#16722) 2026-06-24 19:20:51 +09:00
wozeparrotandGitHub dfea9e7994 llama: fused silu mul quantize mxfp8 (#16704) 2026-06-23 16:59:50 -07:00
chenyuandGitHub 33b635d23a Tensor.train -> TRAINING [PR] (#16705) 
* Tensor.train -> TRAINING [PR]

* doc
 2026-06-22 15:13:22 -04:00
wozeparrotandGitHub fe9b19b12d llama: more mp mem fixes (#16701) 
* llama: more mp mem fixes

* clean: unused

* fix: batch
 2026-06-22 10:54:35 -04:00
wozeparrotandGitHub 70cac72781 llama: realize weight init (#16623) 2026-06-15 23:00:19 -07:00
qazalandGitHub f998b9930a fp8 gemm inv_scale in epilogue (#16625) 
* fuse scale

* remove python inv_scale

* more inv_scale removal

* more cleanups

* cleaner

* diff polish

* work

* rename

* simpler

* simpler

* compute

* c

* Revert "c"

This reverts commit 8941fec7ca.

* Revert "compute"

This reverts commit 9db573a6d3.

* Revert "simpler"

This reverts commit 910ad33f87.

* Revert "simpler"

This reverts commit bf75d235a1.

* s_g

* update types

* less diff noise

* remove
 2026-06-15 18:44:41 +09:00
qazalandGitHub 4d34590b7d llama: less E kernels (#16517) 2026-06-12 19:49:25 +09:00
wozeparrotandGitHub e770805d21 llama: mxfp8 (#16574) 2026-06-11 22:15:24 -07:00
wozeparrotandGitHub f11f63007d llama: immediate scaling on flag (#16494) 2026-06-04 10:30:00 -07:00
chenyuandGitHub 0faaf6df26 remove kwargs from arange and linspace [PR] (#16505) 
it used to have requires_grad and device, now both are removed
 2026-06-04 10:32:37 -04:00
qazalandGitHub 3b1a5f9770 llama: a_bT and aT_b bf16 gemms (#16487) 
* hk_bf16_gemm

* enable in 8b

* cleanups

* rename to USE_HK_BF16_GEMM

* work

* work

* work

* work

* change the gemms

* work

* work

* set as default

* work

* change
 2026-06-04 23:30:21 +09:00
wozeparrotandGitHub fd13080636 deviceless const skip axis check (#16496) 2026-06-03 19:13:20 -07:00
wozeparrotandGitHub 7dcfd144b6 llama: columnwise fp8 scaling (#16480) 2026-06-02 18:55:45 -07:00
wozeparrotandGitHub aab9a5a8a3 llama: allow specifying layer count (#16464) 2026-06-01 15:36:04 -07:00
wozeparrotandGitHub 6795c2d5c9 llama: zero grad this way (#16445) 2026-05-29 20:25:21 -07:00
wozeparrotandGitHub c23652e486 llama: minimize peak init mem (#16440) 2026-05-29 18:00:37 -07:00
wozeparrotandGitHub 6787de9f52 llama: fix mp (#16434) 2026-05-29 11:21:43 -07:00
wozeparrotandGitHub f86966af56 llama: optim amax margin (#16425) 2026-05-28 20:18:11 -07:00
wozeparrotandGitHub 36c8ff70c1 llama: use old scale for dequant in optim (#16417) 2026-05-28 15:21:19 -07:00
George HotzandGitHub edca5df25a flip offset and shape in pad and shrink (#16414) 
* flip offset and shape in pad and shrink

* dumb test
 2026-05-28 11:58:19 -07:00
wozeparrotandGitHub dac3743d75 llama: delayed scaling in optim (#16407) 2026-05-27 15:40:03 -07:00
George HotzandGitHub 8ee3a37524 shrink/pad use (new_shape, offset) (#16405) 
* shrink uses offset and shape

* pad does too

* fix
 2026-05-27 15:13:08 -07:00
wozeparrotandGitHub 3a7a6da7d5 llama: fakedata uses real vocab size (#16389) 2026-05-26 18:58:55 -07:00
qazalandGitHub b73d2d17b9 viz/cli: add --interval (#16363) 
* interval support

* add test_interval

* llama uses interval
 2026-05-26 03:35:06 +09:00
wozeparrotandGitHub 68d2102fd2 llama: offload master weights (#16355) 2026-05-25 08:48:13 -07:00
sirhcmandGitHub c2d06570a5 remove getenv(CI) from core tinygrad (#16326) 2026-05-21 22:20:33 -04:00
chenyuandGitHub 31424cda71 Tensor.requires_grad -> is_param (#16325) 
for optimizer
 2026-05-21 19:39:57 -04:00
wozeparrotandGitHub 0c41317a59 llama: update 405b scripts (#16309) 2026-05-21 14:03:34 -07:00
wozeparrotandGitHub fb718a5e9d llama: realize amax (#16308) 2026-05-21 14:00:48 -07:00
wozeparrotandGitHub afc5bfa183 llama: remove fused grad accum (#16301) 2026-05-21 09:38:40 -07:00
qazalandGitHub e33e058d34 set SPLIT_W13=0 for 8b DP by default (#16302) 2026-05-21 22:09:10 +09:00
sirhcmandGitHub 172f9493e1 move is_dtype_supported to renderer (#16226) 2026-05-20 21:19:37 -04:00
George HotzandGitHub 58d58c1659 remove DEVECTORIZE (#16290) 
* remove DEVECTORIZE

* fully remove DEVECTORIZE
 2026-05-20 13:25:49 -07:00
wozeparrotandGitHub 825f30bf18 llama: apply_grad saves memory (#16275) 2026-05-20 13:14:06 -07:00
chenyuandGitHub 7af7b6703a relax policy ASSERT_MIN_STEP_TIME to 3.2 (#16273) 2026-05-19 22:29:09 -04:00
wozeparrotandGitHub 361553c0a8 llama: match flat_llama with model_train (#16269) 2026-05-19 17:25:56 -07:00
George HotzandGitHub da7414d6dc fix RUN_PICKLE and test it (#16272) 
* add test for openpilot RUN_PICKLE

* fix RUN_PICKLE and test it
 2026-05-19 17:00:25 -07:00
wozeparrotandGitHub a3d59faef6 llama: don't save weight (#16252) 2026-05-18 17:05:45 -07:00
qazalandGitHub 18b102f355 llama: also use 7.1 comgr, update startup_walltime.sh (#16253) 2026-05-19 08:59:02 +09:00
qazalandGitHub 98b8a2b407 llama: use hipcc 7.1 version (#16250) 2026-05-19 08:09:57 +09:00
chenyuandGitHub dcee90aa3f remove requires_grad use in extra/examples (#16238) 
except the ones fed into optimizer
 2026-05-16 18:40:26 -04:00
qazalandGitHub ebcb7b7cc0 fp8 gemm tests with scale args (#16231) 
* update atol

* update fp8 path

* more work

* update profile.sh
 2026-05-16 20:47:58 +09:00
wozeparrotandGitHub 159694347e llama: fix running flat_llama (#16224) 2026-05-15 20:16:48 -07:00
George HotzandGitHub 2549b14ec2 fix caformer onnx run (#16222) 2026-05-15 15:08:36 -07:00
chenyuandGitHub 07a172dbbb remove noop requires_grad_ calls (#16213) 2026-05-15 13:31:10 -04:00
chenyuandGitHub 409bb0c9ad requires_grad cannot be None (#16212) 
final goal is to remove requires_grad, first change the default to True, and don't allow None
 2026-05-15 02:01:04 -04:00
wozeparrotandGitHub b4d267dfd4 llama: only save when small (#16208) 2026-05-14 17:46:29 -07:00
wozeparrotandGitHub 88ac2ac1fd llama: cleanups (#16189) 2026-05-13 17:08:06 -07:00
wozeparrotandGitHub e97f2c1114 llama: only gemm + fa custom kernel (#16180) 
* llama: tie store to grad directly

* llama: set mp flags

* llama: non fused grad fp8 quantize path
 2026-05-12 21:03:49 -07:00