geohot/tinygrad
1
0
Fork 0
forked from tinygrad/tinygrad
Code Pull Requests Actions Activity
14,474 Commits 4 Branches 15 Tags
master
Commit Graph
229 Commits
Author SHA1 Message Date
wozeparrotandGitHub 52596dbf38 gptoss: fused ce (#17672) 2026-08-21 16:19:28 -07:00
wozeparrotandGitHub 72280bb218 gptoss: zero-2 optim (#17581) 2026-08-18 14:28:57 -07:00
qazalandGitHub 2821bd646f late loss.to("CPU") in llama (#17476) 
* late loss.to("CPU") in llama

* acc = 0
 2026-08-10 17:31:50 +09:00
qazalandGitHub f993228b73 llama: accurate mxfp4 mfu (#17388) 
* llama: accurate mxfp4 mfu

* train_llama3 import
 2026-08-04 18:41:57 +09:00
qazalandGitHub be5f62d269 llama: refactor amax stuff and skip in fp4 (#17375) 2026-08-03 20:05:52 +09:00
wozeparrotandGitHub 4f5cadd15d gptoss ci (#17325) 2026-07-31 05:51:58 -07:00
wozeparrotandGitHub 93b74c75fc gptoss: grouped moe (#17322) 2026-07-31 03:25:32 -07:00
wozeparrotandGitHub 056974468e gptoss: split no-wd params (#17233) 2026-07-27 02:58:43 -07:00
wozeparrotandGitHub f0117e98df refactor mlperf optim (#17200) 2026-07-25 10:37:11 -07:00
qazalandGitHub 1cf8f2f68c llama: inplace amax update (#17064) 
* llama: inplace amax update

* remove amax_out return

* work

* fit

* work

* work

* keep

* diff cleanup
 2026-07-20 15:05:41 +09:00
wozeparrotandGitHub 06169f5013 gptoss: small fixes (#17025) 2026-07-14 20:40:23 -07:00
qazalandGitHub cae6696d75 llama: split current and next amax state (#16993) 2026-07-12 18:52:33 +09:00
George HotzandGitHub fdffc6c0c8 remove dtypes base (#16931) 
* remove dtypes base

* find/replace bug

* vcount is junk too
 2026-07-08 11:48:11 -07:00
wozeparrotandGitHub a805ce03b1 gptoss: model train (#16884) 2026-07-06 22:07:50 -07:00
chenyuandGitHub 36c0e81979 deprecate Tensor.training [PR] (#16774) 2026-06-27 11:02:28 -04:00
chenyuandGitHub 33b635d23a Tensor.train -> TRAINING [PR] (#16705) 
* Tensor.train -> TRAINING [PR]

* doc
 2026-06-22 15:13:22 -04:00
wozeparrotandGitHub e770805d21 llama: mxfp8 (#16574) 2026-06-11 22:15:24 -07:00
wozeparrotandGitHub fd13080636 deviceless const skip axis check (#16496) 2026-06-03 19:13:20 -07:00
wozeparrotandGitHub 7dcfd144b6 llama: columnwise fp8 scaling (#16480) 2026-06-02 18:55:45 -07:00
wozeparrotandGitHub 6795c2d5c9 llama: zero grad this way (#16445) 2026-05-29 20:25:21 -07:00
wozeparrotandGitHub c23652e486 llama: minimize peak init mem (#16440) 2026-05-29 18:00:37 -07:00
wozeparrotandGitHub 36c8ff70c1 llama: use old scale for dequant in optim (#16417) 2026-05-28 15:21:19 -07:00
wozeparrotandGitHub 3a7a6da7d5 llama: fakedata uses real vocab size (#16389) 2026-05-26 18:58:55 -07:00
wozeparrotandGitHub 68d2102fd2 llama: offload master weights (#16355) 2026-05-25 08:48:13 -07:00
chenyuandGitHub 31424cda71 Tensor.requires_grad -> is_param (#16325) 
for optimizer
 2026-05-21 19:39:57 -04:00
chenyuandGitHub dcee90aa3f remove requires_grad use in extra/examples (#16238) 
except the ones fed into optimizer
 2026-05-16 18:40:26 -04:00
wozeparrotandGitHub b4d267dfd4 llama: only save when small (#16208) 2026-05-14 17:46:29 -07:00
wozeparrotandGitHub 88ac2ac1fd llama: cleanups (#16189) 2026-05-13 17:08:06 -07:00
wozeparrotandGitHub ab6218bc92 llama mp fixes (#16050) 2026-05-05 15:35:32 -07:00
wozeparrotandGitHub 528d35e306 llama speed 4 (#15993) 2026-04-30 17:14:41 -07:00
wozeparrotandGitHub ef09071073 llama: speed 2 (#15960) 2026-04-28 20:44:37 -07:00
wozeparrotandGitHub 5e861cd2c4 llama: move llama kernels to llama_kernels (#15952) 2026-04-27 22:48:53 -07:00
wozeparrotandGitHub 4b908b6e2c llama: fused ce loss (#15920) 2026-04-24 20:01:24 -07:00
wozeparrotandGitHub 9d134a2848 llama: fix fakedata timing (#15905) 2026-04-23 21:37:03 -07:00
wozeparrotandGitHub 06343092c8 llama: combined w13 (#15803) 2026-04-17 22:27:31 -07:00
wozeparrotandGitHub 9e60e4a7e7 llama: native fp8 (#15733) 2026-04-16 22:16:05 -07:00
chenyuandGitHub 839d37b7bc update median_step_time in model_train.py (#15649) 
BENCHMARK=5 used to pick the 4th largest, not the middle one
 2026-04-08 09:53:59 -04:00
wozeparrotandGitHub 70dbd35023 llama: move custom_kernel into flat_llama (#15643) 2026-04-08 00:19:14 -07:00
wozeparrotGitHubqazal
 		7e54992bf6 fp8 llama (#15588) 
Co-authored-by: qazal <[email protected]>
 2026-04-04 18:24:57 -07:00
qazalandGitHub 09f60d80fd llama: fix FP8=1 FAKEDATA=1 (#15564) 2026-04-01 20:53:03 +09:00
wozeparrotandGitHub 0c3e438229 llama: mllog (#15502) 2026-03-28 11:18:25 -07:00
wozeparrotandGitHub a65e958be9 llama: new apply_grad (#15503) 2026-03-26 19:39:25 -07:00
sirhcmandGitHub bc180a963c deprecate <dev>=1 in favor of DEV=<dev> (#15467) 
* start work on target

* add test

* update actions to use DEV

* update docs

* update readmes

* tests need that too

* update example

* update tests (comments)

* fix that test

* ruff

* mypy

* oops

* remove getenvs

* don't add Target yet

* and the test

* lint

* and docs

* more stuff

* assert

* few more fixes

* test assert
 2026-03-26 03:48:03 -04:00
wozeparrotandGitHub da2031266a llama: correct 8b init (#15397) 2026-03-24 13:41:41 -07:00
wozeparrotandGitHub 87c4ec1724 llama: use flat llama (#15353) 2026-03-19 22:12:38 -07:00
wozeparrotandGitHub a191ac0566 llama: use mlperf model (#15257) 2026-03-13 08:08:32 -07:00
wozeparrotandGitHub 4fab320abe llama: clean (#15224) 2026-03-11 13:33:59 -07:00
wozeparrotandGitHub 05d6d9120a llama offload null (#15222) 2026-03-11 10:04:31 -07:00
wozeparrotandGitHub 525a178966 llama: jit more (#15199) 2026-03-10 11:04:59 +08:00
wozeparrotandGitHub 4544da1c54 llama3 fixes part3 (#15152) 2026-03-05 01:17:54 -08:00