sirhcm/tinygrad
1
0
Fork 0
forked from tinygrad/tinygrad
Code Pull Requests 2 Actions 4 Activity
14,300 Commits 336 Branches 15 Tags
simple_rangeify
Commit Graph
33 Commits
Author SHA1 Message Date
qazalandGitHub 44f1f45cd5 llama: custom silu kernels (#17462) 
* start by copying the C

* uop kernel

* cleanup tests

* estimates is part of SPEC
 2026-08-10 16:43:01 +09:00
qazalandGitHub f258708d7d llama: custom quantize_mxfp4+transpose kernel (codex) (#17434) 
* llama: custom quantize_mxfp4+transpose kernel (codex)

* rename to cpp

* inline

* cleanup

* lds load_bf16x4

* more tests, add Estimates
 2026-08-06 16:13:28 +09:00
chenyuandGitHub 8e2f175542 const(dtype, b) -> const(b, dtype) [PR] (#17328) 
prep for dtype removal
 2026-07-31 09:46:37 -04:00
George HotzandGitHub 52c9e5a99e rename LOOP -> WEAK and STRONGLOOP -> LOOP (#17283) 2026-07-29 10:38:36 -07:00
George HotzandGitHub 451120c6e1 make .barrier implicit (kimi) (#17275) 
* make .barrier implicit (kimi)

* simplier

* lil

* remove tinygrad stock barriers

* readable

* lil
 2026-07-28 22:34:57 -07:00
George HotzandGitHub 57ae1bc7a7 rename MULTI to UNSHARD (#17267) 
* rename MULTI to UNSHARD

* comment updates (glm)

* rename method to unshard
 2026-07-28 16:51:41 -07:00
chenyuandGitHub 13ca9bd8a6 remove dtypes.index again (#17104) 
also reverted some dtype change, the split made things needlessly complicated
 2026-07-20 20:30:04 -04:00
qazalandGitHub 1cf8f2f68c llama: inplace amax update (#17064) 
* llama: inplace amax update

* remove amax_out return

* work

* fit

* work

* work

* keep

* diff cleanup
 2026-07-20 15:05:41 +09:00
qazalandGitHub 7dd3422c63 llama: replace two stage amax with atomics (#17063) 
* atomic amax in c kernels

* quantize fp8 UOp kernel

* diff
 2026-07-17 19:27:10 +09:00
qazalandGitHub cae6696d75 llama: split current and next amax state (#16993) 2026-07-12 18:52:33 +09:00
qazalandGitHub 9ea7de020b llama: fuse grad scale in gemm epilogue (#16992) 2026-07-12 18:15:01 +09:00
chenyuandGitHub ba2c68b1ed start dtypes.index [PR] (#16949) 2026-07-09 15:12:23 -04:00
George HotzandGitHub fdffc6c0c8 remove dtypes base (#16931) 
* remove dtypes base

* find/replace bug

* vcount is junk too
 2026-07-08 11:48:11 -07:00
chenyuandGitHub 3a9b87e48f remove Ops.DEVICE from Ops.PROGRAM [PR] (#16797) 2026-06-29 17:41:31 -04:00
qazalandGitHub b8224e19a7 llama: remove unused fp8 transpose kernel (#16763) 2026-06-26 16:31:13 +09:00
wozeparrotandGitHub e0c69d7a12 llama: fused grad quantize (#16731) 2026-06-24 23:25:39 -04:00
George HotzandGitHub 0a8e61d0c5 switch to the new memory coaleser [pr] (#16716) 
* switch to the new memory coalese

* move that stuff

* copy in allowed length logic

* mulitple buffers

* new coalese is better

* fine

* earlier

* fixes

* work

* work

* valid

* stack on index const
 2026-06-23 18:03:48 -07:00
wozeparrotandGitHub dfea9e7994 llama: fused silu mul quantize mxfp8 (#16704) 2026-06-23 16:59:50 -07:00
wozeparrotandGitHub 05249466ed llama: fused quantize mxfp8 (#16667) 2026-06-18 16:02:28 -07:00
qazalandGitHub f998b9930a fp8 gemm inv_scale in epilogue (#16625) 
* fuse scale

* remove python inv_scale

* more inv_scale removal

* more cleanups

* cleaner

* diff polish

* work

* rename

* simpler

* simpler

* compute

* c

* Revert "c"

This reverts commit 8941fec7ca.

* Revert "compute"

This reverts commit 9db573a6d3.

* Revert "simpler"

This reverts commit 910ad33f87.

* Revert "simpler"

This reverts commit bf75d235a1.

* s_g

* update types

* less diff noise

* remove
 2026-06-15 18:44:41 +09:00
qazalandGitHub 854eac09c6 llama: no E_ copy after bf16 GEMM (#16458) 2026-06-02 14:14:13 +09:00
qazalandGitHub 29b47a0057 llama: update local amax implementation after ParamArgs change (#16446) 
* local amax failing test

* update _local_abs_max_fxn
 2026-05-30 16:55:43 +09:00
qazalandGitHub 452c7d4230 llama: don't allocate grad_xw13 in bf16 (#16359) 2026-05-28 04:33:07 +09:00
qazalandGitHub eecd4706ff fix mailbox comment, add types (#16360) 2026-05-25 22:24:00 +09:00
qazalandGitHub bbfe4f80ec quantize_fp8 kernels in uops (#16288) 
* add tests

* simple UOp kernel is n^2

* fast kernel matching c++, opts_to_apply=()

* remove cpp

* simple o(n) kernel, two passes

* fuse the loops

* works on DEV=CPU

* multi regression test

* fix multi, this can possibly be its own bugfix

* test cleanups

* minimal diff

* match C in UOps

* Revert "match C in UOps"

This reverts commit 0bef740c30.

* edit test

* match speed with C try 2

* needs_second_gpu

* cleanup
 2026-05-22 20:54:06 +09:00
wozeparrotandGitHub afc5bfa183 llama: remove fused grad accum (#16301) 2026-05-21 09:38:40 -07:00
qazalandGitHub 1e0fffe256 fused ce llama kernel in UOps (#16263) 
* work

* using uops

* delete things

* work

* work

* higher level uops

* cleanups
 2026-05-20 19:45:28 +09:00
wozeparrotandGitHub e97f2c1114 llama: only gemm + fa custom kernel (#16180) 
* llama: tie store to grad directly

* llama: set mp flags

* llama: non fused grad fp8 quantize path
 2026-05-12 21:03:49 -07:00
wozeparrotandGitHub 730fa66bf3 llama speed 6 (#16071) 2026-05-06 20:51:03 -07:00
wozeparrotandGitHub ab6218bc92 llama mp fixes (#16050) 2026-05-05 15:35:32 -07:00
wozeparrotandGitHub ef09071073 llama: speed 2 (#15960) 2026-04-28 20:44:37 -07:00
qazalandGitHub b3f0f8d349 llama: fix missing label_smoothing arg (#15955) 2026-04-29 02:12:14 +09:00
wozeparrotandGitHub 5e861cd2c4 llama: move llama kernels to llama_kernels (#15952) 2026-04-27 22:48:53 -07:00