nimlgen/tinygrad
1
0
Fork 0
forked from tinygrad/tinygrad
Code Pull Requests Activity
5,174 Commits 334 Branches 15 Tags
ddda9420bec953740b2d31eaee5c7040facf72a9
Commit Graph
651 Commits
Author SHA1 Message Date
George HotzandGitHub ddda9420be push contract through cast to fix test_float2_acc (#5581) 
* push contract through cast to fix test_float2_acc

* no_vectorized_alu applies to cast too
 2024-07-19 09:30:26 -07:00
chenyuandGitHub 3f590c3b31 some limit_dims to limit global merging (#5489) 
only supports merging dims in a way that does not surpass limit, no splitting yet
 2024-07-19 12:17:46 -04:00
George HotzandGitHub e04704faff put acc first again (#5580) 2024-07-19 08:55:19 -07:00
chenyuandGitHub fc5b9f8dc9 Kernel.required_optimizations and Kernel.hand_coded_optimizations returns self (#5576) 
[run_process_replay]
 2024-07-19 10:55:14 -04:00
George HotzandGitHub 0ad87021e2 move acc to end (#5568) 
* move acc to end

* confirmed pictures are the same

* relax that

* Update test_ops.py
 2024-07-19 03:06:52 -07:00
George HotzandGitHub 2de82b8a5d remove get_lazyop_info (#5570) 
* don't use get_lazyop_info more

* keep that min

* no ptx for that test
 2024-07-19 03:05:33 -07:00
chenyuandGitHub efccb1c3ba swap global for size 3 too (#5567) 
hc path resnet on green 10% faster
 2024-07-18 23:31:15 -04:00
chenyuandGitHub abe29a05b0 swap first and last global in hcopt / hc tc path (#5566) 2024-07-18 18:54:44 -04:00
George HotzandGitHub 946da97820 swap action (#5565) 
* swap action

* don't allow same action expressed differently

* oops, was reversed

* one line is fine

* only swap
 2024-07-18 15:19:40 -07:00
George HotzandGitHub 223d9283ee fix float4 acc by moving contracts (#5559) 2024-07-18 11:30:16 -07:00
George HotzandGitHub c41cd55556 remove vectorized alu in expander [run_process_replay] (#5561) 2024-07-18 11:27:40 -07:00
George HotzandGitHub a7fec05acc fix broken store rule [run_process_replay] (#5558) 
* remove unused store rule [run_process_replay]

* that should preserve behavior i think
 2024-07-18 11:07:34 -07:00
George HotzandGitHub 923e0fe0b8 fix half4 folding (#5556) 2024-07-18 10:47:39 -07:00
George HotzandGitHub d1a7279605 indexing fold with casted bool (#5551) 
* cast bool is where

* universal transform is wrong
 2024-07-18 10:02:29 -07:00
kormannandGitHub 2c4add6844 pretty print lazy op per default (#5505) 
* pretty lop

* min diff

* walrus

* fix

* min diff

* simplify

* pretty helper function

* ws

* pretty uop upat

* tests

* stricter tests

* test passes

* ws

* stronger upat test

* delete print_tree

* min diff

* stricter exp test

* fix merge

* stronger uops eval test

* +readable and deep upat test

* +readable and deep upat test

* sort inv fix

* fix

* revert allowed_len
 2024-07-18 09:34:08 -07:00
chenyuandGitHub e569c927cf remove Kernel.shape_offsets [run_process_replay] (#5544) 
the only use case now can be further simplified
 2024-07-17 23:16:47 -04:00
George HotzandGitHub fa7e734b49 MetaOps.KERNEL (#5543) 2024-07-17 19:41:23 -07:00
George HotzandGitHub a6e70f8a71 clean up expand function [run_process_replay] (#5538) 
* clean up expand function [run_process_replay]

* lil cleaner

* add a type
 2024-07-17 15:02:00 -07:00
George HotzandGitHub 1a68854766 PatternMatcher add (#5532) 
* PatternMatcher add [run_process_replay]

* f4 dynamic

* test_failure_36 is fixed

* fix PTX
 2024-07-17 12:44:42 -07:00
qazalandGitHub 0a7872a62f use exec_alu in uops flop counting (#5511) 
* use exec_alu for uops flop counting

* deal with sint
 2024-07-17 22:39:27 +03:00
George HotzandGitHub 1242b302fa expand UOps with rewrite rules (#5501) 
* expand UOps with rewrite rules [run_process_replay]

* progress

* much closer

* close, way less bugs

* bunch of expander tests

* fix contract

* ops tests pass

* fix barrier

* mostly passing

* bitcast in expanded ops

* support more expand merges

* all tests pass maybe

* fix empty EXPAND

* fix LIN fuzzing

* add ALL_SAME assert

* all same

* all same work

* raise CompileError

* pass fuzz linearizer

* revert whitespace

* fix nv tensor core test

* fix mypy

* bug fix

* fuzzer passes

* put tests back

* expand arg to idx
 2024-07-17 10:17:50 -07:00
George HotzandGitHub 158221b36b expand tests from uop_expander [run_process_replay] (#5524) 
* expand tests from uop_expander

* more changes from the branch
 2024-07-17 09:22:36 -07:00
George HotzandGitHub 42c25cc961 fix fixup_ast (#5523) 
* fix fixup_ast

* these lin failures are fixed
 2024-07-17 08:52:21 -07:00
chenyuandGitHub 4ad83d032e remove Kernel.lazyops [run_process_replay] (#5517) 
always use Kernel.ast.lazyops
 2024-07-16 19:47:42 -04:00
chenyuandGitHub 5aad043522 cleanup fixup_ast local shape long line [run_process_replay] (#5513) 2024-07-16 17:29:38 -04:00
gswanggandGitHub 203161c75d refactor VECTORIZE/GEP rules (#5507) 2024-07-16 09:41:23 -07:00
chenyuandGitHub 1ccd987e6a simpler tc permaxis in fixup_ast.fix_st [run_process_replay] (#5502) 2024-07-15 21:35:32 -04:00
George HotzandGitHub 9d4c3c553c prepare expand to support multiexpand [run_process_replay] (#5503) 2024-07-15 18:21:24 -07:00
chenyuandGitHub fd43d33b7d shave some lines from transcend math [run_process_replay] (#5500) 
* shave some lines from transcend math [run_process_replay]

* put input_dtype back
 2024-07-15 21:02:24 -04:00
chenyuandGitHub eef43c9f49 include dims in kernel/nv invalid err msg (#5487) 2024-07-14 22:51:30 -04:00
chenyuandGitHub c80801c266 len(full_shape)-ki.upcasted -> first_upcasted (#5485) 
[run_process_replay]
 2024-07-14 20:21:18 -04:00
chenyuGitHubqazal
 		613a1dbeed render lidx starting with 0 (#5478) 
* render lidx starting with 0

changed from
```
  int gidx0 = gid.x; /* 4096 */
  int lidx4 = lid.x; /* 8 */
  int gidx1 = gid.y; /* 7 */
  int lidx5 = lid.y; /* 8 */
  int gidx2 = gid.z; /* 7 */
  int lidx6 = lid.z; /* 2 */
```
to
```
  int gidx0 = gid.x; /* 4096 */
  int lidx0 = lid.x; /* 8 */
  int gidx1 = gid.y; /* 7 */
  int lidx1 = lid.y; /* 8 */
  int gidx2 = gid.z; /* 7 */
  int lidx2 = lid.z; /* 2 */
```

the existing one started from pre-limited global dims which skip number if there are more than 3 global dims

* don't need start_dim

---------

Co-authored-by: qazal <[email protected]>
 2024-07-14 16:34:04 -04:00
qazalandGitHub 0b3a34e3b1 vectorize folding [run_process_replay] (#5470) 
* test_gep_vec_fold

* remove that

* fix process replay

* lint
 2024-07-14 09:41:48 +03:00
Carson PowersandGitHub ef578b4de8 new UOp style patterns [run_process_replay] (#5444) 
* express permute srcs in uop

* loop folding / sum collapse pats -> uop style

* UNMUL, const, phi on DEFINE_ACC pats -> uop style

* fix: cvar not const

* DEFINE_ACC w/o inputs, VECTORIZE-PHI-GEP pats -> uop style

* fix VECTORIZE-PHI-GEP pat

* contractor, reducer, float4 pats -> uop style

* arange folding .where

* one more

* revert permute expression in UOp
 2024-07-13 17:21:08 -07:00
George HotzandGitHub 942c58be90 BEAM_COMPARE=2 validates the correctness of BEAM kernels (#5458) 
* beam compare 2

* found issue maybe

* correct, not fail

* full rand

* less numpy

* extra simplify doesn't fix it

* reorder

* no numpy

* check in reverse

* test new tensor behavior

* better error msg
 2024-07-13 13:53:43 -07:00
wozeparrotandGitHub 2427f149a3 threefry as pattern matcher (#5371) 2024-07-13 11:59:03 -07:00
chenyuandGitHub de6ab56458 clean up transcend math with uop syntactic sugar [run_process_replay] (#5455) 
* clean up transcend math with uop syntactic sugar [run_process_replay]

* that?

* maybe
 2024-07-13 14:00:14 -04:00
chenyuandGitHub d2933d3548 simplify transcend math [run_process_replay] (#5454) 
there are some (x - x) in dfadd2_f2_f2_f2, dfmul2_f2_f2_f2, dfdiv2_f2_f2_f2 that were removed by pattern matcher
 2024-07-13 12:43:31 -04:00
qazalandGitHub 23b907efbb restore process replay runs by their id (#5453) 2024-07-13 19:32:34 +03:00
qazalandGitHub b8c9298164 verify_lazyop in for WMMA and group_for_reduces (#5448) 
* try passing no tc and group for reduces

* minor

* use op.arg

* group_for_reduces
 2024-07-13 18:06:19 +03:00
hiketteiandGitHub 3a7262d923 [Patch] Fixed an invaild value of fp64 xlog(DBL_MIN) (#5441) 
* [Patch] Removed weird NaN Handling in xlog2 resulting in different output around 1e-203

* Patch: compare the value of xlog(x) using y, allowing x <= 1e-200

* mypy

* fuzzer tests for log2

* fix tests: use approximate dbl_min, fp64 fails at nv

* update: gradually increment the scale (if y is not inf)
 2024-07-13 01:11:53 -04:00
George HotzandGitHub 414aa6ee98 minor improvments to matcher speed [run_process_replay] (#5439) 
* minor improvments to matcher speed [run_process_replay]

* oh, put that back
 2024-07-12 20:41:41 -07:00
hiketteiGitHubchenyu
 		0795139f30 Fix TRANSCENDENTAL=2 fp64 sin (#5385) 
* fixes on transcendental: fix for fp64 payne hanek, refactor for fp16 sin

* revert the changes on test

* refactor on xsin: removed cody_waite_reduction, always use payne_hanek

* Revert "refactor on xsin: removed cody_waite_reduction, always use payne_hanek"

This reverts commit 2fd401f251.

* still need cody_waite_reduction for the very smaller range

* test: added a regression test for transcendental sin

* test: found the worse case ulp 3.5 only in numpy

* give the input as a valid dtype

---------

Co-authored-by: chenyu <[email protected]>
 2024-07-12 23:15:04 -04:00
George HotzandGitHub fb3011ac61 improve matcher speed [run_process_replay] (#5438) 
* improve matcher speed [run_process_replay]

* don't use arg set in ptx
 2024-07-12 20:02:19 -07:00
George HotzandGitHub 03c2dc8bd7 lowerer is kernel [run_process_replay] (#5437) 2024-07-12 18:50:55 -07:00
George HotzandGitHub b8342fb085 independent lowerer [run_process_replay] (#5434) 
* independent lowerer [run_process_replay]

* don't relinearize PTX

* fix ptx

* Revert "fix ptx"

This reverts commit f4e8e059c0.

* Revert "don't relinearize PTX"

This reverts commit f6c12c506c.

* parents is fine, no need for linearization

* remove loop local idxs

* recover stupid loop_idxs
 2024-07-12 18:08:43 -07:00
George HotzandGitHub cddfd8e25d bugfix: group for reduce should check all dimensions (#5431) 2024-07-12 17:02:40 -07:00
George HotzandGitHub fbaf040baf compute full_shape from LazyOp [run_process_replay] (#5429) 
* compute full_shape from LazyOp

* put KernelInfo in the sink

* wrong but pass
 2024-07-12 16:47:08 -07:00
George HotzandGitHub 870dc8c350 s/Linearizer/Lowerer [run_process_replay] (#5428) 2024-07-12 15:54:07 -07:00
chenyuandGitHub 4cd1de038a smaller reshape_and_permute arg in shift_to (#5426) 
adding tuples directly
[run_process_replay]
 2024-07-12 17:46:48 -04:00