geohot/tinygrad
1
0
Fork 0
forked from tinygrad/tinygrad
Code Pull Requests Actions Activity
2,250 Commits 4 Branches 15 Tags
950dc60bc92f84cb7dbdfae2ab0adc56cd2e1678
Commit Graph
359 Commits
Author SHA1 Message Date
Steven Anderson 26c03329f8 better comments 2023-08-02 14:42:11 -04:00
Steven Anderson 4e7e7cdff6 fixing convention issues 2023-08-01 19:45:54 -04:00
Steven Anderson b4d3dbc92a testing cache 2023-08-01 19:02:22 -04:00
Steven Anderson fe7db41db7 testing cache 2023-08-01 16:53:35 -04:00
Steven Anderson 3cef9d9df2 ci should work now 2023-08-01 13:44:23 -04:00
Steven Anderson b9d12a5871 newline warning 2023-08-01 02:13:18 -04:00
Steven Anderson 8e68014299 refactor with ci 2023-07-31 22:34:29 -04:00
Steven Anderson b44463aa66 fixing big offsets 2023-07-31 17:16:26 -04:00
Steven Anderson f887d9b4ba refactor: cleaning 2023-07-31 17:16:26 -04:00
Steven Anderson 0d19f62a59 refactor: imm 2023-07-31 17:16:26 -04:00
Steven Anderson d49ebd9b7a refactor: global load 2023-07-31 17:16:26 -04:00
Steven Anderson d9b7ad23d6 refactor: one store mem 2023-07-31 17:16:26 -04:00
Steven Anderson bdd5e6d31a working except one 2023-07-31 17:16:26 -04:00
Steven Anderson 373f5831ec pow not working 2023-07-31 17:16:26 -04:00
Steven Anderson a898517b39 refactor 2023-07-31 17:16:26 -04:00
Steven Anderson 11219d83aa getting things working 2023-07-31 17:16:26 -04:00
Steven Anderson 6634737b2d fixing imm 2023-07-31 17:16:26 -04:00
Steven Anderson 849cec11bf working 2023-07-31 17:16:26 -04:00
Steven Anderson e046db4693 do not use x's 2023-07-31 17:16:26 -04:00
Steven Anderson 1f90a2e923 eq2 2023-07-31 17:16:26 -04:00
Steven Anderson a55f1a668d eq 2023-07-31 17:16:26 -04:00
Steven Anderson ee3585dca8 fixing some 2023-07-31 17:16:26 -04:00
Steven Anderson 093fd58d58 fixing bugs 2023-07-31 17:16:26 -04:00
Steven Anderson fc4c72cf78 first try 2023-07-31 17:16:26 -04:00
Steven Anderson 247db8d408 refactoring a bit 2023-07-31 17:16:26 -04:00
Steven Anderson 9ab399a67f branching with load 2023-07-31 17:16:26 -04:00
Steven Anderson ff96243699 testing padded conv 2023-07-31 17:16:26 -04:00
Steven Anderson 0f51611ec9 better debugging 2023-07-31 17:16:26 -04:00
Steven Anderson 6cbc5d69d9 testing new memops 2023-07-31 17:16:26 -04:00
George HotzandGitHub 37fa7e96fb Revert "update editorconfig, enforce via CI (#1343)" (#1380) 
This reverts commit da2efecbe2.
 2023-07-31 10:35:50 -07:00
Pavol RusnakandGitHub da2efecbe2 update editorconfig, enforce via CI (#1343) 
* update editorconfig to set unix-style newlines and trim whitespace

* add editorconfig github action to the CI

* fix whitespace
 2023-07-30 18:44:30 -07:00
Cole SutyakandGitHub 2d4e182294 change fetch to allow for local file selection (#1309) 2023-07-23 15:00:16 -04:00
Jacob PradelsandGitHub b112edd2c3 Add pylint trailing whitespace rule (#1314) 2023-07-21 13:37:55 -04:00
madt2709andGitHub d2c1e8409a Update arange to be (start, stop, step) (#1308) 2023-07-21 00:27:23 -04:00
wozeparrotandGitHub 37cc33269a cl fixes for multigpu (#1276) 
* feat: opencl fixes for multigpu usage

* clean: who needs this import anyways
 2023-07-18 19:59:30 -07:00
George HotzandGitHub ab3d281a6e Refactor MemOps (#1256) 
* metal tests pass locally

* define global

* refactor DEFINE_GLOBAL

* move assembly out. it isn't tested

* fix llvm
 2023-07-17 16:36:33 -07:00
StanandGitHub 91f797cd52 Moved mkdir in utils.download_file to diff line (#1249) 
* Moved mkdir to diff line

.mkdir does not return the actual directory being created.

* use walrus operator to simplify
 2023-07-16 00:30:46 -07:00
Yixiang GaoandGitHub a8f2c16f8e add contiguous (#1246) 2023-07-15 08:36:34 -07:00
George HotzandGitHub 67e34b356a good stuff from tensor cores branch (#1199) 2023-07-08 16:58:26 -07:00
Jacky LeeandGitHub e0c2ae8984 Update file paths (#1179) 2023-07-07 18:41:58 -07:00
geohot b8dfbba703 hip_matmul: f16 gemm 2048x2048 gets 36 TFLOPS 2023-07-08 00:35:45 +00:00
StanandGitHub 69d33cab0d Fix: auto create parent dir when downloading file (#1173) 
* Fix: auto create parent dir when downloading file

also removed duplicate import `os`

* Added test for auto parent dir creation when downloading file
 2023-07-07 13:40:29 -07:00
terafoGitHubterafo
 		aa60feda48 Fix naming conflict with huggingface datasets (#1161) 
* Rename in files

* Move files

* Moved to extra/datasets as suggested

* Changes to files

* Fixed stupid mistake

---------

Co-authored-by: terafo <[email protected]>
 2023-07-07 10:43:44 -07:00
StanandGitHub 9b6e57eccd helpers.py: improved test coverage + exception handling (#1165) 
* Fixes + improved test coverage for helpers.py

- added exception handling in `proc`, if an exception was thrown, the thread would hang
- made `_early_exec_process` catch any Exception, before if an exception was thrown before the process was started, it would hand the thread

* Made `_early_exec_process` catch any Exception

 Otherwise, if an exception was thrown before the process was started, it would hang the thread. For example a type error for an argument passed to `subprocess.check_output`

* Fixed `from tinygrad.helpers import Timing` import

oops, for some reason my IDE cleaned that import from extra/helpers.

* Fixed import in llama.py

Another one that I skipped by accident, mybad

* Extracted a class for tests of early exec

* Normalize line endings, windows uses /r/n

* Made `cross_process` not a daemon
 2023-07-07 10:26:05 -07:00
Kunwar Raj SinghGitHubKunwar Raj Singh
 		8391648822 Over 90% on CIFAR with examples/hlb_cifar10.py (#1073) 
* fix eval, lr decay, best eval

* 82.27

* 82.64

* 82.79, reproducable

* add lr sched, 85.26

* 87.42

* 87.94

* 87.42

* tta with flip

* training flip aug

* refactor

* using Tensor for LR is faster

* 89.5

* refactor, flip only train set

* 90.01

* 90.64

* eval jit

* refactor

* only JIT model

* fix eval JIT

* fix eval JIT

* 90.82

* STEPS=900 reaches 90.22

* TTA envvar

* TTA default 0

* fully jit training

* refactor optim

* fix sched

* add label smoothing

* param changes

* patial gelu

* OneCycle with pause

* gelu maybe works

* 90.12

* remove pause lr

* maybe fix lr schedulers

* scheduler test passing

* comments

* try mixup

* shuffle!

* add back the missing last eval

* fix shuffle bugs

* add mixup prob

* fix mixup prob

* 90.19

* correct mixup

* correct mixup

* correct mixup

* 90.24

* 90.33

* refactor, add type hints

* add gradient clipping

* maybe fix test

* full JIT

* back to relu for now

* pass mixup prob as param

* add typehints

* maybe CI works

* try erf gelu

* CI, types

* remove useless import/

* refactor optim

* refactor optim

* try leakyrelu

* try celu

* gelu

* 90.67

* remove grad clip

* remove grad clip tests

* revert params

* add test for OneCycleLR

* 90.62

* fix eval timing

* fix eval timing again

* so where i calculate mixup_prob matters

---------

Co-authored-by: Kunwar Raj Singh <[email protected]>
 2023-07-06 20:46:22 -07:00
Eli FrigoGitHubJacky Lee
 		801564f31b Remove POW llop and add SQRT llop (#1104) 
* fixed division by zero for fast operations

* made et closer to 0

* replace POW llop with SQRT

* updated mlops to swap SQRT and POW llops

* updated hlops to swap POW and SQRT

* added sqrt llop to cpu runtime

* added sqrt llop to cstyle codegen

* added POW llop to llvm ir codegen

* added SQRT llop to torch runtime

* moved pow from mlops to hlops

* found a better way to do reverse pow

* fixed indentation

* added SQRT llop to triton

* update docs to match new llops

* removed POW operator from assembly codegen

* added sqrt and rsqrt to pow hlop

* rewrote pow function in tensor.py

* Adjust tolerance

* Adjust for adamw

* Reduce for Adam too

* removed accidental leftover code

* removed all of accidental code

* added rsqrt test

* removed pow from mlops again

it was added back when resolving merge conflicts

---------

Co-authored-by: Jacky Lee <[email protected]>
 2023-07-05 18:07:58 -07:00
Reza RezvanandGitHub d1356cac27 Fix: Jacobian tests [WIP] (#1126) 
* Fix: Jacobian tests; num_jacobian either bugged or not accurate enough;

* Fix: Jacobian tests;

* Fix: Gradcheck;
 2023-07-05 15:36:22 -07:00
Mehmet KuzucuandGitHub c3173ff281 Add return statement to the train function (#1135) 
add a return statement to the train function in order to provide access to the losses and accuracies lists
 2023-07-05 08:13:38 -07:00
geohot 2f968f8547 ignore cloudpickle type for local mypy 2023-07-04 13:51:20 -07:00
Daniel HipkeandGitHub b4ce23e4b8 Make cross_process use cloudpickle (#1118) 
* fix syntax issues in imagenet_download.py

* use cloudpickle in cross_process to make it work in Python 3.9+

* add cross_process test

* prevent unpickling on every function call

* add cloudpickle to setup.py

* add support for args/kwargs
 2023-07-04 00:47:34 -07:00