5 people 6729f20aab Ring allreduce try 2 (#3852) ...

* Ring allreduce v3

* Configurable size, number of gpus and jit in benchmark

* ScheduleBarrier v0

* GB/s that make sense

* ScheduleBarrier v0.1

* Fallback on 2 GPUs

* ScheduleBarrier v0.2

* ScheduleBarrier v0.3

* ScheduleBarrier v0.3.1

* ScheduleBarrier v0.3.2

* Replace ScheduleBarrier with automatic optimization

* unused import

* fix comment

* typing

* better fallback

* python 3.8

* RING=2 and use ContextVar

* DEBUG >= 2 and change name

* linter

* type

---------

Co-authored-by: George Hotz <[email protected]>
Co-authored-by: chenyu <[email protected]>
Co-authored-by: nimlgen <[email protected]>

2024-03-21 19:17:51 -04:00

..

codegen

use int32 instead of default_int in simplify_phi_loops (#3828)

2024-03-19 17:49:58 -04:00

features

Ring allreduce try 2 (#3852)

2024-03-21 19:17:51 -04:00

nn

hotfix: this makes beautiful mnist work again, not okay

2024-03-18 18:22:44 -07:00

renderer

fix broken loops in llvm (#3751)

2024-03-15 11:57:51 +03:00

runtime

do not create structs every call in CUDAProgram (#3855)

2024-03-21 17:51:40 +03:00

shape

fix View.pad arg type (#3845)

2024-03-20 19:36:02 -04:00

__init__.py

move graph.py and jit.py into features (#3376)

2024-02-12 17:34:34 +01:00

device.py

log optimized kernels and a script to compare with non-optimized ones (#3829)

2024-03-20 19:22:08 -04:00

dtype.py

Rename .sz to .count on DType (#3413)

2024-02-15 15:03:49 +01:00

helpers.py

Ring allreduce try 2 (#3852)

2024-03-21 19:17:51 -04:00

lazy.py

Ring allreduce try 2 (#3852)

2024-03-21 19:17:51 -04:00

mlops.py

reciprocal mlop (#3694)

2024-03-11 16:08:46 -07:00

ops.py

simple LoadOps.ASSIGN (#3745)

2024-03-14 20:44:34 -07:00

realize.py

conceptual small refactor (#3842)

2024-03-20 16:46:14 -04:00

tensor.py

user _resolve_dim in argmax (#3846)

2024-03-20 20:17:30 -04:00