geohot
e69df9e8b6
llm: parallelize long decode reduction
2026-07-21 15:59:46 +00:00
geohot
7cb77733d8
llm: fix short flash decode wave merge
2026-07-21 15:22:42 +00:00
geohot
f5c9330441
llm: reduce tail decode softmax overhead
2026-07-21 10:58:54 +00:00
geohot
8c2d598285
llm: prewarm decode expert route inputs
2026-07-21 10:39:51 +00:00
geohot
10df19afc3
hcq: pass only compiled kernel globals
2026-07-21 10:39:51 +00:00
geohot
a964134597
llm: speed decode at long context
2026-07-21 09:12:27 +00:00
geohot
2357c1e955
llm: precompute expert route indices
2026-07-21 08:59:03 +00:00
geohot
3d2636e592
llm: reduce tail decode attention overhead
2026-07-21 08:02:54 +00:00
geohot
5d360f1bea
llm: use two waves for IQ3 prefill
2026-07-21 07:17:16 +00:00
geohot
d5793c85bd
llm: stabilize flash decode wave merge
2026-07-21 06:03:59 +00:00
geohot
711308bc41
llm: preserve prompt checkpoints and speed long decode
2026-07-21 05:39:17 +00:00
geohot
83ba9dd90f
llm: finish one GPU serving optimizations
2026-07-21 00:30:21 +00:00
geohot
6b28df0f0e
llm: reuse recurrent state dot products
2026-07-20 21:33:13 +00:00
geohot
d00bdb6790
llm: avoid max context prompt allocation
2026-07-20 21:33:13 +00:00
geohot
317a6b0a3e
llm: speed up Qwen3.6 serving
2026-07-20 21:33:13 +00:00
George Hotz and GitHub
f7ce7f330d
llm: minor fixes + tests ( #17099 )
...
* llm: minor fixes + tests
* error
2026-07-20 14:31:20 -07:00
chenyu and GitHub
4b8db13e01
rdna int8 wmma ( #17098 )
...
nice to fix _wmma_name, also more generic tests
2026-07-20 17:00:50 -04:00
sirhcm and GitHub
b1cbd1a43f
pytest: use timeout_method signal ( #17094 )
2026-07-20 15:19:24 -04:00
chenyu and GitHub
dbb0f6067e
clean up ALU rules in spec.py ( #17095 )
2026-07-20 15:18:48 -04:00
chenyu and GitHub
8481eba866
allow-unsafe-pr-checkout for szdiff.yml ( #17096 )
...
it uses sz.py on master to parse the change, should be safe
2026-07-20 15:08:54 -04:00
nimlgen and GitHub
2b96d64496
hcq2: tiny opts and fixes ( #17092 )
2026-07-20 18:46:52 +03:00
Pol Puigdemont Plana and GitHub
ef77963cfd
derivative of logsumexp is independent of max ( #17088 )
...
same as #7009 but for logsumexp and logcumsumexp.
fwd+bwd kernel count 5 -> 3 for both. gradients unchanged
(ties, -inf masks, torch-compared at grad_atol=1e-7).
2026-07-20 06:52:16 -07:00
qazal and GitHub
abba2aebda
llama: correct fused qkv shape assert ( #17086 )
2026-07-20 15:51:21 +09:00
qazal and GitHub
1cf8f2f68c
llama: inplace amax update ( #17064 )
...
* llama: inplace amax update
* remove amax_out return
* work
* fit
* work
* work
* keep
* diff cleanup
2026-07-20 15:05:41 +09:00
chenyu and GitHub
ac3f56a1a2
more shift tests ( #17083 )
2026-07-19 16:05:13 -04:00
chenyu and GitHub
89117d8b9e
use real shift in l2i decomp [pr] ( #17080 )
...
works for variable shift distace too, also fixed signed arithmetic fill
2026-07-19 13:15:06 -04:00
chenyu and GitHub
9970a0aad0
fix Tensor << Tensor for x86 ( #17082 )
...
* fix Tensor << Tensor for x86
* torch
2026-07-19 12:31:05 -04:00
chenyu and GitHub
0146a30125
improve cast to unsign min_max [pr] ( #17078 )
2026-07-18 21:58:41 -04:00
George Hotz and GitHub
b53cd35cff
llm: make tokenizer fast (kimi) ( #17077 )
...
* llm: make tokenizer fast
* simpler
* re.escape + qcom mypy fix
2026-07-18 17:31:59 -07:00
Rick Wierenga and GitHub
82debb4557
only allow x86_64 target arch on X86Renderer ( #17076 )
2026-07-18 19:51:01 -04:00
wozeparrot and GitHub
ee290b3e39
optim: mxfp8 zero 1 allgathers in fp8 ( #17073 )
2026-07-18 07:44:50 -07:00
nimlgen and GitHub
232529ce88
hcq2: simpler sync ( #17069 )
...
* x
* y
* n
2026-07-18 16:27:44 +03:00
qazal and GitHub
24d8681be7
viz: better sidebar collapse ux ( #17072 )
2026-07-18 18:07:58 +09:00
chenyu and GitHub
47629f4bcf
more weak dtype materialization raise ( #17071 )
2026-07-17 23:15:14 -04:00
chenyu and GitHub
f315df29a0
no weak Tensor from and to real buffer ( #17067 )
...
* no weak Tensor from and to real buffer
creation, assign, safe_save
* is_numpy_ndarray to tensor
* one more
2026-07-17 16:09:10 -04:00
George Hotz and GitHub
86a6ad8ed2
llm: split cli.py into serve.py with the HTTP server ( #17065 )
...
* llm: split cli.py into serve.py with the HTTP server
* min edit
2026-07-17 10:45:37 -07:00
George Hotz and GitHub
3ee2baf71d
llm: add tool calling support (kimi) ( #17061 )
...
* llm: add tool calling support
* simpler
* cls
* gpt cleanup
* more gpt cleanups
* tests for tools calling
2026-07-17 10:20:01 -07:00
qazal and GitHub
7dd3422c63
llama: replace two stage amax with atomics ( #17063 )
...
* atomic amax in c kernels
* quantize fp8 UOp kernel
* diff
2026-07-17 19:27:10 +09:00
wozeparrot and GitHub
a836c3822a
gptoss: 3d mx block scale ( #17062 )
2026-07-16 23:30:24 -07:00
sirhcm and GitHub
6f1176ea90
benchmarks: test usbgpu copy speeds on comma ( #17060 )
2026-07-17 02:02:21 -04:00
George Hotz and GitHub
46172bb7c7
llm: add optional jinja template support (kimi) ( #17058 )
...
* add jinja template support (kimi)
* fix tests
* lil
* more crap to fallback
2026-07-16 19:02:58 -07:00
chenyu and GitHub
88826a6f35
no weak dtype for randn_like either ( #17055 )
2026-07-16 18:29:12 -04:00
chenyu and GitHub
3bfd62e915
fix 0 size tolist to match numpy ( #17054 )
2026-07-16 17:42:52 -04:00
nimlgen and GitHub
709babb97c
system: remove sibling functions of PCIDevice ( #17052 )
2026-07-17 00:15:32 +03:00
George Hotz and GitHub
d8b83daac6
set tc_upcast_axes to None when done with it ( #17053 )
...
* set tc_upcast_axes to None when done with it
* no tag needed
2026-07-16 14:15:21 -07:00
stylishvoid and GitHub
c74149c973
avoid repeated parsing and toposort in _valid_priority [PR] ( #17049 )
...
* avoid repeated parsing and toposort in _valid_priority
* use backward_slice_with_self instead
2026-07-16 16:24:08 -04:00
chenyu and GitHub
6fa0b2b19e
materialize weak dtype casts to default ( #17051 )
...
in clone and _buffer
2026-07-16 16:12:33 -04:00
George Hotz and GitHub
4d8c3d3fc9
add test_hgemm to test_tiny ( #17050 )
...
* add test_hgemm to test_tiny
* dsp skip
2026-07-16 13:12:10 -07:00
George Hotz and GitHub
2b1146b3f4
further clean up wmma ( #17048 )
...
* further clean up wmma
* comment
2026-07-16 11:43:23 -07:00
chenyu and GitHub
f6a92d0a16
sum_acc_dtype(weak) is weak ( #17047 )
...
also no explicit weak for rand
2026-07-16 14:32:37 -04:00
geohot
sirhcmandGitHub