mirror of
https://github.com/tinygrad/tinygrad.git
synced 2026-09-04 08:05:54 +00:00
geohot
cd2aaccb25
the prefill path is fully symbolic again (master's generate, one prefill graph for every chunk size, no static-tail decode): padded steps are exact no-ops in the scan (beta=0, alpha=exp(0)=1), flash attention positions queries at start_pos instead of valid_kv_len-M, and quant linears pad to the chunk bucket prefill 401 tok: 284 -> 348 tok/s on Qwen3.8-27B IQ4_XS (tail chunks no longer decode token-by-token), decode unchanged at 45 tok/s
Three groups of tests run in CI backend -- tests that run on each backend null -- tests that don't require any backend unit -- tests that only run on a single backend in CI