* remove RETURNED, replace with optional global BUFFERs
* mint identity slots from the global counter for all symbolic buffer uops
* is_unbound: derive from buffer is None [WIP, sharded red]
* Revert "is_unbound: derive from buffer is None [WIP, sharded red]"
This reverts commit 5670828d49655d5c5633618ec3b3db7c804d0896.
* just unbound, no optional
* fix issue
* lil
* kimi wip
* fixes
* simpler diff
* just label that
* lil
* remove num returned
* Revert "remove num returned"
This reverts commit 56fd0b9b5a16810b9f4b2a1499f97400d1eec848.
* tighten rewrite
* that was junk
* delete Ops.FUNCTION/GETTUPLE/TUPLE: call outputs are AFTER on RETURNED placeholders
value-producing calls: the body is a plain parametric program that stores outputs
into output PARAMs (slots after the input PARAMs). the RETURNED placeholders are
inputs to the call, bound to the output PARAMs positionally wherever the call is
resolved, and callers AFTER on them like normal buffers. gradient flows through
the generic AFTER rule; everything is just Ops.CALL.
* RETURNED identity is its placement in the call srcs, not a nonce
slot=-1 merging collapses duplicate-signature outputs into one uop (t+1,t+2 grads
and multi-grad backward calls dedupe wrongly), and skipping the uop cache breaks
schedule_cache (stale linear hits since structural keys assume interning). instead
the RETURNED's placement (output index among call srcs) is its identity: identical
call constructions merge deterministically, positions never collide.
* resolve RETURNED afters in the tensor graph like values (master parity with gettuple)
- remove the CONTIGUOUS wrap of tagged call-output afters, it forced call outputs
(e.g. local shard amax) into their own buffer/kernel instead of inlining
- inline RETURNED afters at transform time via returned_after_finalize, dissolving
to values for consumers; calls with bound-variable or unresolved UNSHARD args
keep the schedule-time resolution path
- allow movement ops (flat-storage views) in kernel graph value positions in the spec
- port embedding backward + extra/llama_kernels (local_abs_max, rmsnorm) to the new API
* use SINK, not GROUP, for gradient value containers
spec.py only blesses GROUP of stores/groups/loops; the gradient value bundles
(the forward values, root_grad seeds, and the after->call gradient edge) are
plain value containers, and SINK-of-values is already in the spec.
also fix extra/llama_kernels/rmsnorm: returned_outputs is a property
* CALL is positional: RETURNS work in any src position, convention lives in call_outputs
- all resolution paths (gradient, precompile transform, binding) locate RETURNEDs
by identity, not by "last srcs"; only call_outputs builds the args-first layout
- grad_fxn padding aligns grads with the call's actual src positions
- add test_two_return/precompiled
* source-compat shim for maketuple/gettuple so foreign code built before the redesign keeps working
UOp.maketuple returns a _LegacyTupleValues holder; .call builds call_outputs;
CALL.gettuple(i) is returned_outputs[i]. the produced graphs are identical to
the new-api versions, so nn/extra/mlperf code is reverted to upstream text
* simplify function.py call construction + drop the resolved-call cache
- function.py: single and tuple returns both build the call through call_outputs
- tensor.py: resolve_function is deterministic and interned, the global cache was unneeded
* bind zero-offset views of flat storage to the storage instead of padding them
call args need offset 0 and enough length, not views: flat_storage collapses the
zero-offset contiguous view chain to the sized storage base, so resolved call args
are storage-bare like master (no PAD/SHRINK chains in the kernel graph)
* spec.py: drop stray rebase-collision edits, keep only the RETURNED changes
* test_multitensor: revert to master, the gettuple shim covers it
* materialize all tagged RETURNED afters into real buffers
call outputs need real storage regardless of whether they are finals of the current
realize: deferred/stateful outputs (the fp8 grad-amax mailbox) are consumed by later
realize steps as call args, where a resolved value would have no ranges
* call input buffers: wrap RETURNED-based afters, not real-buffer afters
precompiled call input binding kept any AFTER unwrapped; an AFTER on a RETURNED
placeholder has no storage behind it, so its value leaked into the kernel graph with
no consumer able to register ranges (llama3 8B fp8 mailbox pipeline crash).
materialize afters whose base has no buffer identity instead.
this was the fix matching master for the REDUCE-has-no-ranges crash and restores
the llama-kernels amax kernel count
* call slots are src positions, always; never rearrange
one upstream cause behind the three P1 findings: the raw CALL machinery binds
positionally (resolve_function params, gradient padding) but a second args-first
convention crept in where RETURNEDs get moved to trailing slots. position is
identity now:
- transform_precompiled_call keeps RETURNEDs' original src positions: outs take
their places, other args become input buffers; no slot renumbering
- implicit gradients are emitted aligned to original src positions (None at
RETURNED positions)
- flat_storage drops the hand-rolled contiguity analysis: reshape itself is the
flat-prefix check (it raises ValueError); strided views materialize first
* nits on call slot positions; regression tests for interspersed RETURNED
- flat_storage back to pad_to().reshape() (reshape keeps movement views, it is not a contiguity check)
- input_buffer checks has_buffer_identity(after_ok=True)
- TestArgOrder: interspersed RETURNED (plain + precompiled transform), its gradient,
padded and strided function inputs
* device fixes
* TestArgOrder: padded regression uses zero-start padded/shrunk view
* TestArgOrder: clone to force buffer identity in padded/strided regression tests
* slim: revert prepare formatting, drop reverted-bug tests, restore viz guards, clean comments, mirror returned on param
* gut transform_precompiled_call, delete returned_after_finalize
the transform keeps master's shape; the prepare-stage resolve_AFTER rule already
inlines plain call outputs, and materialization is owned by the input-buffer rule
(afters on real buffers bind, afters on RETURNEDs contiguous)
* update spec for returned
* transform_precompiled_call: inline the input-buffer rule, drop sorted() (body stores are already slot-ordered)
* drop dead RETURNED-era rules: prepare's after-shell strip (leftover from returned_after_finalize, which is gone), redundant pattern-covered SINK check, defensive slot-sorts (bodies are slot-ordered by construction)
* drop final_tags: final outputs of value calls materialize at sink construction
The set of finals is already known precisely (the big_sink's srcs), so track
nothing: wrap each final AFTER-on-RETURNED in CONTIGUOUS right after numbering.
Precompiled calls are excluded - transform_precompiled_call in the flatten
pass gives their outputs real buffers, and wrapping before that transform
leaves a stale tag that breaks the output copy.
* drop unused default_dtype import
* delete Ops.FUNCTION: value-producing calls are just CALLs with a TUPLE body
a CALL with a TUPLE body (always void dtype) is value-producing and gradient-able,
outputs are extracted with GETTUPLE like before; all other CALLs are opaque.
TUPLE/GETTUPLE are untouched
* match the TUPLE body in the UPat, not in the rewrite
* remove FUNCTION from tinyspec (regen pdf) and viz
* fix and spec
* move shape into arg for param/buffer
* no param_from_shape
* drop gratuitous syntax changes
* image is a in-graph view, folded into the param arg at render; drop dead multi param sharding
* view_as helper, simpler resolve_function, spec update
* spec: param/buffer are flat storage, no shape input
* image dims live in the param arg from transform_to_image; tighten kernel graph spec
* kernel graph spec: only RESHAPE/SHRINK over storage values, not all movement
* kernel graph: call args are storage, not views (pm_no_view_args); assert in spec
* strip views at the kernel graph level (pm_no_views), move into rangeify
* touchups
* add precompile to call
* put get back
* something
* after structure
* alt
* keep it call
* resolve call
* resolve linear call
* precompile works with llm
* revert rangeify
* color for debugging
* getenv PRECOMPILE
* clean up deco pattern
* fully recursive sink scheduling
* revert llama
* fix SPEC=2
* embedding is slow
* failing
* float is fine
* null
* it fails
* simplify embedding with broadcasting
* ATOMIC_ADD incoming
* min change
* simpler test
* better test
* fix test
* real test
* simpler
* cleanups
* types and names
* _zero_kernel
* grad multi
* hack
* none
* multi unshard
* more for call
* don't tag in call
* good
* call_multi
* call_multi wow claude is useless
* embedding backward mutli test
* test passes
* fix as_param
* shape_to_shape_arg
* add clip
* before cast
* fix spec=2, use atomics
sirhcmandGitHub