forked from tinygrad/tinygrad
George HotzandGitHub
60cc1d7827
* delete Ops.FUNCTION/GETTUPLE/TUPLE: call outputs are AFTER on RETURNED placeholders value-producing calls: the body is a plain parametric program that stores outputs into output PARAMs (slots after the input PARAMs). the RETURNED placeholders are inputs to the call, bound to the output PARAMs positionally wherever the call is resolved, and callers AFTER on them like normal buffers. gradient flows through the generic AFTER rule; everything is just Ops.CALL. * RETURNED identity is its placement in the call srcs, not a nonce slot=-1 merging collapses duplicate-signature outputs into one uop (t+1,t+2 grads and multi-grad backward calls dedupe wrongly), and skipping the uop cache breaks schedule_cache (stale linear hits since structural keys assume interning). instead the RETURNED's placement (output index among call srcs) is its identity: identical call constructions merge deterministically, positions never collide. * resolve RETURNED afters in the tensor graph like values (master parity with gettuple) - remove the CONTIGUOUS wrap of tagged call-output afters, it forced call outputs (e.g. local shard amax) into their own buffer/kernel instead of inlining - inline RETURNED afters at transform time via returned_after_finalize, dissolving to values for consumers; calls with bound-variable or unresolved UNSHARD args keep the schedule-time resolution path - allow movement ops (flat-storage views) in kernel graph value positions in the spec - port embedding backward + extra/llama_kernels (local_abs_max, rmsnorm) to the new API * use SINK, not GROUP, for gradient value containers spec.py only blesses GROUP of stores/groups/loops; the gradient value bundles (the forward values, root_grad seeds, and the after->call gradient edge) are plain value containers, and SINK-of-values is already in the spec. also fix extra/llama_kernels/rmsnorm: returned_outputs is a property * CALL is positional: RETURNS work in any src position, convention lives in call_outputs - all resolution paths (gradient, precompile transform, binding) locate RETURNEDs by identity, not by "last srcs"; only call_outputs builds the args-first layout - grad_fxn padding aligns grads with the call's actual src positions - add test_two_return/precompiled * source-compat shim for maketuple/gettuple so foreign code built before the redesign keeps working UOp.maketuple returns a _LegacyTupleValues holder; .call builds call_outputs; CALL.gettuple(i) is returned_outputs[i]. the produced graphs are identical to the new-api versions, so nn/extra/mlperf code is reverted to upstream text * simplify function.py call construction + drop the resolved-call cache - function.py: single and tuple returns both build the call through call_outputs - tensor.py: resolve_function is deterministic and interned, the global cache was unneeded * bind zero-offset views of flat storage to the storage instead of padding them call args need offset 0 and enough length, not views: flat_storage collapses the zero-offset contiguous view chain to the sized storage base, so resolved call args are storage-bare like master (no PAD/SHRINK chains in the kernel graph) * spec.py: drop stray rebase-collision edits, keep only the RETURNED changes * test_multitensor: revert to master, the gettuple shim covers it * materialize all tagged RETURNED afters into real buffers call outputs need real storage regardless of whether they are finals of the current realize: deferred/stateful outputs (the fp8 grad-amax mailbox) are consumed by later realize steps as call args, where a resolved value would have no ranges * call input buffers: wrap RETURNED-based afters, not real-buffer afters precompiled call input binding kept any AFTER unwrapped; an AFTER on a RETURNED placeholder has no storage behind it, so its value leaked into the kernel graph with no consumer able to register ranges (llama3 8B fp8 mailbox pipeline crash). materialize afters whose base has no buffer identity instead. this was the fix matching master for the REDUCE-has-no-ranges crash and restores the llama-kernels amax kernel count * call slots are src positions, always; never rearrange one upstream cause behind the three P1 findings: the raw CALL machinery binds positionally (resolve_function params, gradient padding) but a second args-first convention crept in where RETURNEDs get moved to trailing slots. position is identity now: - transform_precompiled_call keeps RETURNEDs' original src positions: outs take their places, other args become input buffers; no slot renumbering - implicit gradients are emitted aligned to original src positions (None at RETURNED positions) - flat_storage drops the hand-rolled contiguity analysis: reshape itself is the flat-prefix check (it raises ValueError); strided views materialize first * nits on call slot positions; regression tests for interspersed RETURNED - flat_storage back to pad_to().reshape() (reshape keeps movement views, it is not a contiguity check) - input_buffer checks has_buffer_identity(after_ok=True) - TestArgOrder: interspersed RETURNED (plain + precompiled transform), its gradient, padded and strided function inputs * device fixes * TestArgOrder: padded regression uses zero-start padded/shrunk view * TestArgOrder: clone to force buffer identity in padded/strided regression tests * slim: revert prepare formatting, drop reverted-bug tests, restore viz guards, clean comments, mirror returned on param * gut transform_precompiled_call, delete returned_after_finalize the transform keeps master's shape; the prepare-stage resolve_AFTER rule already inlines plain call outputs, and materialization is owned by the input-buffer rule (afters on real buffers bind, afters on RETURNEDs contiguous) * update spec for returned * transform_precompiled_call: inline the input-buffer rule, drop sorted() (body stores are already slot-ordered) * drop dead RETURNED-era rules: prepare's after-shell strip (leftover from returned_after_finalize, which is gone), redundant pattern-covered SINK check, defensive slot-sorts (bodies are slot-ordered by construction) * drop final_tags: final outputs of value calls materialize at sink construction The set of finals is already known precisely (the big_sink's srcs), so track nothing: wrap each final AFTER-on-RETURNED in CONTIGUOUS right after numbering. Precompiled calls are excluded - transform_precompiled_call in the flatten pass gives their outputs real buffers, and wrapping before that transform leaves a stale tag that breaks the output copy. * drop unused default_dtype import