tinygrad

11,549 Commits 336 Branches 15 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
George HotzandGitHub	b9eb5b5d49	clean up the LLM tokenizer (#12653 ) * clean up the LLM tokenizer * simple tokenizer is actually simple * ugh write good code	2025-10-14 14:22:01 +08:00
George HotzandGitHub	6ed2dfd187	delete the arange dim mismatch restriction (#11568 ) * delete the arange dim mismatch restriction * skip that test race	2025-08-07 13:46:17 -07:00
leopf GitHub George Hotz	4f0ee4e982	BPE tokenizer (#11415 ) * BPE works * refactor tok * oops * basic tests * fix eval * smaller diff * fix error * proper vocab decoding * use regex for splitting * escape ucatrange * full compat --------- Co-authored-by: George Hotz <[email protected]>	2025-08-04 09:52:38 -07:00

Author

SHA1

Message

Date

George HotzandGitHub

b9eb5b5d49

clean up the LLM tokenizer (#12653 )

* clean up the LLM tokenizer

* simple tokenizer is actually simple

* ugh write good code

2025-10-14 14:22:01 +08:00

George HotzandGitHub

6ed2dfd187

delete the arange dim mismatch restriction (#11568 )

* delete the arange dim mismatch restriction

* skip that test race

2025-08-07 13:46:17 -07:00

4f0ee4e982

BPE tokenizer (#11415 )

* BPE works

* refactor tok

* oops

* basic tests

* fix eval

* smaller diff

* fix error

* proper vocab decoding

* use regex for splitting

* escape ucatrange

* full compat

---------

Co-authored-by: George Hotz <[email protected]>

2025-08-04 09:52:38 -07:00