 George HotzandGitHub
|
ec00cefa5b
|
llm is the only app (#15779)
* tinygrad/llm is the only app
* upd pyproject
* claude refs
* scoping
* min diff
|
2026-04-17 10:44:48 +08:00 |
|
|
George HotzandGitHub
|
b9eb5b5d49
|
clean up the LLM tokenizer (#12653)
* clean up the LLM tokenizer
* simple tokenizer is actually simple
* ugh write good code
|
2025-10-14 14:22:01 +08:00 |
|
|
|
4f0ee4e982
|
BPE tokenizer (#11415)
* BPE works
* refactor tok
* oops
* basic tests
* fix eval
* smaller diff
* fix error
* proper vocab decoding
* use regex for splitting
* escape ucatrange
* full compat
---------
Co-authored-by: George Hotz <[email protected]>
|
2025-08-04 09:52:38 -07:00 |
|
|
leopfandGitHub
|
557ca7d757
|
testing SimpleTokenizer against OASST1 (#11214)
|
2025-07-14 17:09:31 -07:00 |
|