Commit Graph
128 Commits
Author SHA1 Message Date
Edward BettsandGitHub a6203b5ecd Correct spelling mistakes (#314) 2022-04-05 05:22:18 -07:00
Ildar KashaevandGitHub d065040df0 Fix yolo param name (#306) 2022-02-10 01:40:24 -07:00
geohot 2a10116bfa support drawing graphs 2022-01-16 10:45:58 -08:00
George Hotz 2cae2dfa07 don't crash the dataloader for imagenet 2022-01-16 08:41:26 -08:00
George Hotz 907ff7dbb6 imagenet is training 2022-01-15 23:27:31 -08:00
George Hotz d1e082e0ef add imagenet training support 2022-01-15 23:16:38 -08:00
George Hotz ade2af7ac0 data doesn't require grad 2022-01-15 22:41:27 -08:00
George Hotz 0973e54eb0 fix DEBUG for GPU 2022-01-15 22:14:28 -08:00
cjg91andGitHub 7025c9bbeb Transfer learning for ResNet (#295)
* Transfer learning for ResNet

* moved ResNet depth specifics into the class
2022-01-15 23:22:10 -05:00
George Hotz 55d792b065 Revert "fixup resnet"
This reverts commit 4eabe677ed.
2022-01-15 20:22:01 -08:00
George Hotz 4eabe677ed fixup resnet 2022-01-15 20:21:02 -08:00
George Hotz e0bef0bd01 training is False by default 2022-01-15 19:57:41 -08:00
George Hotz 8ec2341cca fix bn training 2022-01-15 19:47:01 -08:00
geohot c0c2c0b041 support larger ViT models 2021-12-12 10:45:10 -08:00
geohot 4b7c31b5b7 break vit into it's own file 2021-11-30 11:19:22 -05:00
geohot 46bbbcf7f0 model touchups 2021-11-30 11:13:34 -05:00
geohot 835869974c clean up vit code 2021-11-30 10:58:03 -05:00
geohot 535f02cc64 use sequential 2021-11-30 00:25:39 -05:00
geohot de938c2d9d vit is now tested 2021-11-30 00:23:06 -05:00
geohot aff810e722 unify transformer block 2021-11-29 18:58:15 -05:00
geohot 58ed46963e fix broadcastdot 2021-11-29 18:54:57 -05:00
geohot 125e74293f promote layernorm to tensor op 2021-11-29 18:08:21 -05:00
geohot dca076dbf1 remove dumb nn ops 2021-11-29 18:05:31 -05:00
geohot 70544e7e9f sum hook override 2021-11-29 17:14:24 -05:00
geohot 8097b8f7d6 vit works 2021-11-29 16:28:14 -05:00
geohot 7c07c5efdd plz fix vit 2021-11-29 15:45:19 -05:00
geohot ca160504e1 affine is always the last dim 2021-11-29 15:22:49 -05:00
geohot e86f7a4aa3 deterministic 2021-11-29 15:10:15 -05:00
geohot f909ab194f gelu with broken test 2021-11-29 15:00:50 -05:00
geohot 9ce881f88c fix bug in getitem, drop int axis 2021-11-29 14:01:24 -05:00
geohot 99b6051467 add ff_dim to transformer 2021-11-29 12:40:52 -05:00
geohot b0f14b4af8 move datasets into datasets 2021-10-30 19:55:50 -07:00
geohot ba28761894 move yolo into examples/yolo 2021-10-30 19:46:00 -07:00
geohot 121d5a17ee use tinynn for Conv2d 2021-10-30 19:40:44 -07:00
Guglielmo CamporeseandGitHub 2b7589db64 Added ResNet-{18, 34, 50, 101, 152} (#271)
* added resnets

* fix minor

* fix minor

* resnet in models

* added resnet test

* added resnet train test

* added linear, conv2d nn tests

* fix minor in extra/training

* resnet in models

* fix minor

* fix tolerance for linear in nn test

* fix eval, this causes cpu and gpu UT failing

* revert transformer test

* fix minor for CPU test

* improved model get_params for sequential layer

* fix minor for params counting

* commented broken ops tests

* improved train for resnet
2021-06-21 09:37:24 -07:00
geohot d3f169b267 move good models to models, add a training step test 2021-06-19 11:24:15 -07:00
Jacky LeeandGitHub 611d81dcb4 Add asserts for non-zero indices (#264) 2021-06-13 21:14:46 -07:00
Josh SmithandGitHub ad756f6112 minor optimizations & cleaning (#257)
* use isinstance, some optimizations & whitespace removal

* revert whitespace changes

* revert more whitespace

* some more cleanup

* revert fstring (not a fan of the {{}})

* fix typo

* fix typo
2021-06-02 09:57:15 -07:00
geohot b80cacb416 fix GPU efficientnet example 2021-05-26 17:29:35 -07:00
20kdcandGitHub 2653d33292 vgg7 (image upscaling) implementation - not the best, but it works (#255)
* vgg7 implementation - not the best, but it works

* VGG7 implementation: Spread nansbane to deter NaNs, maybe improved training experience

* VGG7 implementation: Fix training, for real this time

Results actually attempt to approximate the input

* VGG7 implementation: Sample probability management
2021-05-12 23:48:51 -07:00
SkoshandGitHub 81bf933a91 Improved __getitem__ (#254)
* Some progress on yolov3

* Removed some debugging comments… Also, the forward pass eats all RAM for some reason

* forward pass almost runs

* forward pass runs almost

* forward pass runs, now we gotta load the weights

* loading weights works

* fetches config and weights

* everything kind of works, postprocessing of output still needs to be implemented, temp_process_results kind of works, but its kind of terrible, and not how things should be done

* some changes

* fixed some bugs in the forward pass and load_weights function, now outputs more correct values, however some values are still loaded incorrectly

* Something is wrong with the forward pass, Conv2d tests added

* forward pass almost outputs correct values, gotta fix one more thign

* yolo works

* some final changes

* reverting changes

* removed dataloader

* fixed some indentation

* comment out failing test, somehow it fails CI even though it passes on my computer…

* fixed wrong probabilities

* added webcam option to YOLO, now just need to add bounding boxes and speed it up

* some progress towards adding bounding boxes

* trying to speed up yolo layer on GPU, still faster on CPU but with 30GB ram usage

* Faster inference times, bounding boxes added correctly, webcam works, but is slow, and there is a memory leak when running on CPU... Also added tinygrads output on the classic dog image

* removed some debugging print statements

* updated result image

* something weird is going on, mean op on GPU tensor randomly faults, copying a tensor from GPU->CPU takes 10+ seconds…

* Improved __getitem__

* Updated

* Updated __getitem__

* Linebreaks

* Maybe this works?

* Added MNIST locally, tests run now
2021-05-05 22:15:22 -07:00
SkoshandGitHub 78aa147b39 [WIP] YOLO working on tinygrad! (#245)
* Some progress on yolov3

* Removed some debugging comments… Also, the forward pass eats all RAM for some reason

* forward pass almost runs

* forward pass runs almost

* forward pass runs, now we gotta load the weights

* loading weights works

* fetches config and weights

* everything kind of works, postprocessing of output still needs to be implemented, temp_process_results kind of works, but its kind of terrible, and not how things should be done

* some changes

* fixed some bugs in the forward pass and load_weights function, now outputs more correct values, however some values are still loaded incorrectly

* Something is wrong with the forward pass, Conv2d tests added

* forward pass almost outputs correct values, gotta fix one more thign

* yolo works

* some final changes

* reverting changes

* removed dataloader

* fixed some indentation

* comment out failing test, somehow it fails CI even though it passes on my computer…

* fixed wrong probabilities

* added webcam option to YOLO, now just need to add bounding boxes and speed it up

* some progress towards adding bounding boxes

* trying to speed up yolo layer on GPU, still faster on CPU but with 30GB ram usage

* Faster inference times, bounding boxes added correctly, webcam works, but is slow, and there is a memory leak when running on CPU... Also added tinygrads output on the classic dog image

* removed some debugging print statements

* updated result image

* something weird is going on, mean op on GPU tensor randomly faults, copying a tensor from GPU->CPU takes 10+ seconds…
2021-04-25 18:06:52 -07:00
geohot ce77dda805 yolov5 v4 2021-01-05 07:56:17 -08:00
AsimandGitHub 1c148f2fe4 fixed example broken after gpu refactor (#238) 2021-01-05 07:41:54 -08:00
geohot 6842ad9ec8 minor cleanups, yolo work 2021-01-03 08:14:16 -08:00
geohot 895d142503 start trying to load yolo v5 2021-01-02 12:51:55 -08:00
Marcel BischoffandGitHub 42b4761025 transformer >99.98% test accuracy in ~30s (#230)
* transformer

* BS might divide len(Y_test)

* outoput when accuracy is high

* more readeable

* fixed loss in serious_mnist for new API
2021-01-02 07:45:09 -08:00
geohot f9170505b3 if you like your transformers twice as slow, use the GPU 2020-12-29 17:14:23 -05:00
geohot 3f8e137b6f extra/transformer 2020-12-29 14:14:00 -05:00
geohot bcb3ceeca3 set training in functions 2020-12-28 22:45:46 -05:00