-
-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[FEATURE] Toy model for testing #1781
Comments
Alternatively, what is the smallest model currently in timm? |
@adamjstewart if you need even smaller we could figure out a model to add an even smaller one to, if it needs weights with an imagenet classifier would need to train |
That's still fairly large. I don't think we need pre-trained weights, we don't want to download anything during testing and most of our images are multispectral anyway. |
@rwightman any more thoughts on this? Still hoping for an incredible simple 1 or 2 layer model intended only for testing purposes. |
@adamjstewart So, 1-2 layer models don't make sense as it breaks the model APIs and all assumptions re feature maps, etc. I have made some much smaller models in the 300-400K param range, they have valid classifiers and something I can use for regression tests, proxies for full sized models meaninfully. So hopefully a bit better for your use case but not exactly what you want. Will be merged with some other changes soon... e.g. https://huggingface.co/timm/test_vit.r160_in1k
|
Another thing that could be done within test modules, create & register own defs... e.g. If one had a unit test module, within that can define test specific models outside timm and register and use via timm API. Of course again many models have constraints in terms of needing at least 4 feature stages, channel counts predefined, etc. But some can be finaggled around that or have more configurability...
|
This looks great. Still working on bumping SMP to support modern timm, and don't necessarily want to require the latest version of timm yet, but I will definitely come back to this someday and speed up all of our tests with smaller models. |
@adamjstewart if you need any points re 'modern timm' let me know, for the most part, especially if it was mostly just models being used and not importing less used modules / layers directly it should be really straight forward and I did lots of regression checks. |
I think we've got it mostly figured out, but I'll let you know if we get stuck anywhere. |
Is your feature request related to a problem? Please describe.
In TorchGeo's unit tests, we want to avoid any large models that require a lot of memory or time to run.
Describe the solution you'd like
I would like to have a toy model with only 1 or 2 layers that runs as quickly as possible while still ensuring compatibility with other timm models.
Describe alternatives you've considered
We're currently writing our own fake models and monkeypatching timm but there are many discrepancies and it isn't the best test.
The text was updated successfully, but these errors were encountered: