-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Support using backbones from pytorch-image-models (timm) for TSN #880
Conversation
@kennymckormick typo, right? should be densenet161? |
Codecov Report
@@ Coverage Diff @@
## master #880 +/- ##
==========================================
- Coverage 83.57% 83.49% -0.09%
==========================================
Files 132 132
Lines 9966 9977 +11
Branches 1718 1720 +2
==========================================
+ Hits 8329 8330 +1
- Misses 1218 1226 +8
- Partials 419 421 +2
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
Yes, that's a typo. Would you please help correct it in this PR? |
Thanks for the clarification, will fix the typo |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work, please ping me if checkpoints are ready.
tsn-swin-transformer ckpt/log/json could be found here. Results are quite good: top1/5 accuracy are 77.31/92.88 However, since our kinetics400 dataset is quite different(Now I'm using kinetics400 provided by cvdf), I did some tests with tsn-densenet161
So the accuracy of tsn-swin-transformer ckpt may be different between our datasets. Could you please test the tsn-swin-transformer on your kinetics400 and update accuracy in the TSN readme? If the accuracy is not good enough, maybe you have to help train this model... |
Great, I'm gonna test it on our validation dataset. Besides, I'm going to upload our Kinetics-400 validation dataset. |
Seems some updates need to be committed to pass CI. |
CI raises ImportError, mmcv-full is not properly installed, not sure why that happens |
Motivation
pytorch-image-models is one of the best pytorch classification projects, with a large number of backbones, including both cnns and transformers. It would be interesting if we can train tsn models (and other 2d recognizers) with various backbones.
Modification
Following #679 #720, add codes, docs, unittest.
Use cases (Optional)
Modify model settings, an example is listed as follows.
timm.create_model(backbone_type, **backbone)
https://github.com/rwightman/pytorch-image-models/blob/23c18a33e4168dc7cb11439c1f9acd38dc8e9824/timm/models/factory.py#L25
model['cls_head']['in_channels']
andimg_norm_cfg
accordingly.Test timm backbones
There are over 500 backbones in timm...... Write a script to test timm backbones.
check_timm_models.py.txt
Please note that
verify=True
are supported for MMAction2 TSN models.img_norm_cfg
, both std and mean should be multiplied by 255.TODO