qk_scale of timm.models.vision_transformer.Block #1999
Unanswered
For-Isolate
asked this question in
Q&A
Replies: 1 comment 1 reply
-
|
@For-Isolate yes, it was removed quite a while back now, it was only used at the very beginning when I transitioned from some initial vit weights I trained myself (with the wrong scale) to porting the original vit weights from google's jax impl. Unfortunately it propagated to some third party implementations :( |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
It can safely be removed anywhere that still uses it (as far as I know) |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Has the qk_scale of timm.models.vision_transformer.Block been removed? There is still this parameter in the mae code of facebook. If I want to run this code, shall I directly remove this parameter or do I need to make any changes
Beta Was this translation helpful? Give feedback.
All reactions