Is the models metadata up to date?

[mpeychev]

Hello, thank you for building and maintaining this repo!

I wanted to confirm if the models metadata is correct and up to date.

For example, the documentation of the BiT models says they are trained on the JFT-300M dataset, while the metadata csv and the code for these models implies ImageNet-21k.

In general, what does the dataset in the "pretrain" column in the metadata csv refer to? I imagine it is the initial dataset on which the model is being trained (before any fine tuning takes place).

[mpeychev]

So, the BiT-S models is pretrained on ImageNet-1k, BiT-M models on ImageNet-21k and BiT-L (not released and not included in timm) on JFT-300M.

So probably the documentation is a bit misleading and the information on the metadata is correct.

[rwightman]

@mpeychev with the one exception of some NoisyStudent EfficientNets that were described as being pretrained on JFT-300M without any labels, Google does not release JFT trained weights. So the best BiT models are all ImageNet-21k.

Metadata is out of date now, after the significant changes for 0.9, the pretrain and fine-tune datasets are encoded in the part of the model name after the ., ie resnetv2_152x4_bit.goog_in21k_ft_in1k is a resnetv2_152x4_bit (v2 BiT specific architecture), pretrained by google on ImageNet-21k and fine-tuned on ImageNet-1k.

[rwightman]

You can also put the model name in the HF hub and there will be a bit more info in many cases, https://huggingface.co/timm/ + resnetv2_152x4_bit.goog_in21k_ft_in1k = https://huggingface.co/timm/resnetv2_152x4_bit.goog_in21k_ft_in1k

[mpeychev]

Yep, I oriented myself.
Thank you for the detailed response and the well-organized library!