EfficientNetV2XL pretrained with imagenet21k has problems with output shape

[romanvelichkin]

import keras_efficientnet_v2
base_model = keras_efficientnet_v2.EfficientNetV2XL(pretrained="imagenet21k")

WARNING:tensorflow:Skipping loading of weights for layer predictions due to mismatch in shape ((1280, 1000) vs (1280, 21843)).
WARNING:tensorflow:Skipping loading of weights for layer predictions due to mismatch in shape ((1000,) vs (21843,)).

Output layer is:
predictions (Dense) (None, 1000) 1281000 dropout_2[0][0]

It has to be non-fine-tuned model, but somehow output layer has wrong shape.

[leondgarse]

The output shape for raw imagenet21k model is num_classes=21843: efficientnet#usage. You have to set it manually if need the exact raw header with num_classes=21843:

import keras_efficientnet_v2
base_model = keras_efficientnet_v2.EfficientNetV2XL(pretrained="imagenet21k", num_classes=21843)
print(base_model.output_shape)
# (None, 21843)

# Run prediction
import tensorflow as tf
from skimage.data import chelsea
from keras_cv_attention_models.imagenet import decode_predictions_imagenet21k
imm = tf.image.resize(chelsea(), base_model .input_shape[1:3]) # Chelsea the cat
pred = base_model (tf.expand_dims(imm / 128. - 1., 0)).numpy()
print(decode_predictions_imagenet21k(pred)[0])
# [('n02124075', 'Egyptian_cat', 0.08923767), ('n02122298', 'kitty, kitty-cat, puss, pussy, pussycat', 0.0891152), ...]

If want the imagenet fine-tuned weights, set pretrained="imagenet21k-ft1k".

[romanvelichkin]

I got it, thank you a lot!

[romanvelichkin]

When I use my own value of num_classes I get same error:

WARNING:tensorflow:Skipping loading of weights for layer predictions due to mismatch in shape ((1280, 30) vs (1280, 21843)).
WARNING:tensorflow:Skipping loading of weights for layer predictions due to mismatch in shape ((30,) vs (21843,)).

So layer before output layer doesn't get weights and model shows worse results than same model from tensorflow.hub.

[leondgarse]

It's just the header layer weights skipped, as the log printed. All layers before that got the pretrained weights. You need to fine-tune a new header for using on your own dataset, or the output layers are just randomly initialized.

[romanvelichkin]

I've compared your version of network with the one from tensorflow.hub. And results between them weren't close. I just want to say that I don't know how to setup your network so it could be same as from hub. That's what I want to point out. Maybe some people will have same problem.

But my general interest was in fine-tuning and your network does it great. So thank you!

[leondgarse]

Uh, got it, sorry I'm not sure your level. :)

• Just updated a verification part here efficientnet#verification-with-original-version. The model weights are ported from Official efficientnetv2, not sure if it's same with tf.hub version.
• Here is a colab test of that verification efficientnetV2_basic_test.ipynb. You can also find there a verification of the weights value, between the original model and a model with num_classes=10. They are all same just except the head layer.