Evaluations with YOLOR models

[Medicmind]

I did some training with YOLOR_P6 with coco_train_script.py using my own object detection dataset. I am trying to do evaluations on the trained model using the following code but get the error message:

ValueError: Cannot assign to variable head_1_2_conv/kernel:0 due to variable shape (1, 1, 256, 18) and value shape (45, 256, 1, 1) are incompatible

The error is occuring on mm.load_weights line. Is there another way I should be doing evaluations with YOLOR models?

import tensorflow as tf
from keras_cv_attention_models.coco import datadrion, losses
from keras_cv_attention_models import yolor
from keras_cv_attention_models.imagenet import data
from PIL import Image
import glob
import numpy as np
from keras_cv_attention_models import model_surgery
from keras_cv_attention_models.imagenet import eval_func

image_size=512
mm = yolor.YOLOR_P6((image_size,image_size,3),num_classes=1)
mm.load_weights('checkpoints/YOLOR_P6_256_LAMB_drion_batchsize_64_randaug_6_mosaic_0.5_RRC_1.0_lr512_0.008_wd_0.02_epoch_105_val_cls_acc_1.0000.h5') #, by_name=True) 

fl='image.jpg'
input_data_type=np.float32
img=Image.open(fl)
image = np.array(img.resize((image_size,image_size)), dtype=input_data_type)
if input_data_type == np.float32:
  image = image / 255.
image= np.expand_dims(image, axis=0)  
ynew = mm.predict(image)
print("pred",ynew)

[leondgarse]

I guess you are using coco_train_script.py --det_header yolor.YOLOR_P6 -d {your_dataset}, not with --use_anchor_free_mode, right? As yolor original training method is under working, so by this, it will be yolor + efficientdet anchors... In this case, it needs some extra works. You may try:

from keras_cv_attention_models import yolor
# rescale_mode is "torch" if not specified in training, use "raw01" if need `[0, 1]` inputs value range.
mm = yolor.YOLOR_P6((512, 512, 3), num_classes=1, num_anchors=9, use_object_scores=False, rescale_mode="torch")
mm.load_weights(...)
...
# Add efficientdet anchors
from keras_cv_attention_models.coco import anchors_func
anchors = anchors_func.get_anchors(input_shape=(512, 512), pyramid_levels=[3, 6]) # YOLOR_P6 using 4 features
mm.decode_predictions.anchors = anchors

# predict and decode
img=Image.open(fl)
image = np.array(img.resize((image_size,image_size)), dtype=input_data_type)
pred = mm(mm.preprocess_input(image))
bboxs, lables, confidences = mm.decode_predictions(pred)[0]

# Show result
from keras_cv_attention_models.coco import data
data.show_image_with_bboxes(image, bboxs, lables, confidences)

[leondgarse]

Wait, use coco_train_script.py --det_header yolor.YOLOR_P6 --anchor_pyramid_levels_max 6 {others} instead. In my auto judging, anchor_pyramid_levels_max will be 5 for yolor if not specified. This is not affecting model structure, but anchor shape for building dataset. But it will not throw error even using --anchor_pyramid_levels_max 5. Just model parameters at pyramid_level 6 will never matching any ground truth, for YOLOR_P6 input_shape=(512, 512, 3) it's stack5_spp_output_bn with feature shape == 512 / 2 ** 6 == (8, 8, 320)...

[Medicmind]

I just tried training with --freeze_backbone_epochs as 200 instead of 32 and the evaluation is giving great results. I could try with --anchor_pyramid_levels_max 6 as well and see if it improves it even more. But even after 200 epochs the val loss was still dropping so should probably try 1000 epochs instead of 200.

[leondgarse]

• The above issue just fixed, I'm calculating anchor_pyramid_levels_max from model output shape now.
• Ya, freezing backbone may work better for small dataset.
• Another trick may notice is that, For Megvii-BaseDetection/YOLOX using mosaic_mix, they disabled it and added another l1_loss in later epochs. For doing this, you may use something like:

  ./coco_train_script.py --det_header yolor.YOLOR_P6 \
  --mosaic_mix_prob 0 --use_l1_loss \  # Note `use_l1_loss` ONLY works for `use_anchor_free_mode`
  --freeze_backbone_epochs 0 --lr_decay_steps 16 --lr_warmup_steps 0 --lr_base_512 0.002 \
  --pretrained {your_previour_pretrained_model.h5} \
  ...

[Medicmind]

Ive tried the following:

./coco_train_script.py --det_header yolor.YOLOR_P6 \
--mosaic_mix_prob 0 --use_l1_loss \  # Note `use_l1_loss` ONLY works for `use_anchor_free_mode`
--freeze_backbone_epochs 0 --lr_decay_steps 16 --lr_warmup_steps 0 --lr_base_512 0.002 \
--pretrained {your_previour_pretrained_model.h5} \

but for all my evaluations box is always a bit too small like below:

It should be putting a box around the whole optic disc. Do you think I need to change the anchors?

[leondgarse]

I'm not sure about these:

• Is it working well before this fine-tune?
• Which anchor are you using plotting this one? It should still be the efficientdet anchors.

Both anchor free mode / efficientdet anchors strategy should work, though I haven't testing them combining with yolor, just they are giving correct training results in my basic training tests using yolox / efficientdet. In my understanding, yolor original training strategy is near with efficientdet one, with different anchors and using object scores.

[leondgarse]

• Just started training test using YOLOXTiny myself, for testing configure like random crop method / data augment combination / anchor free or using anchors.
• Considering if using a script building custom tfds dataset is possible.
• I've just updated coco data with applying positional related randaug like rotate / shear / transpose after mosaic mix, which looks like will increase the randomness:

  from keras_cv_attention_models.coco import data
  train_dataset = data.init_dataset(batch_size=16, mosaic_mix_prob=0.5, magnitude=6)[0]
  data.show_batch_sample(train_dataset)

  

[Medicmind]

Thanks. I'll have to try the update

[leondgarse]

• YOLOR original anchor assigning + loss uploaded, but still not tested. Be caution if you wanna a try.
• I have tried YOLOXTiny+ efficientdet anchors, input_shape 256, it seems not working, the training accuracy keeps decreasing. Don't know the reason still... This may also happening in your previous tests.
• I changed default data augment using random_hsv instead of Randaug color related methods, as it's commonly using in yolox / yolor repos. My basic training tests using YOLOXTiny, input_shape 256 also seems better using random_hsv:

  # YOLOXTiny, admaw, input_shape 256, Randaug color related methods
  Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.160
  Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.276
  Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.162
  Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.041
  Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.158
  Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.277
  Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.179
  Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.295
  Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.312
  Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.076
  Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.343
  Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.532
  
  # YOLOXTiny, admaw, input_shape 256, random_hsv
  Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.169
  Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.288
  Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.171
  Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.048
  Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.164
  Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.298
  Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.182
  Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.299
  Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.316
  Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.092
  Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.343
  Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.531