Reproduction

[dfsae]

Thank you for this great research!

I'm working on a reproduction of thumos'14 dataset as you explained on github, but there are so different results, can you tell me what's wrong?

[klauscc]

Could you share your training log? I cannot know the reason without any information but I guess the most possible reason is that you didn't load the pretrained checkpoint (Swin) properly.

[dfsae]

Hi @klauscc, thanks for the fast reply.

Thank you for your answer!

I'm sorry I didn't provide any information when I asked you

Below is part of the training record you requested from me.

Thank you

2023-05-08 11:22:46,617 - vedatad - WARNING - EvalHook is not in modes ['train']
2023-05-08 11:22:46,617 - vedatad - INFO - Loading weights from ./swin_base_patch244_window877_kinetics400_22k.pth
2023-05-08 11:22:47,061 - vedatad - WARNING - The model and loaded state dict do not match exactly

unexpected key in source state_dict: cls_head.fc_cls.weight, cls_head.fc_cls.bias

missing keys in source state_dict: backbone.layers.0.blocks.0.dummy_tensor, backbone.layers.0.blocks.1.dummy_tensor, backbone.layers.1.blocks.0.dummy_tensor, backbone.layers.1.blocks.1.dummy_tensor, backbone.layers.2.blocks.0.dummy_tensor, backbone.layers.2.blocks.1.dummy_tensor, backbone.layers.2.blocks.2.dummy_tensor, backbone.layers.2.blocks.3.dummy_tensor, backbone.layers.2.blocks.4.dummy_tensor, backbone.layers.2.blocks.5.dummy_tensor, backbone.layers.2.blocks.6.dummy_tensor, backbone.layers.2.blocks.7.dummy_tensor, backbone.layers.2.blocks.8.dummy_tensor, backbone.layers.2.blocks.9.dummy_tensor, backbone.layers.2.blocks.10.dummy_tensor, backbone.layers.2.blocks.11.dummy_tensor, backbone.layers.2.blocks.12.dummy_tensor, backbone.layers.2.blocks.13.dummy_tensor, backbone.layers.2.blocks.14.dummy_tensor, backbone.layers.2.blocks.15.dummy_tensor, backbone.layers.2.blocks.16.dummy_tensor, backbone.layers.2.blocks.17.dummy_tensor, backbone.layers.3.blocks.0.dummy_tensor, backbone.layers.3.blocks.1.dummy_tensor, neck.0.conv1.weight, neck.0.conv1.bias, neck.0.conv2.weight, neck.0.conv2.bias, neck.1.encoder.layers.0.norm1.weight, neck.1.encoder.layers.0.norm1.bias, neck.1.encoder.layers.0.attn.relative_position_bias_table, neck.1.encoder.layers.0.attn.relative_position_index, neck.1.encoder.layers.0.attn.qkv.weight, neck.1.encoder.layers.0.attn.qkv.bias, neck.1.encoder.layers.0.attn.proj.weight, neck.1.encoder.layers.0.attn.proj.bias, neck.1.encoder.layers.0.norm2.weight, neck.1.encoder.layers.0.norm2.bias, neck.1.encoder.layers.0.mlp.fc1.weight, neck.1.encoder.layers.0.mlp.fc1.bias, neck.1.encoder.layers.0.mlp.fc2.weight, neck.1.encoder.layers.0.mlp.fc2.bias, neck.1.encoder.layers.1.norm1.weight, neck.1.encoder.layers.1.norm1.bias, neck.1.encoder.layers.1.attn.relative_position_bias_table, neck.1.encoder.layers.1.attn.relative_position_index, neck.1.encoder.layers.1.attn.qkv.weight, neck.1.encoder.layers.1.attn.qkv.bias, neck.1.encoder.layers.1.attn.proj.weight, neck.1.encoder.layers.1.attn.proj.bias, neck.1.encoder.layers.1.norm2.weight, neck.1.encoder.layers.1.norm2.bias, neck.1.encoder.layers.1.mlp.fc1.weight, neck.1.encoder.layers.1.mlp.fc1.bias, neck.1.encoder.layers.1.mlp.fc2.weight, neck.1.encoder.layers.1.mlp.fc2.bias, neck.1.encoder.layers.2.norm1.weight, neck.1.encoder.layers.2.norm1.bias, neck.1.encoder.layers.2.attn.relative_position_bias_table, neck.1.encoder.layers.2.attn.relative_position_index, neck.1.encoder.layers.2.attn.qkv.weight, neck.1.encoder.layers.2.attn.qkv.bias, neck.1.encoder.layers.2.attn.proj.weight, neck.1.encoder.layers.2.attn.proj.bias, neck.1.encoder.layers.2.norm2.weight, neck.1.encoder.layers.2.norm2.bias, neck.1.encoder.layers.2.mlp.fc1.weight, neck.1.encoder.layers.2.mlp.fc1.bias, neck.1.encoder.layers.2.mlp.fc2.weight, neck.1.encoder.layers.2.mlp.fc2.bias, neck.2.pe.pe, neck.2.reductions.0.0.weight, neck.2.reductions.0.1.weight, neck.2.reductions.0.1.bias, neck.2.reductions.1.0.weight, neck.2.reductions.1.1.weight, neck.2.reductions.1.1.bias, neck.2.reductions.2.0.weight, neck.2.reductions.2.1.weight, neck.2.reductions.2.1.bias, neck.2.reductions.3.0.weight, neck.2.reductions.3.1.weight, neck.2.reductions.3.1.bias, neck.2.trans_layers.0.self_attn.in_proj_weight, neck.2.trans_layers.0.self_attn.in_proj_bias, neck.2.trans_layers.0.self_attn.out_proj.weight, neck.2.trans_layers.0.self_attn.out_proj.bias, neck.2.trans_layers.0.linear1.weight, neck.2.trans_layers.0.linear1.bias, neck.2.trans_layers.0.linear2.weight, neck.2.trans_layers.0.linear2.bias, neck.2.trans_layers.0.norm1.weight, neck.2.trans_layers.0.norm1.bias, neck.2.trans_layers.0.norm2.weight, neck.2.trans_layers.0.norm2.bias, neck.2.trans_layers.1.self_attn.in_proj_weight, neck.2.trans_layers.1.self_attn.in_proj_bias, neck.2.trans_layers.1.self_attn.out_proj.weight, neck.2.trans_layers.1.self_attn.out_proj.bias, neck.2.trans_layers.1.linear1.weight, neck.2.trans_layers.1.linear1.bias, neck.2.trans_layers.1.linear2.weight, neck.2.trans_layers.1.linear2.bias, neck.2.trans_layers.1.norm1.weight, neck.2.trans_layers.1.norm1.bias, neck.2.trans_layers.1.norm2.weight, neck.2.trans_layers.1.norm2.bias, neck.2.trans_layers.2.self_attn.in_proj_weight, neck.2.trans_layers.2.self_attn.in_proj_bias, neck.2.trans_layers.2.self_attn.out_proj.weight, neck.2.trans_layers.2.self_attn.out_proj.bias, neck.2.trans_layers.2.linear1.weight, neck.2.trans_layers.2.linear1.bias, neck.2.trans_layers.2.linear2.weight, neck.2.trans_layers.2.linear2.bias, neck.2.trans_layers.2.norm1.weight, neck.2.trans_layers.2.norm1.bias, neck.2.trans_layers.2.norm2.weight, neck.2.trans_layers.2.norm2.bias, neck.2.trans_layers.3.self_attn.in_proj_weight, neck.2.trans_layers.3.self_attn.in_proj_bias, neck.2.trans_layers.3.self_attn.out_proj.weight, neck.2.trans_layers.3.self_attn.out_proj.bias, neck.2.trans_layers.3.linear1.weight, neck.2.trans_layers.3.linear1.bias, neck.2.trans_layers.3.linear2.weight, neck.2.trans_layers.3.linear2.bias, neck.2.trans_layers.3.norm1.weight, neck.2.trans_layers.3.norm1.bias, neck.2.trans_layers.3.norm2.weight, neck.2.trans_layers.3.norm2.bias, neck.3.lateral_convs.0.conv.weight, neck.3.lateral_convs.0.bn.weight, neck.3.lateral_convs.0.bn.bias, neck.3.lateral_convs.0.bn.running_mean, neck.3.lateral_convs.0.bn.running_var, neck.3.lateral_convs.1.conv.weight, neck.3.lateral_convs.1.bn.weight, neck.3.lateral_convs.1.bn.bias, neck.3.lateral_convs.1.bn.running_mean, neck.3.lateral_convs.1.bn.running_var, neck.3.lateral_convs.2.conv.weight, neck.3.lateral_convs.2.bn.weight, neck.3.lateral_convs.2.bn.bias, neck.3.lateral_convs.2.bn.running_mean, neck.3.lateral_convs.2.bn.running_var, neck.3.lateral_convs.3.conv.weight, neck.3.lateral_convs.3.bn.weight, neck.3.lateral_convs.3.bn.bias, neck.3.lateral_convs.3.bn.running_mean, neck.3.lateral_convs.3.bn.running_var, neck.3.lateral_convs.4.conv.weight, neck.3.lateral_convs.4.bn.weight, neck.3.lateral_convs.4.bn.bias, neck.3.lateral_convs.4.bn.running_mean, neck.3.lateral_convs.4.bn.running_var, neck.3.fpn_convs.0.conv.weight, neck.3.fpn_convs.0.bn.weight, neck.3.fpn_convs.0.bn.bias, neck.3.fpn_convs.0.bn.running_mean, neck.3.fpn_convs.0.bn.running_var, neck.3.fpn_convs.1.conv.weight, neck.3.fpn_convs.1.bn.weight, neck.3.fpn_convs.1.bn.bias, neck.3.fpn_convs.1.bn.running_mean, neck.3.fpn_convs.1.bn.running_var, neck.3.fpn_convs.2.conv.weight, neck.3.fpn_convs.2.bn.weight, neck.3.fpn_convs.2.bn.bias, neck.3.fpn_convs.2.bn.running_mean, neck.3.fpn_convs.2.bn.running_var, neck.3.fpn_convs.3.conv.weight, neck.3.fpn_convs.3.bn.weight, neck.3.fpn_convs.3.bn.bias, neck.3.fpn_convs.3.bn.running_mean, neck.3.fpn_convs.3.bn.running_var, neck.3.fpn_convs.4.conv.weight, neck.3.fpn_convs.4.bn.weight, neck.3.fpn_convs.4.bn.bias, neck.3.fpn_convs.4.bn.running_mean, neck.3.fpn_convs.4.bn.running_var, head.cls_convs.0.conv.weight, head.cls_convs.0.bn.weight, head.cls_convs.0.bn.bias, head.cls_convs.0.bn.running_mean, head.cls_convs.0.bn.running_var, head.cls_convs.1.conv.weight, head.cls_convs.1.bn.weight, head.cls_convs.1.bn.bias, head.cls_convs.1.bn.running_mean, head.cls_convs.1.bn.running_var, head.cls_convs.2.conv.weight, head.cls_convs.2.bn.weight, head.cls_convs.2.bn.bias, head.cls_convs.2.bn.running_mean, head.cls_convs.2.bn.running_var, head.cls_convs.3.conv.weight, head.cls_convs.3.bn.weight, head.cls_convs.3.bn.bias, head.cls_convs.3.bn.running_mean, head.cls_convs.3.bn.running_var, head.reg_convs.0.conv.weight, head.reg_convs.0.bn.weight, head.reg_convs.0.bn.bias, head.reg_convs.0.bn.running_mean, head.reg_convs.0.bn.running_var, head.reg_convs.1.conv.weight, head.reg_convs.1.bn.weight, head.reg_convs.1.bn.bias, head.reg_convs.1.bn.running_mean, head.reg_convs.1.bn.running_var, head.reg_convs.2.conv.weight, head.reg_convs.2.bn.weight, head.reg_convs.2.bn.bias, head.reg_convs.2.bn.running_mean, head.reg_convs.2.bn.running_var, head.reg_convs.3.conv.weight, head.reg_convs.3.bn.weight, head.reg_convs.3.bn.bias, head.reg_convs.3.bn.running_mean, head.reg_convs.3.bn.running_var, head.retina_cls.weight, head.retina_cls.bias, head.retina_reg.weight, head.retina_reg.bias

2023-05-08 11:23:15,707 - vedatad - INFO - Epoch [1][10/200] lr: 0.001162, loss_cls: 0.0106, loss_segment: 0.0000, loss: 0.0106. max mem allocated: 3944.36865234375 MB. max reservered: 4468.0 MB
2023-05-08 11:23:40,613 - vedatad - INFO - Epoch [1][20/200] lr: 0.001342, loss_cls: 1.0855, loss_segment: 0.4389, loss: 1.5244. max mem allocated: 3944.36865234375 MB. max reservered: 4468.0 MB
2023-05-08 11:24:03,620 - vedatad - INFO - Epoch [1][30/200] lr: 0.001522, loss_cls: 0.0128, loss_segment: 0.0000, loss: 0.0128. max mem allocated: 3944.36865234375 MB. max reservered: 4468.0 MB