has_mask may fail on mmdet mmdetection model with {with_mask:False}

[ccomkhj]

from #1054, keyword mask is looked into to check if has_mask is True or False.
However, under the below condition {'with_mask: False}, it fails.

self.model.model.cfg["train_dataloader"]["dataset"]["pipeline"]
[{'backend_args': None, 'type': 'LoadImageFromFile'}, {'type': 'LoadAnnotations', 'with_bbox': True, 'with_mask': False}, {'transforms': [...], 'type': 'RandomChoice'}, {'prob': 0.5, 'type': 'RandomFlip'}, {'type': 'PackDetInputs'}]
train_pipeline = self.model.model.cfg["train_dataloader"]["dataset"]["pipeline"]
train_pipeline
[{'backend_args': None, 'type': 'LoadImageFromFile'}, {'type': 'LoadAnnotations', 'with_bbox': True, 'with_mask': False}, {'transforms': [...], 'type': 'RandomChoice'}, {'prob': 0.5, 'type': 'RandomFlip'}, {'type': 'PackDetInputs'}]
any(isinstance(item, dict) and any("mask" in key for key in item.keys()) for item in train_pipeline)
True

solution could be

@property
    def has_mask(self):
        """
        Returns if model output contains segmentation mask
        """
        # has_mask = self.model.model.with_mask
        train_pipeline = self.model.cfg["train_dataloader"]["dataset"]["pipeline"]
        # has_mask = any(isinstance(item, dict) and any("mask" in key for key in item.keys()) for item in train_pipeline) # AS-IS
        has_mask = any(isinstance(item, dict) and any('mask' in key and value for key, value in item.items()) for item in train_pipeline) # TO-BE
        return has_mask

Shall I make pr for this?

[fcakyon]

@ccomkhj its a great catch, it would be amazing if you could open a PR 🚀

Please update it as ...'mask' in key and value is True for key... so that the code is more explicit and intuitive 👍🏻

[Alias-z]

from #1054, keyword mask is looked into to check if has_mask is True or False.

However, under the below condition {'with_mask: False}, it fails.

self.model.model.cfg["train_dataloader"]["dataset"]["pipeline"]

[{'backend_args': None, 'type': 'LoadImageFromFile'}, {'type': 'LoadAnnotations', 'with_bbox': True, 'with_mask': False}, {'transforms': [...], 'type': 'RandomChoice'}, {'prob': 0.5, 'type': 'RandomFlip'}, {'type': 'PackDetInputs'}]

train_pipeline = self.model.model.cfg["train_dataloader"]["dataset"]["pipeline"]

train_pipeline

[{'backend_args': None, 'type': 'LoadImageFromFile'}, {'type': 'LoadAnnotations', 'with_bbox': True, 'with_mask': False}, {'transforms': [...], 'type': 'RandomChoice'}, {'prob': 0.5, 'type': 'RandomFlip'}, {'type': 'PackDetInputs'}]

any(isinstance(item, dict) and any("mask" in key for key in item.keys()) for item in train_pipeline)

True

solution could be

@property

    def has_mask(self):

        """

        Returns if model output contains segmentation mask

        """

        # has_mask = self.model.model.with_mask

        train_pipeline = self.model.cfg["train_dataloader"]["dataset"]["pipeline"]

        # has_mask = any(isinstance(item, dict) and any("mask" in key for key in item.keys()) for item in train_pipeline) # AS-IS

        has_mask = any(isinstance(item, dict) and any('mask' in key and value for key, value in item.items()) for item in train_pipeline) # TO-BE

        return has_mask

Shall I make pr for this?

Hi, could you please provide the config file so I can try it? 🙂 Maybe it is a special case that I missed before.

[ccomkhj]

from #1054, keyword mask is looked into to check if has_mask is True or False.
However, under the below condition {'with_mask: False}, it fails.

self.model.model.cfg["train_dataloader"]["dataset"]["pipeline"]

[{'backend_args': None, 'type': 'LoadImageFromFile'}, {'type': 'LoadAnnotations', 'with_bbox': True, 'with_mask': False}, {'transforms': [...], 'type': 'RandomChoice'}, {'prob': 0.5, 'type': 'RandomFlip'}, {'type': 'PackDetInputs'}]

train_pipeline = self.model.model.cfg["train_dataloader"]["dataset"]["pipeline"]

train_pipeline

[{'backend_args': None, 'type': 'LoadImageFromFile'}, {'type': 'LoadAnnotations', 'with_bbox': True, 'with_mask': False}, {'transforms': [...], 'type': 'RandomChoice'}, {'prob': 0.5, 'type': 'RandomFlip'}, {'type': 'PackDetInputs'}]

any(isinstance(item, dict) and any("mask" in key for key in item.keys()) for item in train_pipeline)

True

solution could be

@property

    def has_mask(self):

        """

        Returns if model output contains segmentation mask

        """

        # has_mask = self.model.model.with_mask

        train_pipeline = self.model.cfg["train_dataloader"]["dataset"]["pipeline"]

        # has_mask = any(isinstance(item, dict) and any("mask" in key for key in item.keys()) for item in train_pipeline) # AS-IS

        has_mask = any(isinstance(item, dict) and any('mask' in key and value for key, value in item.items()) for item in train_pipeline) # TO-BE

        return has_mask

Shall I make pr for this?

Hi, could you please provide the config file so I can try it? 🙂 Maybe it is a special case that I missed before.

Hi @Alias-z , below is the config. (I deleted some confidential part regarding dataset but I believe it's sufficient to see the difference.)

auto_scale_lr = dict(base_batch_size=4, enable=False)
backend_args = None
base_lr = 0.0002
batch_size = 4
crop_size = (
    1080,
    1080,
)
data_root = './data/2024-07-04_12-40-42/1080'
dataset_type = 'CocoDataset'
default_hooks = dict(
    checkpoint=dict(
        interval=24,
        max_keep_ckpts=1,
        save_best='coco/bbox_mAP_50',
        save_last=False,
        save_optimizer=False,
        save_param_scheduler=False,
        type='CheckpointHook'),
    logger=dict(interval=200, type='LoggerHook'),
    param_scheduler=dict(type='ParamSchedulerHook'),
    sampler_seed=dict(type='DistSamplerSeedHook'),
    timer=dict(type='IterTimerHook'),
    visualization=dict(type='DetVisualizationHook'))
default_scope = 'mmdet'
env_cfg = dict(
    cudnn_benchmark=False,
    dist_cfg=dict(backend='nccl'),
    mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0))
epoch_interval = 24
img_norm_cfg = dict(
    mean=[
        123.675,
        116.28,
        103.53,
    ],
    std=[
        58.395,
        57.12,
        57.375,
    ],
    to_rgb=True)
launcher = 'none'
load_from = None
log_level = 'INFO'
log_processor = dict(by_epoch=False, type='LogProcessor', window_size=200)
max_epochs = 480
meta_info = None
model = dict(
    backbone=dict(
        base_width=4,
        depth=101,
        frozen_stages=1,
        groups=32,
        init_cfg=dict(
            checkpoint='open-mmlab://resnext101_32x4d', type='Pretrained'),
        norm_cfg=dict(requires_grad=True, type='BN'),
        num_stages=4,
        out_indices=(
            0,
            1,
            2,
            3,
        ),
        style='pytorch',
        type='ResNeXt'),
    bbox_head=dict(
        anchor_generator=dict(
            octave_base_scale=8,
            ratios=[
                1.0,
            ],
            scales_per_octave=1,
            strides=[
                8,
                16,
                32,
                64,
                128,
            ],
            type='AnchorGenerator'),
        bbox_coder=dict(
            target_means=[
                0.0,
                0.0,
                0.0,
                0.0,
            ],
            target_stds=[
                0.1,
                0.1,
                0.2,
                0.2,
            ],
            type='DeltaXYWHBBoxCoder'),
        feat_channels=256,
        in_channels=256,
        loss_bbox=dict(loss_weight=2.0, type='GIoULoss'),
        loss_cls=dict(
            alpha=0.25,
            gamma=2.0,
            loss_weight=1.0,
            type='FocalLoss',
            use_sigmoid=True),
        loss_iou=dict(
            loss_weight=1.0, type='CrossEntropyLoss', use_sigmoid=True),
        num_classes=7,
        stacked_convs=4,
        type='DDODHead'),
    data_preprocessor=dict(
        bgr_to_rgb=True,
        mean=[
            123.675,
            116.28,
            103.53,
        ],
        pad_size_divisor=32,
        std=[
            58.395,
            57.12,
            57.375,
        ],
        type='DetDataPreprocessor'),
    neck=dict(
        add_extra_convs='on_output',
        in_channels=[
            256,
            512,
            1024,
            2048,
        ],
        num_outs=5,
        out_channels=256,
        start_level=1,
        type='FPN'),
    test_cfg=dict(
        max_per_img=100,
        min_bbox_size=0,
        nms=dict(iou_threshold=0.6, type='nms'),
        nms_pre=1000,
        score_thr=0.05),
    train_cfg=dict(
        allowed_border=-1,
        assigner=dict(alpha=0.8, topk=9, type='ATSSAssigner'),
        debug=False,
        pos_weight=-1,
        reg_assigner=dict(alpha=0.5, topk=9, type='ATSSAssigner')),
    type='DDOD')
num_classes = 7
optim_wrapper = dict(
    clip_grad=dict(max_norm=35, norm_type=2),
    optimizer=dict(lr=0.0002, type='AdamW', weight_decay=0.05),
    type='OptimWrapper')
param_scheduler = [
    dict(
        begin=0, by_epoch=False, end=500, start_factor=1e-05, type='LinearLR'),
    dict(
        T_max=480,
        begin=0,
        by_epoch=True,
        convert_to_iter_based=True,
        end=480,
        eta_min_ratio=0.01,
        type='CosineAnnealingLR'),
]
resume = False
test_cfg = dict(type='TestLoop')
test_dataloader = dict(
    batch_size=1,
    dataset=dict(
        ann_file=
        './data/2024-07-04_12-40-42/1080/val_sliced.json',
        backend_args=None,
        data_prefix=dict(
            img=
            './data/2024-07-04_12-40-42/1080/val_images_sliced'
        ),
        data_root=None,
        metainfo=None),
        pipeline=[
            dict(backend_args=None, type='LoadImageFromFile'),
            dict(keep_ratio=True, scale=(
                1080,
                1080,
            ), type='Resize'),
            dict(type='LoadAnnotations', with_bbox=True, with_mask=False),
            dict(
                meta_keys=(
                    'img_id',
                    'img_path',
                    'ori_shape',
                    'img_shape',
                    'scale_factor',
                ),
                type='PackDetInputs'),
        ],
        test_mode=True,
        type='CocoDataset'),
    drop_last=False,
    num_workers=2,
    persistent_workers=True,
    sampler=dict(shuffle=False, type='DefaultSampler'))
test_evaluator = dict(
    ann_file=
    './data/2024-07-04_12-40-42/1080/val_sliced.json',
    backend_args=None,
    classwise=True,
    format_only=False,
    metric='bbox',
    type='CocoMetric')
test_pipeline = [
    dict(backend_args=None, type='LoadImageFromFile'),
    dict(keep_ratio=True, scale=(
        1080,
        1080,
    ), type='Resize'),
    dict(type='LoadAnnotations', with_bbox=True, with_mask=False),
    dict(
        meta_keys=(
            'img_id',
            'img_path',
            'ori_shape',
            'img_shape',
            'scale_factor',
        ),
        type='PackDetInputs'),
]
train_cfg = dict(max_epochs=480, type='EpochBasedTrainLoop', val_interval=24)
train_dataloader = dict(
    batch_sampler=dict(type='AspectRatioBatchSampler'),
    batch_size=4,
    dataset=dict(
        ann_file=
        './data/2024-07-04_12-40-42/1080/train.json',
        backend_args=None,
        data_prefix=dict(
            img=
            './data/2024-07-04_12-40-42/1080/train_images'
        ),
        data_root=None,
        filter_cfg=dict(filter_empty_gt=True, min_size=32),
        metainfo=None,
        pipeline=[
            dict(backend_args=None, type='LoadImageFromFile'),
            dict(type='LoadAnnotations', with_bbox=True, with_mask=False),
            dict(
                transforms=[
                    [
                        dict(
                            img_border_value=(
                                0,
                                0,
                                0,
                            ),
                            level=5,
                            max_mag=180.0,
                            prob=0.5,
                            type='Rotate'),
                        dict(
                            allow_negative_crop=True,
                            crop_size=(
                                1080,
                                1080,
                            ),
                            crop_type='absolute_range',
                            type='RandomCrop'),
                    ],
                    [
                        dict(type='RandomAffine'),
                        dict(
                            allow_negative_crop=True,
                            crop_size=(
                                1080,
                                1080,
                            ),
                            crop_type='absolute_range',
                            type='RandomCrop'),
                    ],
                    [
                        dict(
                            hue_delta=5,
                            saturation_delta=30,
                            type='YOLOXHSVRandomAug',
                            value_delta=1),
                        dict(
                            allow_negative_crop=True,
                            crop_size=(
                                1080,
                                1080,
                            ),
                            crop_type='absolute_range',
                            type='RandomCrop'),
                    ],
                ],
                type='RandomChoice'),
            dict(prob=0.5, type='RandomFlip'),
            dict(type='PackDetInputs'),
        ],
        type='CocoDataset'),
    num_workers=8,
    persistent_workers=True,
    sampler=dict(shuffle=True, type='DefaultSampler'))
train_pipeline = [
    dict(backend_args=None, type='LoadImageFromFile'),
    dict(type='LoadAnnotations', with_bbox=True, with_mask=False),
    dict(
        transforms=[
            [
                dict(
                    img_border_value=(
                        0,
                        0,
                        0,
                    ),
                    level=5,
                    max_mag=180.0,
                    prob=0.5,
                    type='Rotate'),
                dict(
                    allow_negative_crop=True,
                    crop_size=(
                        1080,
                        1080,
                    ),
                    crop_type='absolute_range',
                    type='RandomCrop'),
            ],
            [
                dict(type='RandomAffine'),
                dict(
                    allow_negative_crop=True,
                    crop_size=(
                        1080,
                        1080,
                    ),
                    crop_type='absolute_range',
                    type='RandomCrop'),
            ],
            [
                dict(
                    hue_delta=5,
                    saturation_delta=30,
                    type='YOLOXHSVRandomAug',
                    value_delta=1),
                dict(
                    allow_negative_crop=True,
                    crop_size=(
                        1080,
                        1080,
                    ),
                    crop_type='absolute_range',
                    type='RandomCrop'),
            ],
        ],
        type='RandomChoice'),
    dict(prob=0.5, type='RandomFlip'),
    dict(type='PackDetInputs'),
]
val_cfg = dict(type='ValLoop')
val_dataloader = dict(
    batch_size=1,
    dataset=dict(
        ann_file=
        './data/2024-07-04_12-40-42/1080/val_sliced.json',
        backend_args=None,
        data_prefix=dict(
            img=
            './data/2024-07-04_12-40-42/1080/val_images_sliced'
        ),
        data_root=None,
        metainfo=None,
        pipeline=[
            dict(backend_args=None, type='LoadImageFromFile'),
            dict(keep_ratio=True, scale=(
                1080,
                1080,
            ), type='Resize'),
            dict(type='LoadAnnotations', with_bbox=True, with_mask=False),
            dict(
                meta_keys=(
                    'img_id',
                    'img_path',
                    'ori_shape',
                    'img_shape',
                    'scale_factor',
                ),
                type='PackDetInputs'),
        ],
        test_mode=True,
        type='CocoDataset'),
    drop_last=False,
    num_workers=2,
    persistent_workers=True,
    sampler=dict(shuffle=False, type='DefaultSampler'))
val_evaluator = dict(
    ann_file=
    './data/2024-07-04_12-40-42/1080/val_sliced.json',
    backend_args=None,
    classwise=True,
    format_only=False,
    metric='bbox',
    type='CocoMetric')
vis_backends = [
    dict(type='LocalVisBackend')
]
visualizer = dict(
    name='visualizer',
    save_dir='temp_dir',
    type='DetLocalVisualizer',
    vis_backends=[
        dict(type='LocalVisBackend')
    ])

[Alias-z]

Sorry, I was too busy to check it. I just realized I also need the model weights to test this. Is it possible for you for share the config and weights somehow? or is is similar to the config of DDOD-ATSS R-50?