[Fix Serious BUG] Using Albu will crash the trainning

[PeterH0323]

Motivation

When I use Albu for dataset aug, It cause error :

Modification

I debug and found out that becuase the Albu will change the label from array(x) to array([x]), which will crash the trainning.

And I also find out that there are no field gt_labels in all configs file in mmcls. So I think is inherit from mmdet.

Becuase of that I correct it in mmcls with gt_label and added some code to change it back to array(x) after albu change it to incorrect type array([x])

Thx for @Ezra-Yu ,we can find it in here:

https://github.com/albumentations-team/albumentations/blob/a8dc46ee29f9573c8569213f0d243254980b02f1/albumentations/core/composition.py#L276-L282

Note

If need to test this PR before mmcls merge into master branch, we need to reinstall mmcls after modefied this file.

pip uninstall mmcls -y && pip install -e .

Config

train = dict(
    type='ImageNet',
    pipeline=[
        dict(type='LoadImageFromFile'),
        dict(
            type='Albu',
            transforms=[
                dict(
                    type='ShiftScaleRotate',
                    shift_limit=0.0625,
                    scale_limit=0.0,
                    rotate_limit=0,
                    interpolation=1,
                    p=0.5),
                dict(
                    type='RandomBrightnessContrast',
                    brightness_limit=[0.1, 0.3],
                    contrast_limit=[0.1, 0.3],
                    p=0.2),
                dict(type='ChannelShuffle', p=0.1),
                dict(
                    type='OneOf',
                    transforms=[
                        dict(type='Blur', blur_limit=3, p=1.0),
                        dict(type='MedianBlur', blur_limit=3, p=1.0)
                    ],
                    p=0.1),
            ]),
            dict(type='ImageToTensor', keys=['img']),
            dict(type='ToTensor', keys=['gt_label']),
            dict(type='Collect', keys=['img', 'gt_label'])
    ]
),

Checklist

Before PR:

• Pre-commit or other linting tools are used to fix the potential lint issues.
• Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
• The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

• If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
• CLA has been signed and all committers have signed the CLA in this PR.

[PeterH0323]

Hi @mzr1996 @Ezra-Yu
Pls take a look, I think this fix is very important for mmcls 😄

[Ezra-Yu]

Thank you for your report and PR.

I debug and found out that because the Albu will change the label from array(x) to array([x])

So, this is caused by some transforms in 'Albu' that will change the gt_label field in the results dict, which is necessary for the detection and segmentation task, but useless in the classification task.

[PeterH0323]

Thank you for your report and PR.

I debug and found out that because the Albu will change the label from array(x) to array([x])

So, this is caused by some transforms in 'Albu' that will change the gt_label field in the results dict, which is necessary for the detection and segmentation task, but useless in the classification task.

Hi @Ezra-Yu

I tested it , is not some transforms but is all transforms in Albu will change the gt_label shape.

When I train my model, the gt_label is necessary when calculate the loss.
https://github.com/open-mmlab/mmclassification/blob/11df205e399fb6d6367b3040e1b261563c8a13e5/mmcls/models/heads/cls_head.py#L48-L49

And if I don't set gt_label in my config, the trainning won't process with an error: missing gt_label.

[Ezra-Yu]

Just modifying the key 'gt_labels' to 'gt_label' in the original code will solve the bug. And your comment will help people understand the code.

        if 'gt_label' in results:
            if isinstance(results['gt_label'], list):
                results['gt_label'] = np.array(results['gt_label'])
            results['gt_label'] = results['gt_label'].astype(np.int64)

Your code may cause a potential bug in multi-label tasks. If we use transforms of Albu, which would not change the 'gt_label' in multi-label tasks. The results['gt_label'] don't need to be modified. But results['gt_label'].shape != (), multi-label would be change to single label.

[PeterH0323]

In my case, this can't solved the problem, the shape is not equal after results['gt_label'].astype(np.int64)

In my lastest commit, I using copy.deepcopy to backup the original gt_label and update it to dict results after aug. I think this way we can make sure it is the same.

[codecov]

Codecov Report

Merging #918 (f8afc67) into dev (812f3d4) will decrease coverage by 0.85%.
The diff coverage is 0.00%.

❗ Current head f8afc67 differs from pull request most recent head 57517b5. Consider uploading reports for the commit 57517b5 to get more accurate results

@@            Coverage Diff             @@
##              dev     #918      +/-   ##
==========================================
- Coverage   85.45%   84.60%   -0.86%     
==========================================
  Files         132      132              
  Lines        8750     8747       -3     
  Branches     1513     1512       -1     
==========================================
- Hits         7477     7400      -77     
- Misses       1050     1114      +64     
- Partials      223      233      +10     

Flag	Coverage Δ
unittests	84.60% <0.00%> (-0.79%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmcls/datasets/pipelines/transforms.py	88.63% <0.00%> (+0.36%)	⬆️
mmcls/models/backbones/convnext.py	60.00% <0.00%> (-34.79%)	⬇️
mmcls/utils/setup_env.py	72.72% <0.00%> (-22.73%)	⬇️
mmcls/models/utils/helpers.py	79.16% <0.00%> (-20.84%)	⬇️
mmcls/datasets/custom.py	88.52% <0.00%> (-11.48%)	⬇️
mmcls/models/backbones/convmixer.py	88.52% <0.00%> (-9.84%)	⬇️
mmcls/datasets/builder.py	78.37% <0.00%> (-9.46%)	⬇️
mmcls/models/backbones/cspnet.py	89.54% <0.00%> (-3.27%)	⬇️
mmcls/core/visualization/image.py	86.20% <0.00%> (-0.98%)	⬇️
mmcls/datasets/dataset_wrappers.py	71.83% <0.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 11df205...57517b5. Read the comment docs.

[Ezra-Yu]

please add a case to the unit test to check the gt_label.
such as ShiftScaleRotate :

dict(
        type='Albu',
        transforms=[
            dict(
                type='ShiftScaleRotate',
                shift_limit=0.0625,
                scale_limit=0.0,
                rotate_limit=0,
                interpolation=1,
                p=1)
        ]),

[PeterH0323]

please add a case to the unit test to check the gt_label. such as ShiftScaleRotate :

dict(
        type='Albu',
        transforms=[
            dict(
                type='ShiftScaleRotate',
                shift_limit=0.0625,
                scale_limit=0.0,
                rotate_limit=0,
                interpolation=1,
                p=1)
        ]),

Done, I improved the func named test_albu_transform of file tests/test_data/test_pipelines/test_transform.py

[Ezra-Yu]

LGTM.