[Feature] Support Multi-task.

[piercus]

Motivation

Multi-tasking

This is inspired from #675 and #808

Inhomogenous dataset

One of the main change compared to #675 is to take into account some inhomogenous dataset.

By inhomogenous dataset, i mean, some sample may have ground truth value for one task, but not for another task, and we need to use mask in the mult-task training logic in order to modulate which sample to take into account with which loss.

1.x

We tried to follow 1.X standard as much as possible

Made with open-mmlab in mind, not only mmcls

The main idea is to provide a generic and flexible multi-task toolbox.

This is for mmclassification, but we tried to limit the deps between the MultiTask Implementation and mmcls, which is mostly related to the data_sample types that can be handled.

We hope this can be discussed further, and we think making a more generic version of it may be part of another PR.

Modification

Modifications are :

• Dataset MultiTaskDataset + unit tests
• Pipeline FormatMultiTaskLabels + unit tests
• Metric MultiTaskMetric + unit tests
• Head MultiTaskHead + unit tests
• Structure MultiTaskDataSample + unit tests

BC-breaking (Optional)

No

Use cases (Optional)

Simple use case

The basic use case here it to classify multiple classes using the same neural network.

This can be useful if you want to "superclass" ImageNet for example (see https://robustness.readthedocs.io/en/latest/example_usage/custom_imagenet.html)

Nested use case

If you have 4 classification task, 2 about classifying the background, and 2 about classiying the foreground, you can decide to organize your task like

dict(type= 'MultiTaskHead', task_heads={
  "foreground": dict(type= 'LinearMultiTaskHead', task_heads={
    "foreground0": dict(type= 'LinearClsHead', num_classes=3),
    "foreground1": dict(type= 'LinearClsHead', num_classes=3)
  }),
  "background": dict(type= 'LinearMultiTaskHead', task_heads={
    "background0": dict(type= 'LinearClsHead', num_classes=3),
    "background1": dict(type= 'LinearClsHead', num_classes=3)
  })
})

By doing this, you can share layers between some tasks

Checklist

Before PR:

• Pre-commit or other linting tools are used to fix the potential lint issues.
• Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
• The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• The documentation has been modified accordingly, like docstring
• example tutorials not done yet.

After PR:

• If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
• CLA has been signed and all committers have signed the CLA in this PR.

[piercus]

@mzr1996 we have updated metrics and unit tests following your suggestions.

It creates unseen situations as input of following functions :

• In the predict function, data_samples type is now List[ClsDataSample | None] (instead of List[ClsDataSample])
• In the metrics, the input data_sample may have ground truth value and predictions or only predictions, we are checking each data_sample one by one

Please share your thoughts

[piercus]

@mzr1996 an eval_mask has been added please share your feedback

[piercus]

Thanks @marouaneamz for your work on this ! This has been a great internship :-)