[Feature] Add base transform interface

[ly015]

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Please describe the motivation of this PR and the goal you want to achieve through this PR.

This PR introduces a new data transform base interface that consists of a base transform class and several transform wrappers. The new design aims at:

• A unified transform interface (base class)
• Supporting configurable input and output field names in transforms to disentangle datasets and data pipelines
• Scalable and flexible functions supported by transform wrappers

Modification

Please briefly describe what modification is made in this PR.

BC-breaking (Optional)

Does the modification introduce changes that break the backward-compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here and update the documentation.

BaseTransform

BaseTransform is the base class for data transforms. Compared to the old transform design, in the BaseTransform class the transform operation is moved to a method transform() which will be invoked in __call__:

class BaseTransform(metaclass=ABCMeta):

    def __call__(self, results: Dict) -> Dict:

        return self.transform(results)

    @abstractmethod
    def transform(self, results: Dict) -> Dict:
        pass

For those transforms that involve random behavior (e.g. RandomFlip), the intermediate random parameters should be generated by a method decorated with cacheable_method. Below is an example. This allows caching the random parameters to apply identical transform on multiple data samples. See ApplyToMultiple for more details.

Class RandomHorizontalFlip(BaseTransform):

    def __init__(self, p):
        super().__init__()
        self.p = p

    @cacheable_method
    def do_flip(self):
        return np.random.rand() < self.p

    def transform(self, results):
        if self.do_flip():
            results['img'] = results['img'][:, ::-1]
        return results

Transform Wrappers

A transform wrapper is a subclass of BaseTransform that wraps an inner pipeline and adds additional features. This PR introduces the following wrappers:

• Compose
• Remap
• ApplyToMultiple
• RandomChoice

Compose

Compose is just a reimplementation of the original one with the inheritance of BaseTransform. It provides a __iter__ method to support easy iteration over the wrapped transforms.

Remap

Remap supports remapping the filed names of the input and output data of the wrapped transforms. Below is an use case example:

# config
pipeline = [
    dict(type='LoadImageFromFile', key='gt_img'),
    # Use Remap to convert outer (original) field name 'gt_img'
    # to inner (used by inner transforms) filed name 'img'
    dict(type='Remap',
        input_mapping=dict(img='gt_img'),  # The input key mapping
        inplace=True,  # The output key mapping is the revert of the input key mapping, which means mapping 'img' back to 'gt_img'
        transforms=[
            # In all transforms' implementation just use 'img' as a standard field name
            dict(type='Crop', crop_size=(384, 384), random_crop=True),
            dict(type='Normalize'),
        ])
]

ApplyToMultiple

ApplyToMultiple applies identical transforms (defined by wrapped pipeline) on multiple data fields. Below is a use case example, where the transform is applied on both 'lq' (low-quality image) and 'gt' (ground-truth image):

# config
pipeline = [
    dict(type='LoadImageFromFile', key='lq'),  # low-quality image
    dict(type='LoadImageFromFile', key='gt'),  # ground-truth image
    # ApplyToMultiple maps multiple outer fields to standard the inner field and process it with wrapped
    # transforms successively
    dict(type='ApplyToMultiple',
        input_mapping=dict(img=['lq', 'gt']),  # case 1: from multiple outer fields
        # input_mapping=dict(img='images'),  # case 2: from one outer field that contains multiple data elements (e.g. a list)
        inplace=True,
        share_random_param=True,  # Use identical random parameters in every processing
        transforms=[
            dict(type='Crop', crop_size=(384, 384), random_crop=True),
            dict(type='Normalize'),
        ])
]

RandomChoice

RandomChoice wraps multiple pipelines, and randomly chooses one to process the data each time.

# config
pipeline = [
    dict(type='RandomChoice',
        pipelines=[
            [dict(type='RandomHorizontalFlip')],  # subpipeline 1
            [dict(type='RandomRotate')],  # subpipeline 2
        ]
    )
]

Checklist

Before PR:

• I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
• Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
• Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
• New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

• If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects, like MMDet or MMCls.
• CLA has been signed and all committers have signed the CLA in this PR.

[mzr1996]

Look good

[ZwwWayne]

description is missed.

[ZwwWayne]

Remap: output_mapping must be None if ...

[ZwwWayne]

description is missed.

[ZwwWayne]

i.e. kwargs -> i.e., kwargs

[ZwwWayne]

It would be better if the coverage rate if above 85-90%

[zhouzaida]

Can we directly use the inspect.ismethod(func)?

[ly015]

I am afraid we can't. This check happens before the function is bound as a method.

[zhouzaida]

Suggested change

	transforms (list[dict | callable]): Either config
	dicts of transforms or transform objects.
	transforms (list[dict | callable]): Either config
	dicts of transforms or transform objects.

[zhouzaida]

Why do we provide an option self.strict that ignores the missing keys that are not in the results?

[ly015]

It helps to allow more flexible use of the key mapping. For example, for transforms like Resize and Flip, a key mapping defined for aligned img, bbox can also handle the case where there is only img.
Do you think this option is necessary?

[zhouzaida]

It is necessary and should we print the warning log at the same time?

[ly015]

I think the warning would be unnecessary. Here we simply provide a way to allow optional keys, and it should be the wrapped transform to determine whether to raise an exception.

[zhouzaida]

Suggested change

	dict: The input data with remapped keys. This will be the actual
	input of the wrapped pipeline.
	dict: The input data with remapped keys. This will be the actual
	input of the wrapped pipeline.

[ly015]

Shouldn't there be an indent?

[ZwwWayne]

Some of the files need more unit tests.

[ZwwWayne]

We can resolve the comments and merge this PR today. The remaining UT can be added later.