Implement Square Pad and Resize Preprocessors

[f4str]

Description

A new module art.preprocessing.image was created which will contain all image preprocessors. Under this module, new image preprocessors for square padding and resizing were created. Both of these preprocessors work for both the classification and object detection tasks. For classification, the labels are not changed. For object detection, the bounding boxes of the labels are shifted/scaled accordingly based on the padding or resizing.

The Square Pad preprocessor (ImageSquarePad, ImageSquarePadPyTorch, and ImageSquarePadTensorFlowV2) will pad all images to be a square shape. The user is capable of specifying the type of padding (e.g., constant or symmetric) and any additional parameters taken by the corresponding framework-specific padding function. This preprocessor can also take either a ndarray/tensor of the same image size or a list of ndarray/tensor of arbitrary sizes and still square pad accordingly.

The Resize preprocessor (ImageResize, ImageResizePyTorch, and ImageResizeTensorFlowV2) will resize all image to the be same user specified height and width. The user is capable of specifying the interpolation method used for resizing (e.g., bilinear or area). This preprocessor can also take either a ndarray/tensor of the same image size or a list of ndarray/tensor of arbitrary sizes and still resize all images to the same dimensions. This preprocessor will always convert the resized images into one large ndarray/tensor so this is ideal last step for the training pipeline.

For training, the Square Pad preprocessor should be followed by the Resize processor so images of arbitrary sizes can be padded to preserve their aspect ratio and then resized to be fit into a model. These preprocessors are necessary for properly training the YOLO estimators.

Fixes #2130
Fixes #2082

Type of change

Please check all relevant options.

• Improvement (non-breaking)
• Bug fix (non-breaking)
• New feature (non-breaking)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Testing

Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.

• Unit tests for Square Pad preprocessor
• Unit tests for Resize preprocessor

Test Configuration:

• OS
• Python version
• ART version or commit number
• TensorFlow / Keras / PyTorch / MXNet version

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[codecov-commenter]

Codecov Report

Merging #2138 (7a6eac3) into dev_1.15.0 (43d13fc) will increase coverage by 0.01%.
The diff coverage is 87.78%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@              Coverage Diff               @@
##           dev_1.15.0    #2138      +/-   ##
==============================================
+ Coverage       85.64%   85.66%   +0.01%     
==============================================
  Files             299      306       +7     
  Lines           26656    27047     +391     
  Branches         4878     4968      +90     
==============================================
+ Hits            22830    23170     +340     
- Misses           2581     2604      +23     
- Partials         1245     1273      +28     

Impacted Files	Coverage Δ
art/preprocessing/image/image_square_pad/numpy.py	86.36% <86.36%> (ø)
...preprocessing/image/image_square_pad/tensorflow.py	86.36% <86.36%> (ø)
...rt/preprocessing/image/image_square_pad/pytorch.py	86.56% <86.56%> (ø)
art/preprocessing/image/image_resize/tensorflow.py	88.52% <88.52%> (ø)
art/preprocessing/image/image_resize/pytorch.py	88.70% <88.70%> (ø)
art/preprocessing/image/image_resize/numpy.py	88.88% <88.88%> (ø)
art/defences/preprocessor/preprocessor.py	78.63% <100.00%> (ø)
art/preprocessing/image/__init__.py	100.00% <100.00%> (ø)

... and 2 files with indirect coverage changes

[beat-buesser]

Hi @f4str Thank you very much for your pull request! I have added a few questions, what do you think?

[f4str]

Hi @beat-buesser thank you for the comments! I've addressed the review comments and have made the corresponding changes. Feel free to let me know if there any other issues.

[beat-buesser]

Hi @f4str Thank you very much for contributing image preprocessors to ART!