Auto adapting batch size

[eunwoosh]

Summary

• Implement a feature to adapt a batch size according to current GPU memory size automatically.
• It supports action, classification, detection and segmentation tasks.

Detail

This feature finds a batch size which uses almost GPU memory. It generally increases batch size, so, it can reduce a training time when comparing it to small batch size with same epochs. To make it work well, it uses some techniques. First, it tries to find just a big enough batch size instead of maximum runnable batch size. Finding a maximum value need quite much time although the advantage isn't huge compared to finding just a big enough batch size. Second, it estimates an optimal batch size using estimated equation. it can reduce much time compared to other methods. Finally, learning rate is increased sqrt(k) times,
when increasing batch size k times. It's based on experiments and also, it's supported theoretically.

How to test

You can use the feature by adding --learning_parameters.auto_adapt_batch_size Full.
You can also test the feature by running pytest -k auto_adapt_batch_size tests/integration/cli/

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have added e2e tests for validation.
• I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• I have linked related issues.

License

• I submit my code changes under the same Apache License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2023 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[harimkang]

I left one comment.

[sungmanc]

Why do we need to separate auto_decrease_batch_size and auto_adapt_batch_size ?
How about unifying the parameters? auto_adapt_batch_size could include the concept of auto_decrease_batch_size.

[eunwoosh]

Why do we need to separate auto_decrease_batch_size and auto_adapt_batch_size ? How about unifying the parameters? auto_adapt_batch_size could include the concept of auto_decrease_batch_size.

IMHO, because they have different purpose. Purpose of auto_decrease_batch_size is decreasing batch size only when current batch size isn't fit to GPU memory. In contrast, Purpose of auto_adapt_batch_size is making training faster by increasing batch size. And I also concern that if unifying them, sem-seg and ins-seg will use auto_adapt_batch_size as default. Currently, auto_decrease_batch_size is turned on as default in those tasks because they are easy to make GPU OOM.

[harimkang]

How about getting a key named auto_adapt_batch_size as a str like.. auto_adapt_batch_size in ("decrease", "auto") (just example)
I don't see the need to have two separate parameters. Wouldn't it be nice to have multiple behaviors in one?

[eunwoosh]

How about getting a key named auto_adapt_batch_size as a str like.. auto_adapt_batch_size in ("decrease", "auto") (just example) I don't see the need to have two separate parameters. Wouldn't it be nice to have multiple behaviors in one?

I think it's a good idea. I'll apply it after asking others' opinions.

[eunwoosh]

I combined separated arguments to a single one and applied other comments. Could you review my PR? @harimkang @supersoob @sungmanc

[sungmanc]

LGTM, with minor comments

[sungmanc]

Thanks very much for applying my minor suggestion :), LGTM.