[Feature] Support for linearly learning rate decay #1627

Sharpiless · 2021-12-29T04:05:18Z

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

I tried to re-implement pix2seq bug did not find linearly learning rate decay in mmdetection

Modification

add "LinearlyDecayLrUpdaterHook"

test config_file:

lr_config = dict(
    policy='LinearAnnealing',
    min_lr=0.001
)
momentum_config = dict(
    policy='LinearAnnealing',
    min_momentum=0.001,
    by_epoch=False
    # anneal_strategy='linear',
)

lr curve:

BC-breaking (Optional)

Does the modification introduce changes that break the backward-compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

Checklist

Before PR:

I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects, like MMDet or MMCls.
CLA has been signed and all committers have signed the CLA in this PR.

CLAassistant · 2021-12-29T04:05:23Z

All committers have signed the CLA.

teamwong111

Docstring and unit tests are needed.

zhouzaida · 2022-01-05T16:33:43Z

Hi @Sharpiless , is there any progress?

mmcv/runner/hooks/lr_updater.py

mmcv/runner/hooks/momentum_updater.py

…erHook, add unit test add docstring add docstring update linear lr momentum schedule test fix ci Fix CI

add lr and momentum hook to runner and hooks package

zhouzaida · 2022-03-21T15:37:40Z

tests/test_runner/test_hooks.py

@@ -711,6 +711,83 @@ def test_cosine_runner_hook(multi_optimizers):
    hook.writer.add_scalars.assert_has_calls(calls, any_order=True)


+@pytest.mark.parametrize('multi_optimziers', (True, False))


Suggested change

@pytest.mark.parametrize('multi_optimziers', (True, False))

@pytest.mark.parametrize('multi_optimizers', (True, False))

zhouzaida · 2022-03-21T15:37:48Z

tests/test_runner/test_hooks.py

@@ -711,6 +711,83 @@ def test_cosine_runner_hook(multi_optimizers):
    hook.writer.add_scalars.assert_has_calls(calls, any_order=True)


+@pytest.mark.parametrize('multi_optimziers', (True, False))
+def test_linear_runner_hook(multi_optimziers):


Suggested change

def test_linear_runner_hook(multi_optimziers):

def test_linear_runner_hook(multi_optimizers):

zhouzaida · 2022-03-21T15:37:59Z

tests/test_runner/test_hooks.py

+
+    # TODO: use a more elegant way to check values
+    assert hasattr(hook, 'writer')
+    if multi_optimziers:


Suggested change

if multi_optimziers:

if multi_optimizers:

OpenMMLab-Assistant003 · 2023-04-04T13:11:24Z

Hi @Sharpiless！First of all, we want to express our gratitude for your significant PR in the MMCV project. Your contribution is highly appreciated, and we are grateful for your efforts in helping improve this open-source project during your personal time. We believe that many developers will benefit from your PR.

We would also like to invite you to join our Special Interest Group (SIG) private channel on Discord, where you can share your experiences, ideas, and build connections with like-minded peers. To join the SIG channel, simply message moderator— OpenMMLab on Discord or briefly share your open-source contributions in the #introductions channel and we will assist you. Look forward to seeing you there! Join us ：https://discord.gg/UjgXkPWNqA

If you have WeChat account，welcome to join our community on WeChat. You can add our assistant ：openmmlabwx. Please add "mmsig + Github ID" as a remark when adding friends：）
Thank you again for your contribution❤ @Sharpiless

zhouzaida requested a review from teamwong111 December 29, 2021 07:27

teamwong111 reviewed Dec 29, 2021

View reviewed changes

zhouzaida added the Feature label Jan 5, 2022

zhouzaida reviewed Jan 21, 2022

View reviewed changes

mmcv/runner/hooks/lr_updater.py Show resolved Hide resolved

zhouzaida reviewed Jan 21, 2022

View reviewed changes

mmcv/runner/hooks/momentum_updater.py Show resolved Hide resolved

zhouzaida changed the title ~~update support for linearly learning rate decay~~ [Feature] Support for linearly learning rate decay Jan 21, 2022

HAOCHENYE requested review from teamwong111 and zhouzaida January 27, 2022 06:43

HAOCHENYE force-pushed the linearly-decay branch from 8f5eb7e to 6ed7bd3 Compare February 8, 2022 09:59

Sharpiless and others added 2 commits March 7, 2022 18:32

update support for linearly learning rate decay

073cd00

Fix LinearAnnealingLrUpdaterHook, update LinearAnnealingMomentumUpdat…

3c8376a

…erHook, add unit test add docstring add docstring update linear lr momentum schedule test fix ci Fix CI

HAOCHENYE force-pushed the linearly-decay branch from 6ed7bd3 to 3c8376a Compare March 7, 2022 11:00

teamwong111 approved these changes Mar 11, 2022

View reviewed changes

HAOCHENYE added 2 commits March 21, 2022 12:01

Merge branch 'master' into linearly-decay

d11b30e

add lr and momentum hook to runner and hooks package

9802778

add lr and momentum hook to runner and hooks package

zhouzaida reviewed Mar 21, 2022

View reviewed changes

replace multi_optimziers with multi_optimizers

5bb9fa1

HAOCHENYE requested a review from zhouzaida March 25, 2022 17:38

zhouzaida approved these changes Mar 31, 2022

View reviewed changes

zhouzaida merged commit 969e2af into open-mmlab:master Mar 31, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Feature] Support for linearly learning rate decay #1627

[Feature] Support for linearly learning rate decay #1627

Sharpiless commented Dec 29, 2021 •

edited by HAOCHENYE

Loading

CLAassistant commented Dec 29, 2021 •

edited

Loading

teamwong111 left a comment

zhouzaida commented Jan 5, 2022

zhouzaida Mar 21, 2022

zhouzaida Mar 21, 2022

zhouzaida Mar 21, 2022

OpenMMLab-Assistant003 commented Apr 4, 2023

		@@ -711,6 +711,83 @@ def test_cosine_runner_hook(multi_optimizers):
		hook.writer.add_scalars.assert_has_calls(calls, any_order=True)


		@pytest.mark.parametrize('multi_optimziers', (True, False))

	@pytest.mark.parametrize('multi_optimziers', (True, False))
	@pytest.mark.parametrize('multi_optimizers', (True, False))

	def test_linear_runner_hook(multi_optimziers):
	def test_linear_runner_hook(multi_optimizers):

[Feature] Support for linearly learning rate decay #1627

[Feature] Support for linearly learning rate decay #1627

Conversation

Sharpiless commented Dec 29, 2021 • edited by HAOCHENYE Loading

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

CLAassistant commented Dec 29, 2021 • edited Loading

teamwong111 left a comment

Choose a reason for hiding this comment

zhouzaida commented Jan 5, 2022

zhouzaida Mar 21, 2022

Choose a reason for hiding this comment

zhouzaida Mar 21, 2022

Choose a reason for hiding this comment

zhouzaida Mar 21, 2022

Choose a reason for hiding this comment

OpenMMLab-Assistant003 commented Apr 4, 2023

Sharpiless commented Dec 29, 2021 •

edited by HAOCHENYE

Loading

CLAassistant commented Dec 29, 2021 •

edited

Loading