Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Momentum betas logs #956

Open
wants to merge 6 commits into
base: dev
Choose a base branch
from
Open

Momentum betas logs #956

wants to merge 6 commits into from
+106 −59

Conversation

rockerBOO
Copy link
Contributor

@rockerBOO rockerBOO commented Nov 18, 2023 •
edited
Loading

For CyclicLR and OneCycleLR it changes the momentum. This adds logs to support charting these changes.

OneCycleLR

lr_scheduler_args=["max_lr=0.00025", "steps_per_epoch=3", "epochs=10", "base_momentum=0.85", "max_momentum=0.9"]
lr_scheduler_type="OneCycleLR"

CyclicLR (needs the pytorch pytorch/pytorch#113548)
Showing block LR version

lr_scheduler_args=["base_lr=[5e-5,1e-4,1e-4,1e-4,1e-4,1e-4,1e-4,1e-4,1e-4,1e-4,1e-4,1e-4,1e-4,1e-4,1e-4,1e-4]","max_lr=[5e-4,1e-3,1e-3,1e-3,1e-3,1e-3,1e-3,1e-3,1e-3,1e-3,1e-3,1e-3,1e-3,1e-3,1e-3,1e-3]","step_size_up=20", "base_momentum=0.85", "max_momentum=0.99"]
lr_scheduler_type="CyclicLR"

Note I also moved ahead with making a common generate_step_logs function.

Screenshot 2023-11-18 at 14-20-33 Weights Biases

@rockerBOO rockerBOO changed the base branch from main to dev November 20, 2023 02:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@rockerBOO