Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixes to save on fractional save_steps #1643

Merged
merged 1 commit into from
May 20, 2024
Merged

fixes to save on fractional save_steps #1643

merged 1 commit into from
May 20, 2024

Conversation

winglian
Copy link
Collaborator

@winglian winglian commented May 20, 2024

fixes #1642

per hf docs:

save_steps (int or float, optional, defaults to 500) — Number of updates steps before two checkpoint 
saves if save_strategy="steps". Should be an integer or a float in range [0,1). If smaller than 1, will be 
interpreted as ratio of total training steps.

but the hf trainer seems to have a regression where it doesn't properly save on fractional save_steps. This is a temporary workaround for now until this is fixed upstream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

A saves_per_epoch that creates a fraction may create training that don´t have checkpoints
1 participant