Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use consistent spelling for optimise #2203

Merged
merged 3 commits into from
Mar 15, 2023
Merged

Conversation

jeremiahpslewis
Copy link
Contributor

Flux.jl uses alternately optimize and optimise, in various parts of the project, which can be a bit tricky, particularly as the JuMP.jl ecosystem seems to have chosen optimize. Not sure whether it would make sense to support both spellings, but in the meantime it seems best to be internally consistent.

PR Checklist

  • Tests are added
  • Entry in NEWS.md
  • Documentation, if applicable

@codecov-commenter
Copy link

codecov-commenter commented Mar 5, 2023

Codecov Report

Patch coverage has no change and project coverage change: -2.70 ⚠️

Comparison is base (484796c) 82.63% compared to head (ac88afb) 79.93%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2203      +/-   ##
==========================================
- Coverage   82.63%   79.93%   -2.70%     
==========================================
  Files          23       23              
  Lines        1578     1575       -3     
==========================================
- Hits         1304     1259      -45     
- Misses        274      316      +42     
Impacted Files Coverage Δ
src/optimise/optimisers.jl 93.78% <ø> (ø)
src/optimise/train.jl 94.87% <ø> (ø)
src/cuda/cudnn.jl 0.00% <0.00%> (-100.00%) ⬇️
src/functor.jl 38.27% <0.00%> (-36.43%) ⬇️
src/layers/normalise.jl 93.79% <0.00%> (-1.38%) ⬇️
src/layers/conv.jl 88.95% <0.00%> (-0.07%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@ToucheSir
Copy link
Member

The problem is that the majority of current maintainers use Z, but the original code (including the unfortunately named Optimisers.jl) was written by someone who uses S. I guess this is as good a chance as any to debate which one we should use going forward.

@bhvieira
Copy link
Contributor

I'm in favor of "optimiz*" everywhere.
"Z" is much more common worldwide, while "s" is an alternative spelling that also exists everywhere.
This change would require a deprecation warning until the next major release only. And perhaps, in the future a warning can be thrown if people try the old names. Better than keeping aliases imo.

Copy link
Member

@mcabbott mcabbott left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems OK, it's a minor spelling fix to docs. There is no way that the docs are ever likely to be fully consistent about using UK spelling, although I do think that's the convention.

Trying to avoid code which requires one spelling seems like a good prinicple. E.g. we should kill Flux.normalise. (Edit: JuliaML/MLUtils.jl#123 is the issue.)

Am I correct in thinking that the only place where Flux's code cares about the spelling of "optimiser" is the package name Optimisers.jl ? Renaming that would be a much bigger step. Perhaps one we should have contemplated before registering it -- I mean to some other name completely, both to avoid words whose spelling varies, and to avoid the (temporary) clash with the Flux.Optimise sub-module.

NEWS.md Outdated Show resolved Hide resolved
Comment on lines 145 to +146
gradients = gradient(() -> loss(x, y), parameters)
Flux.Optimise.update!(optimizer, parameters, gradients)
Flux.Optimise.update!(optimiser, parameters, gradients)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ought to be re-written in gradients parameters anyway, in which case the variable will be opt_state


You can store the optimiser state alongside the model, to resume training
exactly where you left off. BSON is smart enough to [cache values](https://github.com/JuliaIO/BSON.jl/blob/v0.3.4/src/write.jl#L71) and insert links when saving, but only if it knows everything to be saved up front. Thus models and optimizers must be saved together to have the latter work after restoring.
exactly where you left off. BSON is smart enough to [cache values](https://github.com/JuliaIO/BSON.jl/blob/v0.3.4/src/write.jl#L71) and insert links when saving, but only if it knows everything to be saved up front. Thus models and optimisers must be saved together to have the latter work after restoring.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be re-written for explicit gradients

@@ -145,7 +145,7 @@ function train(; kws...)
return logitcrossentropy(ŷ, y)
end

# Train our model with the given training set using the Adam optimizer and
# Train our model with the given training set using the Adam optimiser and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file to be deleted in favour of the model zoo, perhaps FluxML/model-zoo#394

@@ -45,7 +45,7 @@ end
"""
Momentum(η = 0.01, ρ = 0.9)

Gradient descent optimizer with learning rate `η` and momentum `ρ`.
Gradient descent optimiser with learning rate `η` and momentum `ρ`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file will be deleted in 0.14

@mcabbott mcabbott merged commit eb15eae into FluxML:master Mar 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants