Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cli versioning #4385

Merged
merged 17 commits into from
Dec 11, 2020
Merged

Cli versioning #4385

merged 17 commits into from
Dec 11, 2020

Conversation

rjbou
Copy link
Collaborator

@rjbou rjbou commented Oct 9, 2020

Implementing CLI versioning, handling of the different CLIs from spec.
It is only handled for subcommands and flags, not principal commands. A note is added in the manpage depending of flag validity and requested cli. When a not valid flag is used, a message is also printed to the user with cli indication, etc.

See discussion below for more precisions

Summary of new [N] and deprecated [D] options in 2.1 (diff of manpages from 2.0.7 & master).

opam commands

  • lock [N]
  • option [N]

opam globals options

  • --cli [N]

package build options

  • --assume-depexts [N]
  • --locked (behaviour change)
  • --lock-suffix [N]
  • --no-depexts [N]
  • --unlock-base [D]
  • --update-invariant [N]

opam admin add-hashes

  • --packages [N]

opam config

  • set [D]
  • unset [D]
  • set-global [D]
  • unset-global [D]
  • var [D]
  • exec [D]

opam env

  • --check [N]

opam info/show

  • --all-versions [N]
  • --just-file [N]
  • --sort [N]
  • --file [D]

opam install

  • --check [N]
  • --depext-only [N]
  • --ignore-conflicts [N]

opam pin

  • scan [N]
  • --normalise [N]
  • --with-version [N]

opam switch

  • invariant [N]
  • set-invariant [N]
  • set-base [D]
  • install [D]
  • --force [N]
  • --formula [N]
  • --freeze [N]
  • --no-action [N]

opam unpin

  • --normalise [N]
  • --with-version [N]

opam upgrade

  • --installed [N]

opam var

  • --global [N]

@rjbou rjbou added AREA: UI PR: WIP Not for merge at this stage labels Oct 9, 2020
@rjbou rjbou added this to the 2.1.0~beta3 milestone Oct 9, 2020
@rjbou rjbou requested review from AltGr and dra27 October 9, 2020 07:20
@rjbou rjbou changed the title Cli Cli versioning Oct 9, 2020
@AltGr
Copy link
Member

AltGr commented Oct 9, 2020

Wow, pretty heavy infrastructure work... Seems pretty clean given the scope :)
Two minor comments:

  • it would be nice to add (ref)tests for the different kinds of arguments (flags, vflags, positional parameters, commands, etc.) and the different cases (below version where it was introduced, deprecated, just fine, etc.) Having the corresponding output in the git log also serves as a good advertisement for the PR ;)
  • you may have overlooked the case of a flag that was introduced strictly after 2.0, but is already obsolete or removed; this might happen in the future. Or maybe I just missed something ?

@rjbou
Copy link
Collaborator Author

rjbou commented Oct 9, 2020

flag that was introduced strictly after 2.0, but is already obsolete or removed

If a flag is introduced in 3.0, deprecated in 3.1 and removed in 3.2 :

  • using opam 3.0 -> no error
  • using opam > 3.1 -> clear deprecation/error message
  • using opam > 3.2 with a cli < 3.2 -> clear error message as the flag is no more implemented

Some notes on the implementation:

  • opam option are not concerned by this PR (lock, option)
  • In the message, there is no differentiation between a subcommand, an option, etc.
  • Format changes are not handled neither

+1 for reftests!!!

Comment on lines 784 to 788
let cli2_0 = OpamCLIVersion.of_string "2.0"
let cli2_1 = OpamCLIVersion.of_string "2.1"
let cli2_3 = OpamCLIVersion.of_string "2.3"
let cli3_0 = OpamCLIVersion.of_string "3.0"
let valid_cli_legacy = Valid_since cli2_0
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are dropped here, but maybe there is a better place to put them. I thought about directly in OpamCLIVersion, but it would be quite more verbose to write/read.

src/client/opamArg.ml Outdated Show resolved Hide resolved
src/client/opamArg.mli Outdated Show resolved Hide resolved
@rjbou rjbou removed the PR: WIP Not for merge at this stage label Oct 9, 2020
@dra27
Copy link
Member

dra27 commented Oct 21, 2020

Finally looking properly at this - many thanks for taking the heavy lifting of this task list! A few things are not working quite as I envisaged in the spec, but I expect this to be tweaking, rather than a wholesale problem!

The idea with the CLI versioning is that we only deprecate features, rather than commands. So, taking opam install --unlock-base / --update-invariant as an example. Suppose I have a switch with ["ocaml-base-compiler" {= "4.10.0"}] for an invariant. The following should happen in each case:

  • opam install ocaml.4.10.1: advertise --update-invariant (this is already correct)
  • X OPAMCLI=2.0 opam install ocaml.4.10.1: advertise --unlock-base (i.e. the 2.0 behaviour) (this is a tweak to the previous message based on the CLI version)
  • opam install ocaml.4.10.1 --unlock-base: should display [ERROR] --unlock-base was removed in opam 2.1. Use --update-invariant instead, or set OPAMCLI=2.0. (i.e. the warning presently displayed should be a hard error returning an exit code for a bad command line). Note that for 2.0 removed flag, we advertise setting OPAMCLI=2.0, for a 2.1 removed feature in future we'd say use --cli=2.1. The point is we didn't "deprecate" --unlock-base we have actually removed and the user must change - but, as with Dune variables, etc., we retain a nice message rather than brutally saying "Unrecognised command line option".
  • OPAMCLI=2.0 opam install ocaml.4.10.1 --unlock-base should proceed without error - at the moment it displays a warning about --unlock-base which shouldn't be there. i.e. it's OK to use 2.0 features without warning as long as you specified the version you were after.
  • OPAMCLI=2.0 opam install ocaml.4.10.1 --update-invariant: [ERROR] --update-invariant was added in opam 2.1 and is not available for CLI version 2.0. At present even --update-invariant is always display a warning [WARNING] unlock-base is deprecated in 2.1 (since 2.1), it will be removed in 2.3. Use --update-invariant instead. The main point here is that new 2.1 command line features should not be available if you selected the 2.0 CLI explicitly.
  • OPAMCLI=2.0 opam install --help - the help message for --unlock-base doesn't need to mention opam 2.3 - the flag is gone in 2.1.
  • opam install --help - the help for --unlock-base should say "Removed in 2.1; use --update-invariant instead". We can choose to remove the help message completely at any point but we don't need to say when. I would say probably that we get rid of flags like that at 3.0.

edit: add checkbox

@rjbou
Copy link
Collaborator Author

rjbou commented Nov 3, 2020

OPAMCLI=2.0 opam install --help - the help message for --unlock-base doesn't need to mention opam 2.3 - the flag is gone in 2.1.

No mention of removal if help called with 2.0 (can be changed).

OPAMCLI=2.0 opam install ocaml.4.10.1: advertise --unlock-base (i.e. the 2.0 behaviour) (this is a tweak to the previous message based on the CLI version)

As it is a message in the code, and not in option management, a specific behavior need to be called when the message is displayed. The problem here is that we don't have the information about the requested cli version in all modules, only on opam-client & those depending on ones.

@rjbou
Copy link
Collaborator Author

rjbou commented Nov 3, 2020

There is some options that was removed in 2.0 betas and reintroduced in 2.0 (cf. #3685, for non erroneous message). In cli versioning it means that it was introduced in 2.0 and removed in 2.0, that errors. It is possible to remove them completely in 2.1, but some behaviour can't be reintroduced as it was not even present in 2.0. It concerns no-aspcud, alias-of and no-autoinstall.

@rjbou rjbou linked an issue Nov 18, 2020 that may be closed by this pull request
Copy link
Member

@dra27 dra27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks basically wonderful - some notes and comments on the API, but there's nothing which looks major:

  • For --no-aspcud, --just-file (old behaviour), --no-autoinstall, --alias-of, the behaviour has changed from "ignore the flag" to "syntax error", so I'd keep OpamArg.deprecated_option for 2.0 so that it's ignored in OPAMCLI=2.0 (for compatibility) and is an error in opam 2.1+
  • I don't follow the need for the polymorphic variant on the content field in OpamArg.Mk, unless there's a weird interaction with Cmdliner that I'm not yet seeing?
  • OPAMCLI=2.0 opam option should display an error, OPAMCLI opam option --global does, I guess it's just some threading of cli version for the default behaviour of a new command?
  • Given that all the uses of, say, OpamArg.print_short_flag have had to be updated to include cli, I'd make validity non-optional just to be really sure that if it gets added elsewhere that adding the version isn't forgotten.
  • It's not this PR which has done it, but we've lost the description for the opam admin command somewhere along the line:
COMMANDS
       admin


       clean
           Cleans up opam caches

Makefile Outdated Show resolved Hide resolved
src/client/opamCLIVersion.ml Outdated Show resolved Hide resolved
src/client/opamCLIVersion.mli Outdated Show resolved Hide resolved
src/client/opamArg.mli Outdated Show resolved Hide resolved
src/client/opamArg.mli Outdated Show resolved Hide resolved
src/client/opamArg.ml Outdated Show resolved Hide resolved
src/client/opamArg.ml Outdated Show resolved Hide resolved
src/client/opamArg.ml Outdated Show resolved Hide resolved
src/client/opamCommands.ml Outdated Show resolved Hide resolved
src/client/opamCommands.ml Outdated Show resolved Hide resolved
@dra27
Copy link
Member

dra27 commented Nov 26, 2020

Are there still some un-pushed commits? You've ticked "--no-aspcud, --just-file (old behaviour), --no-autoinstall, --alias-of", for example, but those CLI options aren't yet restored? Similarly for ?validity changing to ~validity.

@rjbou
Copy link
Collaborator Author

rjbou commented Nov 26, 2020

All comments that are marked as resolved but not present in the PR. I'll post a response to the main post when I'll finish. The pushed commits are only from suggestions comments.

@rjbou
Copy link
Collaborator Author

rjbou commented Nov 26, 2020

Thanks for the detailed review!
So so so... On resolved comments : some are closed with no comment (so it was simply integrated), or with a comment (integrated and commented). Remaining ones are on discussion.
Some global answers

  • For --no-aspcud, --just-file (old behaviour), --no-autoinstall, --alias-of, the behaviour has changed from "ignore the flag" to "syntax error", so I'd keep OpamArg.deprecated_option for 2.0 so that it's ignored in OPAMCLI=2.0 (for compatibility) and is an error in opam 2.1+

I kept just messages printing to not reintroduce the function. For --file I re-implemented a consistent (imho) old behaviour, so it is handled by the new system.

I don't follow the need for the polymorphic variant on the content field in OpamArg.Mk, unless there's a weird interaction with Cmdliner that I'm not yet seeing?

In fact, there is no need of a polymorphic variant, a simple variant is sufficient. I'm not against having a variant instead of polymorphic variant, it's just a new type to declare.
On the need of specifying Valid or Remove, cf. #4385 (comment)

OPAMCLI=2.0 opam option should display an error, OPAMCLI opam option --global does, I guess it's just some threading of cli version for the default behaviour of a new command?

Not yet implemented, it needs another mechanism (this one handles only options & flags). But with all this all done, it should be not so hard to add.

Given that all the uses of, say, OpamArg.print_short_flag have had to be updated to include cli, I'd make validity non-optional just to be really sure that if it gets added elsewhere that adding the version isn't forgotten.

Agree on it, updated accordingly.

It's not this PR which has done it, but we've lost the description for the opam admin command somewhere along the line

It was removed with changes in 97c2252

@rjbou
Copy link
Collaborator Author

rjbou commented Nov 26, 2020

Commands added!

| None -> { validity with content = `Valid content}
| Some _ -> { validity with content = `Removed content}

let cli_from valid = { valid ; removed = None; content = () }
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can also be implemented for a more simple use as

val cli_from: string -> valitidy
let cli_from version = 
  try
    let valid = OpamCLIVersion.of_string version in
    if OpamCLIVersion.is_supported valid then
      { valid; removed = None; content = () }
    else
      OpamConsole.error_and_exit `Internal_error "not supported"
  with Failure _ -> OpamConsole.error_and_exit `Internal_error "not a valid cli"

but it will require to define newer version as supported (for example when we know that we want to deprecate an option in a future opam cli version), and we loose a strong check at compile time replaced by a runtime failure.

@dra27
Copy link
Member

dra27 commented Dec 1, 2020

It's not this PR which has done it, but we've lost the description for the opam admin command somewhere along the line

It was removed with changes in 97c2252

Oops - duly amending from "we've lost the description" to "I lost the description" 😊

Copy link
Member

@dra27 dra27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor nits (all as code suggestions). Why did the restored --no-aspcud (opam-install) and --alias-of/--no-autoinstall (opam-switch) end up referencing the 2.2 cli? I was expecting for those to be CLI errors in 2.1. It's just 2.0 that wants to display the message that they're being ignored (if, for example, there are still scripts using --no-ascpud - it's OK that they need to switch to using OPAMCLI=2.0, but that's the only change 2.0 scripts should have to have added).

The implementation for subcommands looks great (and is working as I anticipated too), thanks.

A few doc string updates, but otherwise I think this is ready to go in - thanks again for the considerable amount of work on it!

src/client/opamArg.ml Outdated Show resolved Hide resolved
src/client/opamArg.ml Outdated Show resolved Hide resolved
src/client/opamCommands.ml Outdated Show resolved Hide resolved
src/client/opamCommands.ml Outdated Show resolved Hide resolved
src/client/opamArg.mli Outdated Show resolved Hide resolved
src/client/opamCommands.ml Outdated Show resolved Hide resolved
src/client/opamCommands.ml Outdated Show resolved Hide resolved
src/client/opamArg.ml Outdated Show resolved Hide resolved
src/client/opamArg.ml Outdated Show resolved Hide resolved
src/client/opamArg.ml Outdated Show resolved Hide resolved
@dra27
Copy link
Member

dra27 commented Dec 8, 2020

The AppVeyor failure can be ignored - the Ubuntu test failure appears to be because of... CLI versioning 😊

Apart from that, I think is ready to go in?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[opam 2.1~alpha] Failure to create an empty local switch
3 participants