Using hidden-version for OCaml beta releases

[dra27]

This is a tracking issue for the current status of using the hidden-version flag to replace the ocaml-beta package.

Using ocaml/opam-repository@e4a28e4:

opam switch create current ocaml --repos=current=git+https://github.com/ocaml/opam-repository.git#e4a28e4544404cb8b7fb2dfec4458de645fc4361

will create a switch with ocaml-base-compiler.4.11.1 installed and "ocaml" as the switch invariant.

Unclear message in opam upgrade

opam upgrade behaves correctly, but the message about ocaml.4.13.0 is incorrect (ocaml.4.11.1 is installed):

dra@thor:~/opam$ ./opam upgrade
Everything as up-to-date as possible (run with --verbose to show unavailable upgrades).

The following packages are not being upgraded because the new versions conflict with other installed packages:
  - ocaml-base-compiler.4.12.0~alpha1
    ∗ ocaml.4.13.0 is installed and requires ocaml-base-compiler = 4.13.0
  - ocaml-config.2
However, you may "opam upgrade" these packages explicitly, which will ask permission to downgrade or uninstall the conflicting packages.
Nothing to do.

Unexpectedly offering alpha releases

However, both opam upgrade ocaml and opam upgrade ocaml-base-compiler offer 4.12.0~alpha1, which is incorrect:

dra@thor:~/opam$ ./opam upgrade ocaml
The following actions will be performed:
  ∗ install   ocaml-options-vanilla 1
  ↗ upgrade   ocaml-base-compiler   4.11.1 to 4.12.0~alpha1 [required by ocaml]
  ↻ recompile ocaml-config          1                       [uses ocaml-base-compiler]
  ↗ upgrade   ocaml                 4.11.1 to 4.12.0
===== ∗ 1   ↻ 1   ↗ 2 =====

[dra27]

The current implementation of hidden-version is supposed to ensure that a package marked hidden-version will only be returned if there was no other solution. That's failing in the opam upgrade ocaml stage, since it's not lagging a dependency correctly. This could be worked around by marking ocaml.4.12.0 as hidden-version until release (that's not great, though, although it does have the side-effect of pinging all switches with an alpha/beta/rc compiler since the package will have changed!)

[dra27]

The implementation is therefore working correctly for opam switch create. The issue with opam upgrade ocaml is that this translates to opam install 'ocaml>4.11.1' and so the only solution is with a hidden-version package.

[kit-ty-kate]

In my opinion hidden-version should not have been the name used. avoid-version or something similar holds up better to its behaviour.

@AltGr Would it be possible to ?:

1. Rename it to avoid-version
2. Change the behaviour of hidden-version to "remove from the CUDF universe unless it is explicitly requested" by the user?

[dra27]

@kit-ty-kate - removing from the CUDF universe makes upgrading the explicitly installed packages tricky. For example, if you have expressly installed ocaml-base-compiler.4.12.0~alpha1, you do not expressly want to have to install ocaml-base-compiler.4.12.0~alpha2 so I think it wants to be slightly more nuanced (on a comment I'm just writing...!)

[dra27]

(I'm totally neutral on the subject of the name of the flag, given the myriad rubbish ones I went through before this was added!)

[dra27]

One possibility. The current criteria work completely correctly for switch creation - you will only get a hidden-version package if you've specified criteria which can only be satisfied that way (so ocaml, ocaml>=4.11, etc. will always give release versions).

Before subsequent install/upgrade operations, I think it's then that the universe can be pruned. So for all installed packages, if the installed version doesn't have the hidden-version flag then you prune any versions of that package which do have it from the universe.

[kit-ty-kate]

@kit-ty-kate - removing from the CUDF universe makes upgrading the explicitly installed packages tricky. For example, if you have expressly installed ocaml-base-compiler.4.12.0~alpha1, you do not expressly want to have to install ocaml-base-compiler.4.12.0~alpha2 so I think it wants to be slightly more nuanced (on a comment I'm just writing...!)

mh, true.

Before subsequent install/upgrade operations, I think it's then that the universe can be pruned. So for all installed packages, if the installed version doesn't have the hidden-version flag then you prune any versions of that package which do have it from the universe.

yes, that would be perfect I think!

[dra27]

The downside to the approach is that you can essential "opt in" to beta releases when you create the switch, but then can't change that.

[AltGr]

There is a small issue with opam upgrade foo, which is interpreted as install foo>currently-installed-version when such a version is found (to make sure that upgrading this specific package is prioritised over other upgrades) ; I think the fix for that is as simple as

modified   src/client/opamClient.ml
@@ -194,7 +194,11 @@ let compute_upgrade_t
             then (n, None)
             else
             let atom = (n, Some (`Gt, nv.version)) in
-             if OpamPackage.Set.exists (OpamFormula.check atom)
+             if OpamPackage.Set.exists
+                 (fun nv ->
+                    OpamFormula.check atom nv &&
+                    not (OpamFile.OPAM.has_flag Pkgflag_HiddenVersion
+                           (OpamSwitchState.opam t nv)))
                 (Lazy.force t.available_packages)
             then atom
             else (n, None)

This solves the issue in the case of opam upgrade ocaml-base-compiler, but not in the case of ocaml, since as @dra27 pointed out, the newer version of ocaml is not itself hidden.

Checking the whole dependency tree would be too far-fetched, but I would be in favour of adding the hidden-version flag to the ocaml package as well, and removing it once officially released. It's easy to adjust the comparison function so that changing this flag doesn't trigger a recompilation of existing switches (fields like synopsis are already excluded from this constraint)