Cannot install irmin-unix.1.3.3 in a ocaml-base-compiler.4.05.0 switch

[erikmd]

# opam config report
# opam-version      2.0.6 
# self-upgrade      no
# system            arch=x86_64 os=linux os-distribution=debian os-version=10
# solver            builtin-mccs+glpk
# install-criteria  -removed,-count[version-lag,request],-count[version-lag,changed],-changed
# upgrade-criteria  -removed,-count[version-lag,solution],-new
# jobs              7
# repositories      2 (http) (default repo at 1d8fdb21)
# pinned            0
# current-switch    4.05.0+pfitaxel

With @Aleridia, we try to install irmin in a OCaml 4.05.0 switch, but we get the following issue:

$ opam switch create 4.05.0+pfitaxel ocaml-base-compiler.4.05.0

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
[ocaml-base-compiler.4.05.0] found in cache

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
∗ installed base-bigarray.base
∗ installed base-threads.base
∗ installed base-unix.base
∗ installed ocaml-base-compiler.4.05.0
∗ installed ocaml-config.1
∗ installed ocaml.4.05.0
Done.
# Run eval $(opam env) to update the current shell environment

$ eval $(opam env)

$ opam update

<><> Updating package repositories ><><><><><><><><><><><><><><><><><><><><><><>
[coq-released] synchronised from https://coq.inria.fr/opam/released
[default] synchronised from https://opam.ocaml.org
Now run 'opam upgrade' to apply any package updates.

$ opam install irmin-unix.1.3.3
[ERROR] Sorry, resolution of the request timed out.
        Try to specify a simpler request, use a different solver, or increase the allowed time by setting OPAMSOLVERTIMEOUT to a bigger value (currently, it is set to 60.0 seconds).

Any ideas? some other infos I should provide to make this report more complete?

[rjbou]

The solver take too much time find a solution.
You can try to set up the environment variable OPAMSOLVERTIMEOUT. On my setup, I put an 3600 value, and finally it ended in around 10 min

00:02.368 SOLVER  Calling solver builtin-mccs+glpk with criteria -removed,-count[version-lag,request],-count[version-lag,changed],-changed
09:18.113 CUDF    Solver call done in 555.745

To reduce this time, you can also fix some dependencies version.

[erikmd]

Thanks @rjbou for your quick reply and suggestion!

actually I am a bit puzzled to notice this behavior, given the installation attempt was done in a fresh switch… (it didn't work either in a non-empty switch anyway)

Do you think we can find a better solution than this workaround?
(the plan was to propose adding irmin-unix.1.3.3 to an upcoming PR in learn-ocaml, but having a deps resolution time of 10mn by default is quite cumbersome)
in particular: do you think there is an underlying issue in the opam deps resolution that we could highlight, e.g., by outputting some log during this phase? or should some of the dependencies of irmin-unix be slightly modified in the opam repo to avoid this?

$ opam show irmin-unix.1.3.3
…
depends:      "ocaml" {>= "4.01.0"}
              "jbuilder" {>= "1.0+beta10"}
              "irmin" {>= "1.3.0" & < "2.0.0"}
              "irmin-mem" {>= "1.3.0" & < "2.0.0"}
              "irmin-git" {>= "1.3.0" & < "2.0.0"}
              "irmin-http" {>= "1.3.0" & < "2.0.0"}
              "irmin-fs" {>= "1.3.0" & < "2.0.0"}
              "git-unix" {>= "1.11.4" & < "2.0.0"}
              "irmin-watcher" {>= "0.2.0"}
              "alcotest" {with-test}
              "mtime" {with-test & >= "1.0.0"}
…

(Cc @yurug @AltGr just FYI)

[avsm]

We have some upcoming solver improvements in a future version of opam that will resolve these, but the current workaround in a released opam is indeed to tighten the specifications. You could have a go at tightening the bounds on some of the dependencies and submitting that to opam-repository, where we could merge it as a workaround. (for historical versions of packages, tightening bounds is no big deal).

As a higher level question, is there any reason you can't use irmin.2.1 for learn-ocaml? There are significant improvements from the last few years that you'll miss with irmin.1.x. cc @samoht @craigfe

[erikmd]

Thanks @avsm for your feedback!

As a higher level question, is there any reason you can't use irmin.2.1 for learn-ocaml?

the main reason was just that learn-ocaml is tied to ocaml 4.05.0 currently (notably as its ppx codebase relies on the 4.05 AST) so a possible strategy would have been to add irmin.1.4 for now to address ocaml-sf/learn-ocaml#348, then later migrate to irmin 2.1 (as soon as the ocaml-version is bumped)

Otherwise, another solution (that would avoid this irmin migration) would be to use irmin 2.1 from now on, making learn-ocaml master require, say, ocaml 4.07.1

@yurug do you have an opinion / objection on this idea?

[AltGr]

Thanks for providing a failure case, and sorry for the trouble. I'll record it for our solver benches.
The message should actually be changed:
Try to specify a simpler request is meant in the SMT sense. Actually vague requests are the most difficult, and the most precise the request, the easiest it is for the solver. So "try adding more constraints" would be a better advice.

[AltGr]

Cudf file for the solver request: https://gist.github.com/AltGr/7d7bc4544dc53a1e28f52b25ac80d282

[AltGr]

Using the z3 solver variant, the resolution succeeds in 15s. You need opam compiled with z3 installed (opam install z3), and run with --solver=builtin-z3.

[erikmd]

Hi @AltGr

"try adding more constraints" would be a better advice.

OK! so I've just opened PR ocaml/opam-repository#16508 following @avsm's suggestion.

FYI after the proposed pinnings, the constraints resolution with the default solver only takes 17.9s:

$ time bash -c "yes n | opam install irmin-unix.1.3.3"
The following actions will be performed:
  ∗ install conf-gmp                1           [required by zarith]
  ∗ install mirage-no-xen           1           [required by nocrypto]
  ∗ install conf-m4                 1           [required by ocamlfind]
  ∗ install ocamlbuild              0.14.0      [required by mtime, nocrypto]
  ∗ install cmdliner                1.0.4       [required by git-unix]
  ∗ install base-num                base        [required by num]
  ∗ install mirage-no-solo5         1           [required by nocrypto]
  ∗ install conf-perl               1           [required by zarith]
  ∗ install dune                    1.11.4      [required by cohttp-lwt-unix,
                                                crunch]
  ∗ install ocamlfind               1.8.1       [required by mtime, nocrypto]
  ∗ install uchar                   0.0.2       [required by jsonm]
  ∗ install num                     0           [required by sexplib]
  ∗ install stdlib-shims            0.1.0       [required by cohttp]
  ∗ install sexplib0                v0.13.0     [required by cohttp-lwt]
  ∗ install seq                     0.2.2       [required by fmt, re, lwt]
  ∗ install result                  1.4         [required by irmin]
  ∗ install ppx_derivers            1.2.1       [required by ppx_deriving]
  ∗ install ocaml-compiler-libs     v0.12.1     [required by ppxlib]
  ∗ install mmap                    1.1.0       [required by lwt]
  ∗ install magic-mime              1.1.2       [required by cohttp-lwt-unix]
  ∗ install macaddr                 4.0.0       [required by ipaddr]
  ∗ install jbuilder                transition* [required by irmin-unix]
  ∗ install dune-configurator       1.0.0       [required by lwt]
  ∗ install cpuid                   0.1.2       [required by nocrypto]
  ∗ install cppo                    1.6.6       [required by ppx_deriving]
  ∗ install bigarray-compat         1.0.0       [required by hex]
  ∗ install zarith                  1.9.1       [required by nocrypto]
  ∗ install topkg                   1.0.1       [required by mtime, nocrypto]
  ∗ install ppx_tools               5.0+4.05.0  [required by ppx_deriving]
  ∗ install ocamlgraph              1.8.8       [required by irmin]
  ∗ install calendar                2.04        [required by webmachine]
  ∗ install base-bytes              base        [required by decompress]
  ∗ install re                      1.9.0       [required by webmachine]
  ∗ install dispatch                0.4.1       [required by webmachine]
  ∗ install ocaml-migrate-parsetree 1.7.3       [required by ppx_deriving]
  ∗ install base                    v0.13.2     [required by ppx_fields_conv,
                                                fieldslib, ppx_sexp_conv]
  ∗ install cppo_ocamlbuild         1.6.6       [required by ppx_deriving]
  ∗ install cstruct                 5.1.1       [required by irmin]
  ∗ install uutf                    1.0.2       [required by jsonm]
  ∗ install ptime                   0.8.5       [required by crunch]
  ∗ install mtime                   1.2.0       [required by git-unix]
  ∗ install fmt                     0.8.8       [required by irmin,
                                                irmin-watcher]
  ∗ install stringext               1.6.0       [required by cohttp]
  ∗ install ocplib-endian           1.1         [required by git]
  ∗ install decompress              0.7         [required by git]
  ∗ install base64                  3.2.0       [required by cohttp]
  ∗ install astring                 0.8.3       [required by irmin,
                                                irmin-watcher]
  ∗ install stdio                   v0.13.0     [required by ppxlib]
  ∗ install parsexp                 v0.13.0     [required by sexplib]
  ∗ install fieldslib               v0.13.0     [required by cohttp]
  ∗ install ppx_deriving            4.2.1       [required by nocrypto]
  ∗ install mstruct                 1.4.0       [required by git]
  ∗ install hex                     1.4.0       [required by irmin]
  ∗ install jsonm                   1.0.1       [required by irmin]
  ∗ install crunch                  3.2.0       [required by irmin-http]
  ∗ install uri                     3.1.0       [required by irmin]
  ∗ install lwt                     5.2.0       [required by irmin,
                                                irmin-watcher]
  ∗ install ocb-stubblr             0.1.1-1     [required by nocrypto]
  ∗ install domain-name             0.3.0       [required by ipaddr]
  ∗ install ppxlib                  0.13.0      [required by ppx_fields_conv,
                                                ppx_sexp_conv]
  ∗ install sexplib                 v0.13.0     [required by git-http,
                                                nocrypto]
  ∗ install logs                    0.7.0       [required by irmin, git-unix,
                                                irmin-watcher]
  ∗ install cstruct-lwt             5.1.1       [required by nocrypto]
  ∗ install ipaddr                  4.0.0       [required by conduit-lwt-unix]
  ∗ install ppx_sexp_conv           v0.13.0     [required by nocrypto]
  ∗ install ppx_fields_conv         v0.13.0     [required by cohttp]
  ∗ install irmin-watcher           0.3.0*      [required by irmin-unix]
  ∗ install irmin                   1.4.0*      [required by irmin-unix]
  ∗ install git                     1.11.5      [required by irmin-git]
  ∗ install uri-sexp                3.1.0       [required by cohttp]
  ∗ install nocrypto                0.5.4-2     [required by git-unix]
  ∗ install ipaddr-sexp             4.0.0       [required by conduit-lwt-unix]
  ∗ install irmin-mem               1.3.0*      [required by irmin-unix]
  ∗ install irmin-fs                1.3.0*      [required by irmin-unix]
  ∗ install irmin-git               1.3.0*      [required by irmin-unix]
  ∗ install cohttp                  2.5.1       [required by webmachine]
  ∗ install conduit                 1.5.0       [required by conduit-lwt]
  ∗ install webmachine              0.5.0       [required by irmin-http]
  ∗ install cohttp-lwt              2.1.3       [required by irmin-http]
  ∗ install conduit-lwt             1.5.0       [required by conduit-lwt-unix]
  ∗ install irmin-http              1.3.3*      [required by irmin-unix]
  ∗ install git-http                1.11.4      [required by git-unix]
  ∗ install conduit-lwt-unix        1.5.0       [required by git-unix]
  ∗ install cohttp-lwt-unix         2.1.3       [required by git-unix]
  ∗ install git-unix                1.11.5*     [required by irmin-unix]
  ∗ install irmin-unix              1.3.3
===== ∗ 86 =====
Do you want to continue? [Y/n] n

real	0m17,920s
user	0m17,751s
sys	0m0,085s

[AltGr]

Thanks for the feedback!
Hint: use --show instead of yes n | :)

[dra27]

This is hopefully improved both with opam 2.1.0 and with the recompiled solver in opam 2.0.9. It was also mitigated in opam-repository. Please re-open if not!