[new release] dune-build-info, dune, dune-configurator, dune-action-plugin, dune-private-libs and dune-glob (2.0.0+draft2)

[rgrinberg]

Embed build informations inside executable

• Project page: https://github.com/ocaml/dune
• Documentation: https://dune.readthedocs.io/

CHANGES:

• The action field in the alias stanza is not available starting lang dune 2.0. The alias field in the rule stanza is a replacement. (Deprecate actions in the alias Stanza dune#2846, fixes
  2681, @rgrinberg)

• Introduce alias and package fields to the rule stanza. This is the
  preferred way of attaching rules to aliases. (Add alias field to rules dune#2744, @rgrinberg)

• Add field (optional) for executable stanzas (Add optional executables dune#2463, fixes executables: (optional) equivalent? dune#2433, @bobot)

• Infer targets for rule stanzas expressed in long form (Infer rule targets in long-form rule stanzas as well dune#2494, fixes Rule target inferrence doesn't work with write-file dune#2469,
  @NathanReb)

• Indicate the progress of the initial file tree loading (Show the progress of the file tree scanning dune#2459, fixes dune running in $HOME is super slow dune#2374,
  @bobot)

• Build .cm[ox] files for executables more eagerly. This speeds up builds at
  the cost of building unnecessary artifacts in some cases. Some of these extra
  artifacts can fail to built, so this is a breaking change. (Build .cmx/.o files more eagerly for executables dune#2268, @rgrinberg)

• Do not put the <package>.install files in the source tree unless -p or
  --promote-install-files is passed on the command line (opam pin dune fails as dune.install is not promoted dune#2329, @diml)

• Change implicit_transive_deps to be false. Implicit transitive deps now must
  be manually enabled (Change implicit_transive_deps to be false for 2.0 dune#2306, @rgrinberg)

• Compilation units of user defined executables are now mangled by default. This
  is done to prevent the accidental collision with library dependencies of the
  executable. (Mangle compilation units of dune generated modules dune#2364, fixes Add name mangling to dune binaries dune#2292, @rgrinberg)

• Enable (explicit_js_mode) by default. (Explicit js mode dune#1941, @nojb)

• Add an option to clear the console in-between builds with
  --terminal-persistence=clear-on-rebuild

• Stop symlinking object files to main directory for stanzas defined jbuild
  files (Remove compat symlinks for jbuild files dune#2440, @rgrinberg)

• Library names are now validated in a strict fashion. Previously, invalid names
  would be allowed for unwrapped libraries (Turn a bunch of compatibility warnings into errors dune#2442, @rgrinberg)

• mli only modules must now be explicitly declared. This was previously a
  warning and is now an error. (Turn a bunch of compatibility warnings into errors dune#2442, @rgrinberg)

• Modules filtered out from the module list via the Ordered Set Language must
  now be actual modules. (Turn a bunch of compatibility warnings into errors dune#2442, @rgrinberg)

• Actions which introduce targets where new targets are forbidden (e.g.
  preprocessing) are now an error instead of a warning. (Turn a bunch of compatibility warnings into errors dune#2442, @rgrinberg)

• No longer install a jbuilder binary. (Good bye jbuilder 👋 dune#2441, @diml)

• Stub names are no longer allowed relative paths. This was previously a warning
  and is now an error (Error when stub names are relative paths dune#2443, @rgrinberg).

• Define (paths ...) fields in (context ...) definitions in order to set or
  extend any PATH-like variable in the context environment. (Add (paths ...) field to (context ...) definition dune#2426, @nojb)

• The diff action will always normalize newlines before diffing. Perviousy, it
  would not do this normalization for rules defined in jbuild files. (Remove jbuild style diff comparison dune#2457,
  @rgrinberg)

• Modules may no longer belong to more than one stanza. This was previously
  allowed only in stanzas defined in jbuild files. (Remove more old jbuild support dune#2458, @rgrinberg)

• Remove support for jbuild-ignore files. They have been replaced by the the
  dirs stanza in dune files. (Remove support for jbuild-ignore files dune#2456, @rgrinberg)

• Add a new config option sandboxing_preference, the cli argument --sandbox,
  and the dep spec sandbox in dune language. These let the user control the level of
  sandboxing done by dune per rule and globally. The rule specification takes precedence.
  The global configuration merely specifies the default.
  (Global sandboxing setting dune#2213, @aalekseyev, @diml)

• Remove support for old style subsystems. Dune will now emit a warning to
  reinstall the library with the old style subsystem. (Remove support for old style subsystems dune#2480, @rgrinberg)

• Add action (with-stdin-from ) to redirect input from
  when performing . (Add (with-stdin-from ...) dune#2487, @nojb)

• Change the automatically generated odoc index to only list public modules.
  This only affects unwrapped libraries ([odoc] module visibility and entry list dune#2479, @rgrinberg)

• Set up formatting rules by default. They can be configured through a new
  (formatting) stanza in dune-project (Dune2 formatting dune#2347, fixes dune 2: set up formatting rules by default dune#2315, @emillon)

• Change default target from @install to @all. (Change the default target to @all dune#2449, fixes [PROPOSAL] Change the Default Target dune#1220,
  @rgrinberg)

• Include building stubs in @check rules. (@rgrinberg, Compile stubs for @check dune#2530)

• Get rid of ad-hoc rules for guessing the version. Dune now only
  relies on the version written in the dune-project file and no
  longer read VERSION or similar files (Simplify the computing of version numbers dune#2541, @diml)

• In (diff? x y) action, require x to exist and register a
  dependency on that file. (Improvements to diff? semantics dune#2486, @aalekseyev)

• On Windows, an .exe suffix is no longer added implicitly to binary names that
  already end in .exe. Second, when resolving binary names, .opt variants are no
  longer chosen automatically. (Cleanup handling of .opt and .exe dune#2543, @nojb)

• Make (diff? x y) move the correction file (y) away from the build
  directory to promotion staging area. This makes corrections work with
  sandboxing and in general reduces build directory pollution. (Improvements to diff? semantics dune#2486,
  @aalekseyev, fixes [RFC] Better semantic of diff? dune#2482)

• c_flags, c_names and cxx_names are now supported in executable
  and executables stanzas. (Allow c_flags, c_names, cxx_names in executable stanzas dune#2562, @nojb)
  Note: this feature has been subsequently extended into a separate
  foreign_stubs field. (Add support for foreign libraries dune#2659, RFC [RFC] Foreign libraries dune#2650, @snowleopard)

• Remove git integration from $ dune upgrade (Minor improvements for Build_info and install dune#2565, @rgrinberg)

• Add a --disable-promotion to disable all modification to the source
  directory. There's also a corresponding DUNE_DISABLE_PROMOTION environment
  variable. (Add --disable-promotion option dune#2588, fix Feature request: CLI flag or env var to disable .merlin generation dune#2568, @rgrinberg)

• Add a forbidden_libraries field to prevent some library from being
  linked in an executable. This help detecting who accidently pulls in
  unix for instance (Add a forbidden_libraries field to executable stanzas dune#2570, @diml)

• Fix incorrect error message when a variable is expanded in static context:
  %{lib:lib:..} when the library does not exist. (Fix incorrect message "unknown macro" dune#2597, fix Improve error message for %{lib:...} dune#1541,
  @rgrinberg)

• Add --sections option to $ dune install to install subsections of .install
  files. This is useful for installing only the binaries in a workspace for
  example. (Add --sections option to install/uninstall dune#2609, fixes dune install ignores the (vendor) markers dune#2554, @rgrinberg)

• Drop support for jbuild and jbuild-ignore files (Move jbuild support into a separate library dune#2607, @diml)

• Add a dune-action-plugin library for describing dependencies direcly in
  the executable source. Programs that use this feature can be run by a new
  action (dynamic-run ...). (dune-action-plugin - expressing dependencies directly in source code dune#2635, @staronj, @aalekseyev)

• Stop installing the ocaml-syntax-shims binary. In order to use
  future_syntax, one now need to depend on the ocaml-syntax-shims
  package (Stop installing the ocaml-syntax-shims binary dune#2654, @diml)

• Add support for dependencies that are re-exported. Such dependencies
  are marked withre_export and will automatically be provided to
  users of a library (Add re_exports for libraries dune#2605, @rgrinberg)

• Add a deprecated_library_name stanza to redirect old names after a
  library has been renamed (Add a deprecated_library_name stanza dune#2528, @diml)

• Error out when a preprocessor_deps field is present but not
  preprocess field is. It is a warning with Dune 1.x projects
  (Warn about unused preprocessor_deps dune#2660, @Julow)

• Dune will use -output-complete-exe instead of -custom when compiling
  self-contained bytecode executables whenever this options is available
  (OCaml version >= 4.10) (Use -output-complete-exe if available dune#2692, @nojb)

• Add action (with-accepted-exit-codes <pred> <action>) to specify the set of
  successful exit codes of <action>. <pred> is specified using the predicate
  language. (Add (with-exit-codes <pred> <action>) action dune#2699, @nojb)

• Do not setup rules for disabled libraries (Fix @all alias should only build enabled libraries dune#2491, fixes @all doesn't work with enabled_if on libraries dune#2272, @bobot)

• Configurator: filter out empty flags from pkg-config (Configurator: fix noisy empty compiler command-line arguments dune#2716, @AltGr)

• no_keep_locs is a no-op for projects that use lang dune older than 2.0. In
  projects where the language is at least 2.0, the field is now forbidden.
  (Make no_keep_locs a no-op dune#2752, fixes Delete (no_keep_locs) dune#2747, @rgrinberg)

• Extend support for foreign sources and archives via the (foreign_library ...)
  stanza as well as the (foreign_stubs ...) and (foreign_archives ...) fields.
  (Add support for foreign libraries dune#2659, RFC [RFC] Foreign libraries dune#2650, @snowleopard)

• Add (deprecated_package_names) field to (package) declaration in
  dune-project. The names declared here can be used in the (old_public_name)
  field of (deprecated_library_name) stanza. These names are interpreted as
  library names (not prefixed by a package name) and appropiate redirections are
  setup in their META files. This feaure is meant to migrate old libraries which
  do not follow Dune's convention of prefixing libraries with the package
  name. (Allow top-level libraries in old_public_name dune#2696, @nojb)

• The fields license, authors, maintainers, source, bug_reports,
  homepage, and documentation of dune-project can now be overriden on a
  per-package basis. (Allow to set per-package information dune#2774, @nojb)

• Change the default modes field of executables to (mode exe). If
  one wants to build a bytecode program, it now needs to be explicitly
  requested via (modes byte exe). (Only build native executables by default dune#2851, @diml)

• Allow ccomp_type as a variable for evaluating enabled_if. (Add ccomp_type as an enabled_if variable dune#2855, @dra27,
  @rgrinberg)

• Stricter validation of file names in select. The file names of conditional
  sources must match the prefix and the extension of the resultant filename.
  (Stricter control of select filenames dune#2867, @rgrinberg)

• Add flag disable_dynamically_linked_foreign_archives to the workspace file.
  If the flag is set to true then: (i) when installing libraries, we do not
  install dynamic foreign archives dll*.so; (ii) when building executables in
  the byte mode, we statically link in foreign archives into the runtime
  system; (iii) we do not generate any dll*.so rules. (Add flag [disable_dynamically_linked_foreign_archives] to the workspace file dune#2864, @snowleopard)

• Reimplement the bootstrap procedure. The new procedure is faster and
  should no longer stack overflow (New bootstrap dune#2854, @dra27, @diml)

[rgrinberg]

@kit-ty-kate any issues apart from the tests? I will disable those for the final release.

[kit-ty-kate]

@kit-ty-kate any issues apart from the tests? I will disable those for the final release.

If you say the tests are fine, apart from that there are a few failures that I didn't catch the first time in parany and minicli (Error: 'action' was deleted in version 2.0 of the dune language. Use a rule stanza with the alias field instead) and they are both missing a dune-project file so this should be fine.

Also there were one failure in the tests related on how the ounit package (installed in <prefix>/lib/oUnit) is now a redirection for ounit2 using the required field in the META file. However this won't work with dune 2.0 (if a project uses (lang dune 2.0) of course). cc @gildor478 who should be made aware of this.

l'll do a PR to fix those packages. Other than that it looks fine.

[rgrinberg]

If you say the tests are fine, apart from that there are a few failures that I didn't catch the first time in parany and minicli (Error: 'action' was deleted in version 2.0 of the dune language. Use a rule stanza with the alias field instead) and they are both missing a dune-project file so this should be fine.

The tests are fine. Their issue is that they’re not correctly factored into packages, so it’s a pain to run them in opam’s CI.

Also there were one failure in the tests related on how the ounit package (installed in <prefix>/lib/oUnit) is now a redirection for ounit2 using the required field in the META file. However this won't work with dune 2.0 (if a project uses (lang dune 2.0) of course). ***@***.*** who should be made aware of this.

Yes, I’m not sure how to work around this. The easiest way is to port oUnit to dune. In dune, we have a way to alias library names without going through META files. We might be able to offer a workaround. It’s possible for dune to interpret META defined libraries without artifacts as aliases. This would make them work (implicit_transitive_deps false). I’ve copied Jeremie to see what he thinks of the idea.

…

On Nov 15, 2019, 1:55 AM +0900, Kate ***@***.***>, wrote: > @kit-ty-kate any issues apart from the tests? I will disable those for the final release. Also there were one failure in the tests related on how the ounit package (installed in <prefix>/lib/oUnit) is now a redirection for ounit2 using the required field in the META file. However this won't work with dune 2.0 (if a project uses (lang dune 2.0) of course). cc @gildor478 who should be made aware of this. l'll do a PR to fix those packages. Other than that it looks fine. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[gildor478]

Yes, I’m not sure how to work around this. The easiest way is to port oUnit to dune. In dune, we have a way to alias library names without going through META files. We might be able to offer a workaround. It’s possible for dune to interpret META defined libraries without artifacts as aliases. This would make them work (implicit_transitive_deps false). I’ve copied Jeremie to see what he thinks of the idea.

ounit is already ported to dune, but I had to "adapt" to adapt in between dune and OPAM the transition, so I might have made mistake in the process.
https://github.com/gildor478/ounit

Feel free to send me a fix for ounit, I don't really know how to fix the situation to make a clean transition to from lib/oUnit to lib/ounit2.

[rgrinberg]

This feature is only available since dune 2.0 unfortunately. Though It’s possible that we’ll backport it. If we decide to backport, I will apply it to ounit.

…

On Nov 15, 2019, 4:58 PM +0900, Sylvain Le Gall ***@***.***>, wrote: > Yes, I’m not sure how to work around this. The easiest way is to port oUnit to dune. In dune, we have a way to alias library names without going through META files. We might be able to offer a workaround. It’s possible for dune to interpret META defined libraries without artifacts as aliases. This would make them work (implicit_transitive_deps false). I’ve copied Jeremie to see what he thinks of the idea. ounit is already ported to dune, but I had to "adapt" to adapt in between dune and OPAM the transition, so I might have made mistake in the process. https://github.com/gildor478/ounit Feel free to send me a fix for ounit, I don't really know how to fix the situation to make a clean transition to from lib/oUnit to lib/ounit2. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[gildor478]

Is there anything I could do that will help people to not have problem using dune 2 and OUnit?

[ghost]

@kit-ty-kate @rgrinberg, what is the issue with ounit and dune 2? If an ocamlfind package only has a META file and no dune-package file, dune will read the META file.

[kit-ty-kate]

@kit-ty-kate @rgrinberg, what is the issue with ounit and dune 2? If an ocamlfind package only has a META file and no dune-package file, dune will read the META file.

The issue is for the compatibility package if people use it and (lang dune 2.0). The latest version of ounit is just a redirection to ounit2, however that doesn't work for people using (lang dune 2.0) as it won't include -I <prefix>/lib/ounit2 (like it did with (lang dune 1.x)) during compilation. As such, the compatibility package doesn't work anymore for people (strictly) using dune 2.0. One possible fix would be to copy the content of <prefix>/lib/ounit2 to <prefix>/lib/oUnit.

Closing this PR since #15334 is open

[rgrinberg]

Just to update everyone, we in fact reverted the default for implicit_transitive_deps in 2.0. So this is a non issue regardless.

…

On Nov 21, 2019, 8:01 PM +0700, Kate ***@***.***>, wrote: > @kit-ty-kate @rgrinberg, what is the issue with ounit and dune 2? If an ocamlfind package only has a META file and no dune-package file, dune will read the META file. The issue is for the compatibility package if people use it and (lang dune 2.0). The latest version of ounit is just a redirection to ounit2, however that doesn't work for people using (lang dune 2.0) as it won't include -I <prefix>/lib/ounit2 (like it did with (lang dune 1.x)) during compilation. As such, the compatibility package doesn't work anymore for people (strictly) using dune 2.0. One possible fix would be to copy the content of <prefix>/lib/ounit2 to <prefix>/lib/oUnit. Closing this PR since #15334 is open — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[kit-ty-kate]

oh.. Too bad, I liked that change in dune 2.0. Is it planned to be defaulted to false again in future version?

[ghost]

We are just waiting for better compiler support basically, in the form of "hidden includes". Once this is supported, it can become the default in the next major version.

[rgrinberg]

oh.. Too bad, I liked that change in dune 2.0. Is it planned to be defaulted to false again in future version?

It's still recommended to use in your own personal projects. It just has some caveats that are going to surprise some users.