Commits on Jul 3, 2023

1. Port to dune …

   Motivation
   ==========
   
   With a port to dune, the ctypes library can be embedded in larger dune
   projects simply by including it in the directory tree of the bigger
   project. This in turn will allow MirageOS unikernels to use Ctypes
   seamlessly as part of embedded compilation, using the standard
   cross-compilation and library variants support built into Dune. The goal
   is to make Ctypes the default FFI interface to MirageOS, and have it
   work out of the box on all of the backends. All existing OS platforms
   should be supported as well before merge, including Windows.
   
   Porting Approach
   ================
   
   This PR still installs ocamlfind libraries in the same way as the
   previous Makefile infrastructure, so backwards compat is preserved.
   However, the findlib schema has been slightly modified to separate out
   the foreign library from the core ctypes library. We now install:
   
   - ctypes that contains ctypes.top and ctypes.stubs as before. There are
     aliases to ctypes.foreign that redirect existing uses of those.
   - ctypes-foreign contains the foreign library.
     All of the configuration logic for libffi is now in
     src/ctypes-foreign-base, so deleting these directories will remove the
     ffi build logic without touching the core library.
   
   Since dune by default has stricter warnings enabled (the default
   --profile=dev mode), the PR currently sprinkles files with [@@@warning
   tags. The warnings can be fixed as well if desired, but that would muddy
   this PR so not done yet.
   
   Layout
   ======
   
   The previous packaging would install ctypes and ctypes.stubs in the same
   library. Depending on the build system, it means that it's possible to
   specify a dependency on ctypes and use ctypes.stubs succesfully. This is
   not the case with Dune since it installs these in different directories.
   This means that it is necessary to patch these reverse dependencies,
   though it can be argued that they were relying on a bug.
   
   Followup work
   =============
   
   A test uses OCaml syntax to skip test on windows. Once it's acceptable
   to use Dune 3.9, it should be updated to use build_if.
   
   Test and dynamic libraries
   ==========================
   
   Due to a different linking model, there are some changes in the test
   suite regarding dynamic library loading.
   
   Dynamic symbols in ocaml <4.06
   ------------------------------
   
   The setup for tests is that each test executable tests some properties
   using both "stubs" and "foreign" strategies to refer to some symbols
   define in `tests/clib/` (the `test_functions` library).
   
   The symbols are accessed through both strategies:
   - as a linked symbol (through an `external` via stubs)
   - through the dynamic loader (through `dlopen` via foreign)
   
   Dune links the stubs statically, so by default the symbols would not be
   visible to the dynamic loader. OCaml sets `-Wl,-E` in `LDFLAGS` to make
   these symbols visible at runtime, but until
   ocaml/ocaml@edbba02
   (between 4.05 and 4.06), this was ignored when linking executables.
   
   So this commit adds these flags when building tests for older versions.
   An alternative would be to use `(link_flag)` in an `(env)` stanza but
   this requires dune 3.
   
   This issue does not affect the original make-based build because it
   assembles the test executables a bit differently: the `test_functions`
   library is linked dynamically to the test executable. So the symbols are
   already visible to the dynamic loader, even when `-Wl,-E` is not set.
   
   clib loading
   ------------
   
   This change is specific to windows (but does not change the behavior on
   Linux)
   
   It modifies the test setup a bit so that the contents of `clib` are
   dynamically loaded at the beginning of each test. This ensures that
   later foreign calls (that use the default handle) will succeed.
   
   It was not necessary before because `clib` was linked dynamically. Now
   that it is linked statically, dynamic symbols do not have access to the
   symbols in the main executable. This is a problem on windows which does
   not have the concept of `-Wl,--export-dynamic`. Also, on windows the
   stubs DLL can not be loaded directly so we recompile the library using
   plain `%{cc}` instead of going through ocamlopt and flexlink.
   
   Instrumentation
   ===============
   
   This uses dune instrumentation for coverage The instructions are now:
   
       opam install bisect_ppx
       dune runtest --instrument-with bisect_ppx --force
       bisect-ppx-report html
   
   See <https://github.com/aantron/bisect_ppx#Dune>
   
   Depext handling
   ===============
   
   This port uses conf-libffi instead of hardcoding depexts in
   ctypes-foreign.

   

   emillon committed Jul 3, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for 3bc53d4
   • Browse repository at this point

   Copy the full SHA

   3bc53d4 View commit details

   Browse the repository at this point in the history