Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Motivation ========== With a port to dune, the ctypes library can be embedded in larger dune projects simply by including it in the directory tree of the bigger project. This in turn will allow MirageOS unikernels to use Ctypes seamlessly as part of embedded compilation, using the standard cross-compilation and library variants support built into Dune. The goal is to make Ctypes the default FFI interface to MirageOS, and have it work out of the box on all of the backends. All existing OS platforms should be supported as well before merge, including Windows. Porting Approach ================ This PR still installs ocamlfind libraries in the same way as the previous Makefile infrastructure, so backwards compat is preserved. However, the findlib schema has been slightly modified to separate out the foreign library from the core ctypes library. We now install: - ctypes that contains ctypes.top and ctypes.stubs as before. There are aliases to ctypes.foreign that redirect existing uses of those. - ctypes-foreign contains the foreign library. All of the configuration logic for libffi is now in src/ctypes-foreign-base, so deleting these directories will remove the ffi build logic without touching the core library. Since dune by default has stricter warnings enabled (the default --profile=dev mode), the PR currently sprinkles files with [@@@warning tags. The warnings can be fixed as well if desired, but that would muddy this PR so not done yet. Layout ====== The previous packaging would install ctypes and ctypes.stubs in the same library. Depending on the build system, it means that it's possible to specify a dependency on ctypes and use ctypes.stubs succesfully. This is not the case with Dune since it installs these in different directories. This means that it is necessary to patch these reverse dependencies, though it can be argued that they were relying on a bug. Followup work ============= A test uses OCaml syntax to skip test on windows. Once it's acceptable to use Dune 3.9, it should be updated to use build_if. Test and dynamic libraries ========================== Due to a different linking model, there are some changes in the test suite regarding dynamic library loading. Dynamic symbols in ocaml <4.06 ------------------------------ The setup for tests is that each test executable tests some properties using both "stubs" and "foreign" strategies to refer to some symbols define in `tests/clib/` (the `test_functions` library). The symbols are accessed through both strategies: - as a linked symbol (through an `external` via stubs) - through the dynamic loader (through `dlopen` via foreign) Dune links the stubs statically, so by default the symbols would not be visible to the dynamic loader. OCaml sets `-Wl,-E` in `LDFLAGS` to make these symbols visible at runtime, but until ocaml/ocaml@edbba02 (between 4.05 and 4.06), this was ignored when linking executables. So this commit adds these flags when building tests for older versions. An alternative would be to use `(link_flag)` in an `(env)` stanza but this requires dune 3. This issue does not affect the original make-based build because it assembles the test executables a bit differently: the `test_functions` library is linked dynamically to the test executable. So the symbols are already visible to the dynamic loader, even when `-Wl,-E` is not set. clib loading ------------ This change is specific to windows (but does not change the behavior on Linux) It modifies the test setup a bit so that the contents of `clib` are dynamically loaded at the beginning of each test. This ensures that later foreign calls (that use the default handle) will succeed. It was not necessary before because `clib` was linked dynamically. Now that it is linked statically, dynamic symbols do not have access to the symbols in the main executable. This is a problem on windows which does not have the concept of `-Wl,--export-dynamic`. Also, on windows the stubs DLL can not be loaded directly so we recompile the library using plain `%{cc}` instead of going through ocamlopt and flexlink. Instrumentation =============== This uses dune instrumentation for coverage The instructions are now: opam install bisect_ppx dune runtest --instrument-with bisect_ppx --force bisect-ppx-report html See <https://github.com/aantron/bisect_ppx#Dune> Depext handling =============== This port uses conf-libffi instead of hardcoding depexts in ctypes-foreign. Co-authored by: Anil Madhavapeddy <anil@recoil.org>
- Loading branch information
