Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LibTomMath backend support #155

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open

Conversation

antoinemine
Copy link
Collaborator

This PR adds LibTomMath support as alternative backend to GMP and MPIR.
This is especially useful for compilation under Windows.

Using LibTomMath is enabled using the -tommath option in configure.

Tested under Linux with LibTomMath versions 1.2.1 and 1.3.0.

Caveats:

  • not all Zarith functions are supported with LibTomMath (they raise a failure instead); all main arithmetic, Boolean, and bit operations are supported though
  • the binding is not as well optimized for memory and time as the GMP one
  • hash functions give different results between GMP and LibTomMath
  • the marshalling formats for GMP and LibTomMath are not compatible
  • compilation with 1.3.0 gives a depreciation warning for mp_expt_u32; however, 1.2.1 does not have the replacement function and I did not find any convenient way to check the library version number!

Enable with ./configure -tommath
Requires LibTomMath 1.2.0.
Not all Zarith functions are supported.
Not fully tested.
This backend is also slower than the GMP / MPIR version.
Building against the develop branch of LibTomMath is necessary on Windows as get mp_get_float is missing in 1.2.0. However, due to API changes, this version does not compile against 1.2.0 anymore.
Z.backend is a string with value "GMP", "MPIR" or "LibTomMath".
Test zq.ml continues after uncountering unimplemented functions.
Test zq.ml now has backend-dependent reference outputs.
independent from the digit size used by LibTomMath.
- reverted the hashing function in LibTomMath backend (simple and fast, but not GMP compatible)
- the tests now depends on the digit_bits value
- some infomration about LibTomMath in the README
Add unsigned int support for `fits_` and `to_` functions
early clearning of LibTomMath integers
check for memory leaks
@antoinemine
Copy link
Collaborator Author

I'd be interested in some (even light) feedback on this PR, as we have interested users for a LibTomMath backend (this is available on Windows, unlike GMP, and still developed, unlike MPIR). @xavierleroy, what do you think?

@xavierleroy
Copy link
Contributor

Well, you're adding a 2500-line C file that is mostly a cut-and-paste from a 3500-line C file (caml_z.c), so it's going to be hard to maintain.

Also, GMP is available on Windows, see e.g.
https://packages.msys2.org/package/mingw-w64-x86_64-gmp
https://github.com/gx/gmp
https://stackoverflow.com/questions/47359417/how-to-compile-gmp-for-windows-using-visual-studio
https://gmplib.org/manual/Notes-for-Particular-Systems#Notes-for-Particular-Systems

So, I need to better understand the Windows problem this PR is trying to solve.

@hhugo
Copy link
Contributor

hhugo commented Aug 23, 2024

(The CI is running the testsuite on windows, with the mingw OCaml port and gmp)

@antoinemine
Copy link
Collaborator Author

Well, I know some users that rely on Visual Studio on Windows, for better or worse. LibTomMath has an official and maintained support for compilation with VS. As far as I understand, this is not really the case for GMP nor MPIR (what I see currently are mainly one-off efforts that are a bit hackey, and repositories stuck to an old version of GMP).

@hhugo
Copy link
Contributor

hhugo commented Aug 24, 2024

Well, I know some users that rely on Visual Studio on Windows, for better or worse. LibTomMath has an official and maintained support for compilation with VS. As far as I understand, this is not really the case for GMP nor MPIR (what I see currently are mainly one-off efforts that are a bit hackey, and repositories stuck to an old version of GMP).

Maybe we should setup the CI to test the msvc port of OCaml then.

@xavierleroy
Copy link
Contributor

I asked @dra27 about where to find a GMP package after his talk on OPAM for Windows, and the short answer was: opam install conf-gmp should quite certainly work for the Mingw64 port of OCaml, but more work is needed for the MSVC port.

Relatedly, there seems to be a binary package of GMP for MSVC provided by vcpkg: https://vcpkg.io/en/package/gmp . With luck, that may be enough to build Zarith with ocaml-msvc.

At any rate. I'd like the people who really want Zarith with MSVC to explore these possibilities, before we embark into adding 2500 lines of C code to support an obscure alternative to GMP.

@dra27
Copy link
Member

dra27 commented Sep 8, 2024

With #157, I was able to build a tests-passing Zarith using MSVC 5.2.0 from opam and this vcpkg.json:

{
  "dependencies": [ "gmp" ],
  "builtin-baseline": "68d349964cb4e8da561fd849d9491e6ba11c5681",
  "overrides": []
}

Having done vcpkg install, I then ran

$ PKG_CONFIG_PATH=/cygdrive/c/Devel/Zarith/vcpkg_installed/x64-windows/lib/pkgconfig ./configure

and then edited the resulting Makefile.config to change /cygdrive/c to C:/, then one can run:

$ make
$ PATH="/cygdrive/c/Devel/Zarith/vcpkg_installed/x64-windows/bin:$PATH" make tests

The good bit is that including vcpkg in opam shouldn't take as long as I thought it might (i.e. it won't need opam 3.0...), but it will take a bit longer than this train journey...

@xavierleroy
Copy link
Contributor

Very nice! Thanks a lot for the experiment and for #157. Long train journeys are great :-)

@christoph-cullmann
Copy link

Hi, the point why we at AbsInt prefer the extra backend is that we can build the upstream tommath on all platforms without any third party patches with both MSVC and clang-cl. This highly increases the trust in that lib there.

With gmp or mpir that is not feasible. I see above links to forks and the https://vcpkg.io/en/package/gmp.html but still the project itself is not interested in that and has no mainline support for MSVC and Co.

I see the point that this adds complexity here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants