stack install hmatrix is ugly

[ndmitchell]

Currently installing hmatrix with stack is difficult. The advice is to download some files then do:

stack install hmatrix --flag hmatrix:openblas --extra-lib-dirs=C:\...\OpenBLAS\lib --extra-include-dir=C:\...\OpenBLAS\include

Alas, that doesn't work because extra-include-dir is not a valid flag anymore. But either way, it would be nice if it was simpler and didn't require any download or complex messing around.

[mboes]

This category of use case is pretty much what my colleague @YPares's @tweag work on Nix support for Stack is meant to help. The idea is: leverage an existing lightweight system to install system packages (such as OpenBLAS) with project-local scope (so no global install and then pointing to Stack the exact location required). See PR #1285.

It's a cross platform solution, so should work where Docker does not (stack --docker is another solution to your problem I think). But I'm not sure yet just how good Nix's support for Windows really is at the moment. So in your particular case it might not help. Yves and I would be curious to get feedback from Windows users such as yourself on actual applications.

[YPares]

@ndmitchell It's actually --extra-include-dirs, you forgot an s.

[ndmitchell]

@YPares - thanks - that's pulled from the hmatrix website, and in this case, I think the command line parser should be smart enough to default if there is only one prefix.

@mboes / @YPares - what would this look like as someone who wants to use a package the depends on hmatrix? Would I have to do anything, or would hmatrix have to tweak their stack.yaml to add Nix info? My guess is that Nix on Windows is not that great (it's a weird intersection of people), but only way to be sure is try it and find out.

[YPares]

The Nix support conforms to the existing Stack model. That is, if you want to build a project, it'll do its magic and transparently deal with all dependencies (haskell or system) for you. It doesn't deal with libraries or apps installed using the global stack config. And owing to the fact that projects are non-compositional, if you're building your own project, you'll have to copy the relevant snippets (e.g. the nix: section in the upstream projects if any) in your own stack.yaml. That's the best we can do without extending the .cabal format proper.

So if you want to build an app that uses hmatrix, if the author of that app added a nix: section to the stack.yaml, stack build should just work (no need to install OpenBLAS manually). If you have a project of your own that depends on hmatrix, you can add such a section to your stack.yaml so that others can build your project without hassle. If you stack install using the global config (ie not in the scope of any project), then you're out of luck: create a stack.yaml first or add the section to your ~/.stack/config.yaml.

[ndmitchell]

That's a little unfortunate. If I am using hmatrix, as a transitive dependency (e.g. I'm really using hnn), then I don't want to care about hmatrix at all - it's entirely an implementation detail of a library I care about. Couldn't Stack/Stackage suck the Nix bits out of the stack.yaml if the hmatrix author included it? Or maybe have some extra metadata somewhere? You can put custom metadata in a Cabal project I think (something like beginning with X or some other prefix) which might work too.

Until stack install "just works" for projects depending on hmatrix there will remain a non-zero cost to making use of hmatrix, which is a bit sad.

[YPares]

I agree. That's just the limit of the model that consists in pushing more and more metadata into the stack.yaml, which is for project metadata, not package metadata. If you want to make ​packages​ self contained, then as you say, the right place to push the metadata is in the .cabal I think, and to ideally have the Cabal library deal with it (it just takes invoking nix-shell during evaluation of Setup.lhs, which would transparently make the required system libraries available).

[ndmitchell]

Maybe if a custom Setup.hs detected an environment variable STACK and then wrote out some information in a predictable file location (perhaps given by the env var), then Stack would know what needed to be done?

[ocramz]

IIUC, the .cabal file can't accept relative paths; I mention this because I'm working on a package that has to build an FFI dependency and link the dynlibs from GHC afterwards. I use the "ugly" workaround of specifying --extra-lib-dirs and --extra-include-dirs in the stack build command using a combination of shell script and makefile. There has to be a platform-independent way to solve this..

[hsenag]

Isn't "pkgconfig" the way for cabal files to specify these dependencies?

[mboes]

Yes and no. pkgconfig names don't always correspond to the distro package names. Which in any case change from distro to distro.

[peti]

Distros could provide a (system-wide) config file that maps the pkgconfig names used on Hackage to the package name that's appropriate for that particular distro. That is what cabal2nix does internally, too.

[sindikat]

For whom it may be useful:

I use NixOS and wanted to install SubHask, which requires HMatrix to be exactly 0.16.1.5. This is how my stack.yaml looks like (with some packages omitted for brevity):

resolver: lts-5.13
packages:
- '.'
extra-deps:
- subhask-0.1.1.0
- hmatrix-0.16.1.5
flags: {}
extra-package-dbs: []
# extra-include-dirs: [/path/to/dir]
extra-lib-dirs:
- /nix/store/mlpy5chf205z48nl2g089g6y7a6g32q0-gfortran-4.9.3/lib/
nix:
  enable: true
  packages:
  - blas
  - liblapack
  - gfortran

[mgsloan]

#2122 is also somewhat relevant. It would allow stack install hmatrix to use the stack.yaml from the tarball.

[mpilgrem]

I am closing given the lack of activity over a long period of time and Stack and hmatrix in 2024 is likely somewhat different to the position in 2016.