Enable compiling cbits directly into Haskell targets

[aherrmann]

Currently haskell_library and the other Haskell rules only take Haskell source files (.hs, .lhs, .hsc, ...) as input to the srcs attribute. In particular, it is not allowed to pass .c files to the srcs attribute, or alternatively a dedicated csrcs attribute.

This forces Hazel to generate separate cc_library targets for any cbits in Hackage packages, introducing additional C library dependencies. These quickly add up, and increase the risk of exceeding the MACH-O header size limit on MacOS on dynamic linking.

GHC itself accepts .c files as sources, and it looks like Cabal passes c-sources directly to ghc. Does anything speak against also accepting .c files as sources in rules_haskell, either in srcs, or a separate csrcs?

[mboes]

I think this is a GHC misfeature. I think rules should be as focused as possible, not bring along the kitchen sink. We already have a rule (cc_library) that works fantastic for building C code. If instead of deferring to that for building C code, we make that a special case of haskell_library, then we need to handle a lot more flags than we do now (flags for Haskell, flags for C etc). I think we should trust cc_library to build C code properly more than we should trust GHC, including taking into account different build modes (dbg, opt etc) and other flags that affect the C output.

I don't understand the MACH-O header size limit argument. Whether the arguments are passed to GCC via GHC or to GCC directly shouldn't make a difference. In both cases you just end up with a collection of object files, which you then have to link with the rest. You can force this static linking of cbits if you like. No extra MACH-O entry in that case.

[aherrmann]

I think we should trust cc_library to build C code properly more than we should trust GHC, including taking into account different build modes (dbg, opt etc) and other flags that affect the C output.

That's a good point. Indeed, handling those flags right would require additional work in rules_haskell, which would probably duplicate cc_library.

I don't understand the MACH-O header size limit argument. Whether the arguments are passed to GCC via GHC or to GCC directly shouldn't make a difference. In both cases you just end up with a collection of object files, which you then have to link with the rest. You can force this static linking of cbits if you like. No extra MACH-O entry in that case.

The issue appears further down the line when something else depends on such a library. Assuming Hackage packages foo and bar, where foo depends on bar. Hazel will generate foo, foo-cbits, bar, and bar-cbits, where foo-cbits depends on bar-cbits, and foo depends on foo-cbits and bar. Transitively, foo also depends on bar-cbits. Due to how Bazel handles transitive dynamic library dependencies, the dynamic library for foo will load foo-cbits, bar-cbits, and bar. I.e. all transitive cbits dependencies. If bar-cbits had been compiled directly into bar, then foo would only load bar and there wouldn't be any -cbits targets.

Fortunately, with the latest work on shortening the length of RPATH entries (#614) and reducing the amount of RPATH entries (#663, #686) we're well below the MACH-O header size limit even for large dependency graphs. So, the current -cbits approach works well enough.