Completes the prototype implementation of Bazel persistent worker mode

[ulysses4ever]

Started in 132d0af. This commit imports worker's code and sets everything to make
the persistent worker mode available to a regular rules_haskell user.

The worker code lives in tools/worker.

The mode is supported in ghc_nixpkgs-toolchain only, at this point.

[ulysses4ever]

Note. CI fails on examples and tutorial because, as it turns out, we accidentally added the requirement to load worker dependencies even when the worker is not used. A solution based on select is currently being crafted.

[ulysses4ever]

Thanks @aherrmann for the review! Hopefully, all the comments are addressed. I also changed the implementation of the worker switch to use select so that we don't require loading worker dependencies when the worker is not used. Please, take a look when have time.

[aherrmann]

Great work, thank you! Only one minor comment about the rule_type tag.

I've tested this on my machine and bazel test --define use_worker=True --worker_sandboxing //... passes.

[ulysses4ever]

@aherrmann thanks!

I've tested this on my machine and bazel test --define use_worker=True --worker_sandboxing //... passes.

Yes, if you enable worker dependencies -- it's all good! But if you don't, plain bazel test //... fails (for me, not sure if it's some local issue). Instead, bazel test //tests/... succeeds. Could you try it? If it fails for you too we might need to fix the README: it mentions bazel test //... now.

[aherrmann]

Yes, if you enable worker dependencies -- it's all good! But if you don't, plain bazel test //... fails (for me, not sure if it's some local issue). Instead, bazel test //tests/... succeeds. Could you try it?

Yes, same for me. That makes sense as //... includes //tools/worker:bin. bazel test //... will also try to build (not execute) non-test targets. You can observe the same if you turn e.g. //tests/binary-simple:binary-simple into a haskell_binary instead of haskell_test and run bazel test //tests/binary-simple/....

[ulysses4ever]

Thanks for checking! It doesn't seem entirely reasonable to me to build non-test targets during bazel test but well… I guess I'll change README then.

[ulysses4ever]

Andreas, I tried to address all your comments. What do you think?

[ulysses4ever]

I tried to use “Apply suggestion” button for the first time in my life to see what happens. I wonder if it's possible to squash the resulting tiny commit, on merge.

[aherrmann]

AFAIK mergify is not configured to squash-merge.

Thanks for addressing the comments. Looks good to me.

[ulysses4ever]

I'll squash manually then. What's the proper way to proceed after that? Add the merge-queue label?

[ulysses4ever]

The netlify failure doesn't seem related to the PR. I'm confused…

[avdv]

Hi @ulysses4ever!

I know, it's been a while... but do still have the worker.proto file laying around somewhere? It would be nice if you could provide it to us. 😄

[ulysses4ever]

Hey! I'll look into it today. I'm surprised that something is missing: I'd expect that any missing file should be easily generated or downloaded or something... But it's been four years since I touched it so I may very well be wrong.

It's kinda ironic that a server machine I was using for it (and which I was using since) was recently (this July) decommissioned, and I had to move to another one, and in the process I nuked a lot of stuff. Probably including the playground for this project. But let's be optimistic and wait until I try to look it up locally.

[ulysses4ever]

Oh, I see, it's the generated file that I committed instead of the source. Shame on me!

[ulysses4ever]

All good. I developed this code as a separate repo first: https://github.com/tweag/bazel-worker/ (should have been a git submodule, perhaps, but they're such a pain...) and it does have worker.proto:

• https://github.com/tweag/bazel-worker/blob/master/src/worker.proto

Note that it's a file provided by Bazel itself:

• https://github.com/bazelbuild/bazel/blob/19293673be14e68598f032b1c61fa991884305ad/src/main/protobuf/worker_protocol.proto

It's fun to see the diff between the two: they extended the protocol quite a bit over the years.

[avdv]

Ah, nice. Thank you for the swift response! 💯