feat: Azure blob storage support

[bakjos]

This PR adds support to use Azure Blob storage as a proxy option.

[mostynb]

Hi- thanks for the contribution, this looks like a nice feature.

I have a bunch of minor ("nitpick") issues to fix, but I don't have an azure account to test this with- I assume that you have tested it though?

[bakjos]

@mostynb thanks for the quick review it all does makes sense, I did test it against azure. I’ll make the changes and push a new commit next week (I’m out the rest of the week)

[bakjos]

@mostynb I think I address all the comments, hope I understood all

[mostynb]

Squashed and landed with some minor formatting and documentation tweaks- thanks again!

[bakjos]

Cool @mostynb Thanks to you!!

[karlhepler]

@bakjos - I'm trying to configure the latest version of bazel-remote to work with Azure, but it's not picking up my settings. It doesn't even seem like it's trying to Azure. Any suggestions? Here's my env file with all of my settings.

BAZEL_REMOTE_MAX_SIZE=100
BAZEL_REMOTE_AZBLOB_STORAGE_ACCOUNT=<redacted>
BAZEL_REMOTE_AZBLOB_CONTAINER_NAME=<redacted>
BAZEL_REMOTE_AZBLOB_AUTH_METHOD=shared_key
BAZEL_REMOTE_AZBLOB_SHARED_KEY=<redacted>

[mostynb]

@karlhepler: I think you might need to set a tenant id also- does adding that help?

[bakjos]

@karlhepler As @mostynb said you need to set the tenant id, this is the one I used to test:

AZURE_TENANT_ID=<redacted> 
BAZEL_REMOTE_AZBLOB_STORAGE_ACCOUNT=<redacted> 
BAZEL_REMOTE_AZBLOB_CONTAINER_NAME=<redacted> 
BAZEL_REMOTE_AZBLOB_AUTH_METHOD=shared_key 
AZURE_STORAGE_ACCOUNT_KEY=<redacted> 
BAZEL_REMOTE_ENABLE_ENDPOINT_METRICS=true

[karlhepler]

Thanks @mostynb and @bakjos. I tried those - I also used your config setup on my end, @bakjos. No luck. It's still trying to hit my local ip. Do I need to set the http proxy or anything else like that?

[bakjos]

@karlhepler are you seeing in the logs something like AzBlob Credentials: using Shared Key is it showing an error like mkdir /data/cas.v2: permission denied ? you can also increase the log level BAZEL_REMOTE_ACCESS_LOG_LEVEL=all, and how are you connecting from bazel?

[karlhepler]

Here's an example of what I'm seeing.

2022/08/31 22:32:47 bazel-remote built with go1.18.2 from git commit 5909df148d2487a94e22abbd1b5b22f53c989652.
2022/08/31 22:32:47 Initial RLIMIT_NOFILE cur: 1048576 max: 1048576
2022/08/31 22:32:47 Setting RLIMIT_NOFILE cur: 1048576 max: 1048576
2022/08/31 22:32:47 Limiting concurrent file removals to 5000
2022/08/31 22:32:47 Loading existing files in /cache.
2022/08/31 22:32:47 Sorting cache files by atime.
2022/08/31 22:32:47 Building LRU index.
2022/08/31 22:32:47 Finished loading disk cache files.
2022/08/31 22:32:47 Loaded 0 existing disk cache items.
2022/08/31 22:32:47 Authentication: disabled
2022/08/31 22:32:47 Mangling non-empty instance names with AC keys: disabled
2022/08/31 22:32:47 Starting HTTP server on address :8080
2022/08/31 22:32:47 HTTP AC validation: enabled
2022/08/31 22:32:47 gRPC AC dependency checks: enabled
2022/08/31 22:32:47 experimental gRPC remote asset API: disabled
2022/08/31 22:32:47 Starting gRPC server on address :9092
2022/08/31 22:32:59  GET 404      172.17.0.1 /ac/0a14ba31115f56c9f9479bf4ecbbe1dbc778f874260dcd444be86baed7429420
2022/08/31 22:33:00  PUT 200      172.17.0.1 /cas/0a14ba31115f56c9f9479bf4ecbbe1dbc778f874260dcd444be86baed7429420
2022/08/31 22:33:00  PUT 200      172.17.0.1 /cas/e209c719f27c9315a2dbde80ccf2017be2ea0dd516104003d39083ba4afd4c57
2022/08/31 22:33:00  PUT 200      172.17.0.1 /cas/6d22a804e1e534a79b9db31f301f9083539a32e6aa843331f5b4c9700eb2b402
2022/08/31 22:33:00  PUT 200      172.17.0.1 /ac/0a14ba31115f56c9f9479bf4ecbbe1dbc778f874260dcd444be86baed7429420
2022/08/31 22:33:00  GET 404      172.17.0.1 /ac/e2ef063a2bcd2d36d72de105befb62ce91bd7859d8a8a83e313551950c5ef420
2022/08/31 22:33:00  PUT 200      172.17.0.1 /cas/e2ef063a2bcd2d36d72de105befb62ce91bd7859d8a8a83e313551950c5ef420
2022/08/31 22:33:00  PUT 200      172.17.0.1 /cas/fab6908d304abc0d07654a41b0f3f1b91592166a8cf94e21d3433e23c383e1ee
2022/08/31 22:33:00  PUT 200      172.17.0.1 /cas/4757dbece35363db36b6e9732db84d11f833e41ca761f8b5f61b9ee055b76a55
2022/08/31 22:33:00  PUT 200      172.17.0.1 /ac/e2ef063a2bcd2d36d72de105befb62ce91bd7859d8a8a83e313551950c5ef420
2022/08/31 22:33:00  GET 404      172.17.0.1 /ac/6d19f4ca748410a1d4a77a0ca5744e1d327f15e6e98d31e7d50699cbf4aa9a51
2022/08/31 22:33:00  GET 404      172.17.0.1 /ac/23cf98e1d9255336a23ce0f242cf5cfdc8eb9a49d1584a531c28c420b756b38d
2022/08/31 22:33:00  GET 404      172.17.0.1 /ac/2a2c611e214035863fd4b1a3a491556dd874bd41bd3115cb3d55cae0f141f4c3
2022/08/31 22:33:00  GET 404      172.17.0.1 /ac/49b090eaf7d4ecd2ea6f883cea810537983b84c3bbf1a1de8bee77722c236f3b
2022/08/31 22:33:00  GET 404      172.17.0.1 /ac/bf6c8869a6a19f3bb0eb2a3089c556f4bca483bd8deab1d285765ce0f99357df
2022/08/31 22:33:00  GET 404      172.17.0.1 /ac/bc47ce584a12e47b65f231e102674bb7a69c538404e1bc6ce4d44832744857da
2022/08/31 22:33:01  PUT 200      172.17.0.1 /cas/3e078c52680b431aaac4f0b933fba8179ea1d6974da1800fb3a6beedd0d311c8

[karlhepler]

I wonder if there are cli args I need to pass. I'm just setting everything in environment variables. Hmmm...

[karlhepler]

I'm also using the latest release. Perhaps it hasn't been released yet?

https://github.com/buchgr/bazel-remote/releases/tag/v2.3.8

[bakjos]

Based on the logs, looks like you're using the 2.3.8 image which doesn't contain the changes to support azure, you'll need to build the image from the latest main in order to use azure, not sure when @mostynb will have plans to publish a new version

[mostynb]

Oops- published a new version just now: https://github.com/buchgr/bazel-remote/releases/tag/v2.3.9

[karlhepler]

Thanks! Now there seems to be a new problem. This new version looks like it requires C. I was running it out of distroless/static before, but now that's not working. I converted it over to distroless/base and I also tried the go_base_image. Both return this error:

./bazel-remote: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory

[mostynb]

Thanks! Now there seems to be a new problem. This new version looks like it requires C. I was running it out of distroless/static before, but now that's not working. I converted it over to distroless/base and I also tried the go_base_image. Both return this error:

./bazel-remote: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory

That's correct, bazel-remote now has a cgo dependency for a native code zstdandard implementation (not used by default, yet).

Are you building custom docker images instead of using the official ones? If so, I'm curious to know why. Either way, would you mind creating a new issue to discuss this, so we don't spam @bakjos?

[karlhepler]

OK. Thanks! Sorry for the spam!