Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Azure blob storage support #560

Closed
wants to merge 7 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 63 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,45 @@ OPTIONS:
value. This flag will be removed. (default: 2)
[$BAZEL_REMOTE_S3_KEY_VERSION]

--azblob.tenant_id value The Azure blob storage tenant id to use when
using azblob proxy backend. [$BAZEL_REMOTE_AZBLOB_TENANT_ID,
$AZURE_TENANT_ID]

--azblob.storage_account value The Azure blob storage storage account to
use when using azblob proxy backend.
[$BAZEL_REMOTE_AZBLOB_STORAGE_ACCOUNT]

--azblob.container_name value The Azure blob storage container name to use
when using azblob proxy backend. [$BAZEL_REMOTE_AZBLOB_CONTAINER_NAME]

--azblob.prefix value The Azure blob storage object prefix to use when
using azblob proxy backend. [$BAZEL_REMOTE_AZBLOB_PREFIX]

--azblob.update_timestamps Whether to update timestamps of object on cache
hit. (default: false) [$BAZEL_REMOTE_AZBLOB_UPDATE_TIMESTAMPS]

--azblob.auth_method value The Azure blob storage authentication method.
This argument is required when an azblob proxy backend is used. Allowed
values: client_certificate, client_secret, environment_credential,
shared_key, default. [$BAZEL_REMOTE_AZBLOB_AUTH_METHOD]

--azblob.shared_key value The Azure blob storage account access key to use
when using azblob proxy backend. Applies to AzBlob auth method(s):
shared_key. [$BAZEL_REMOTE_AZBLOB_SHARED_KEY, $AZURE_STORAGE_ACCOUNT_KEY]

--azblob.client_id value The Azure blob storage client id to use when
using azblob proxy backend. Applies to AzBlob auth method(s):
client_secret. [$BAZEL_REMOTE_AZBLOB_CLIENT_ID, $AZURE_CLIENT_ID]

--azblob.client_secret value The Azure blob storage client secret key to
use when using azblob proxy backend. Applies to AzBlob auth method(s):
client_secret. [$BAZEL_REMOTE_AZBLOB_SECRET_CLIENT_SECRET,
$AZURE_CLIENT_SECRET]

--azblob.cert_path value Path to the certificates file. Applies to AzBlob
auth method(s): client_certificate. [$BAZEL_REMOTE_AZBLOB_CERT_PATH,
$AZURE_CLIENT_CERTIFICATE_PATH]

--disable_http_ac_validation Whether to disable ActionResult validation
for HTTP requests. (default: false, ie enable validation)
[$BAZEL_REMOTE_DISABLE_HTTP_AC_VALIDATION]
Expand Down Expand Up @@ -425,7 +464,30 @@ http_address: 0.0.0.0:8080
#
#http_proxy:
# url: https://remote-cache.com:8080/cache

#
#azblob_proxy:
# tenant_id: TENANT_ID
# storage_account: STORAGE_ACCOUNT
# container_name: CONTAINER_NAME
#
# Check https://github.com/Azure/azure-sdk-for-go/tree/sdk/azidentity/v1.1.0/sdk/azidentity/ to
# read about all the azure auth methods
# storage account shared key
# auth_method: shared_key
# shared_key: APP_SHARED_KEY
#
# client secret credentials
# auth_method: client_secret
bakjos marked this conversation as resolved.
Show resolved Hide resolved
# client_id: APP_ID
# client_secret: APP_SECRET
#
# client certificate credentials
# auth_method: client_certificate
# cert_path: PATH_TO_CERTS FILE
#
# default and environment methods don't have any additional parameters
# auth_method: default/environment_credential

# If set to a valid port number, then serve /debug/pprof/* URLs here:
#profile_port: 7070
# IP address to use, if profiling is enabled:
Expand Down
19 changes: 19 additions & 0 deletions cache/azblobproxy/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
load("@io_bazel_rules_go//go:def.bzl", "go_library")

go_library(
name = "go_default_library",
srcs = [
"auth_methods.go",
"azblobproxy.go",
],
importpath = "github.com/buchgr/bazel-remote/cache/azblobproxy",
visibility = ["//visibility:public"],
deps = [
"//cache:go_default_library",
"//cache/disk/casblob:go_default_library",
"@com_github_azure_azure_sdk_for_go_sdk_azcore//:go_default_library",
"@com_github_azure_azure_sdk_for_go_sdk_storage_azblob//:go_default_library",
"@com_github_prometheus_client_golang//prometheus:go_default_library",
"@com_github_prometheus_client_golang//prometheus/promauto:go_default_library",
],
)
28 changes: 28 additions & 0 deletions cache/azblobproxy/auth_methods.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
package azblobproxy

const (
AuthMethodClientCertificate = "client_certificate"
AuthMethodClientSecret = "client_secret"
AuthMethodEnvironmentCredential = "environment_credential"
AuthMethodDefault = "default"
AuthMethodSharedKey = "shared_key"
)

func GetAuthMethods() []string {
return []string{
AuthMethodClientCertificate,
AuthMethodClientSecret,
AuthMethodEnvironmentCredential,
AuthMethodSharedKey,
AuthMethodDefault,
}
}

func IsValidAuthMethod(authMethod string) bool {
for _, b := range GetAuthMethods() {
if authMethod == b {
return true
}
}
return false
}
Loading