Add native Azure Blob support

[hiddeco]

This PR introduces an Azure Blob BucketProvider implementation,
capable of fetching objects from public and private "container"
buckets.

The supported credential types are:

• ClientSecret with tenantId, clientId and clientSecret Secret
  data fields.
• ClientCertificate with tenantId, clientId and clientSecret Secret
  data (with optionally clientCertificatePassword).
• ManagedIdentity with a clientId Secret data field.
• SharedKey with accountKey Secret data field, the account name is
  extracted from the endpoint URL specified on the object.
• ChainedToken with:
  • Environment credentials
  • ManagedIdentity with AZURE_CLIENT_ID environment variable
    (when available)
  • ManagedIdentity with defaults

If no Secret is provided or the chain can not be established, the
Bucket is assumed to be public.

Successor of #513

[hiddeco]

@laozc can you please take a look and see if this does not cover anything your version did?

Reason for this change is that I had a look at the current Azure libraries out there, and this seems to be the one with the brightest future and maintenance expectations. Given this is a new dependency introduction, this has my preference over one we eventually need to replace (or semi-copy/borrow from another project).

[laozc]

Hi @hiddeco

I'm still verifying this PR in Azure environments.
I think it would be better to use the following order during authentication

1. Check for client secret first to use Service Principal password login
2. Check for client certificate to use Service Principal certificate login
3. Use Client ID with Managed Identity login
4. Use Resource ID with Managed Identity login
5. Use Storage Account key based auth login
6. No auth

This may ensure it works as expected when multiple Managed Identites bound on the same VM node for 3) and 4).

func tokenCredentialFromSecret(secret *corev1.Secret) (azcore.TokenCredential, error) {
	tenantID, hasTenantID := secret.Data[tenantIDField]
	clientID, hasClientID := secret.Data[clientIDField]
	clientSecret, hasClientSecret := secret.Data[clientSecretField]
	clientCertificate, hasClientCertificate := secret.Data[clientCertificateField]
	clientCertificatePassword, _ := secret.Data[clientCertificatePasswordField]
	resourceID, hasResourceID := secret.Data[resourceIDField]

	if hasTenantID && hasClientID {
		if hasClientSecret && string(clientSecret) != "" {
			return azidentity.NewClientSecretCredential(string(tenantID), string(clientID), string(clientSecret), nil)
		}
		if hasClientCertificate && string(clientCertificate) != "" {
			certs, key, err := azidentity.ParseCertificates(clientCertificate, clientCertificatePassword)
			if err != nil {
				return nil, fmt.Errorf("failed to parse client certificates: %w", err)
			}
			return azidentity.NewClientCertificateCredential(string(tenantID), string(clientID), certs, key, nil)
		}
	}

	if hasClientID {
		return azidentity.NewManagedIdentityCredential(&azidentity.ManagedIdentityCredentialOptions{
			ID: azidentity.ClientID(clientID)})
	} else if hasResourceID {
		return azidentity.NewManagedIdentityCredential(&azidentity.ManagedIdentityCredentialOptions{
			ID: azidentity.ResourceID(resourceID)})
	}

	return nil, nil
}

Let me get back to you when I have all these scenarios verified.

[hiddeco]

Thanks for the heads up, I will take your suggestions into account and document the reasoning in-code. Please let me know if anything else pops up, and I'll try to address it swiftly.

[hiddeco]

I have updated the code to your suggestion, you may also want to provide some input on #598 (review) and #598 (comment).

[hiddeco]

Integration tests have been disabled again for pull requests after confirmed to be working in:

• https://github.com/fluxcd/source-controller/runs/5407198631?check_suite_focus=true
• https://github.com/fluxcd/source-controller/runs/5407198870?check_suite_focus=true

[hiddeco]

@laozc please see the current state of the PR and let me know if this is acceptable. I will work on updating the tests in the meantime.

[stefanprodan]

LGTM

Thanks @hiddeco and @laozc

PS> We'll need a followup in #599 with Azure specific docs

[hiddeco]

@stefanprodan documented in 687af2f

[darkowlzz]

LGTM!

[hiddeco]

@laozc are you still running tests, or do you think this can be merged as is?

[pjbgf]

LGTM

Really good stuff @hiddeco @laozc!

[laozc]

@hiddeco Please go head with the merge.
It's great to see the support landed in flux.