feat: secret versions and settings/option names (#109)

* feat: add support for providing secret versions for secret retreival

* feat: use same http client for secret and setting client

* feat: add option for http client for secret and setting client

* feat: add support for provided/custom http clients

* docs: update readme and documentation

README.md

@@ -9,10 +9,14 @@
   * [Prerequisites](#prerequisites)
   * [Example](#example)
 * [Usage](#usage)
-  * [Options](#options)
+  * [Configuration](#configuration)
+    * [Options](#options)
+    * [Environment variables](#environment-variables)
   * [Required](#required)
   * [Parser](#parser)
   * [Pre-populated struct and default values](#pre-populated-struct-and-default-values)
+  * [Context and timeouts](#context-and-timeouts)
+  * [Key Vault secret versions](#key-vault-secret-versions)
   * [App Configuration setting labels](#app-configuration-setting-labels)
   * [Authentication](#authentication)
     * [Built-in credentials](#built-in-credentials)
@@ -73,7 +77,10 @@ go get github.com/KarlGW/azcfg
 
 ### Example
 
-Using a managed identity as credentials on an Azure service. For other authentication and credential methods see the sections [Authentication](#authentication) and [Credentials](#credentials).
+Using a system assigned managed identity as credential on an Azure service. For other authentication and credential methods see the sections [Authentication](#authentication) and [Credentials](#credentials).
+
+This scenario requires minimal configuration, as `azcfg` automatically detects if the platform is configured
+with a managed identity.
 
 ### Example with secrets (Key Vault)
 
@@ -230,8 +237,12 @@ Slices are supported if the secret/setting are comma separated values (spaces ar
 **Numbers**
 `1,2,3`
 
+### Configuration
+
+Configuration of the parser can be done with options provided to `Parse` or to `NewParser` and environment variables.
+Options will override environment variables if provided.
 
-### Options
+#### Options
 
 Options can be set on `Parse` or the parser with `NewParser`. For the available options see [Options](https://pkg.go.dev/github.com/KarlGW/azcfg#Options).
 
@@ -266,6 +277,40 @@ func main() {
 }
 ```
 
+#### Environment variables
+
+These are the environment variables that are available to use to configure parsing.
+
+#### General
+
+* `AZCFG_CLOUD` - Target Cloud (Azure Puplic, Azure Government and Azure China).
+* `AZCFG_CONCURRENCY` - Concurrency limit for secret and setting retrieval.
+* `AZCFG_TIMEOUT` - Timeout for the underlying HTTP client.
+
+#### Authentication
+
+* `AZCFG_TENANT_ID` - Tenant ID for service principal/app registration.
+* `AZCFG_CLIENT_ID` - Client/App ID for service principal/app registration or user assigned managed identity.
+* `AZCFG_CLIENT_SECRET` - Secret for service principal/app registration.
+* `AZCFG_CLIENT_CERTIFICATE` - PEM certificate encoded in Base64 for service principal/app registration.
+* `AZCFG_CLIENT_CERTIFICATE_PATH` - Path to PEM certificate for service principal/app registration.
+* `AZCFG_AZURE_CLI_CREDENTIAL` - Use Azure CLI credential.
+* `AZCFG_APPCONFIGURATION_ACCESS_KEY_ID` - Access key ID for App Configuration.
+* `AZCFG_APPCONFIGURATION_ACCESS_KEY_SECRET` - Access key secret for App Configuration.
+* `AZCFG_APPCONFIGURATION_CONNECTION_STRING` - Connection string for App Configuration.
+
+More details on how to authenticate can be found in the [Authentication](#authentication) section.
+
+#### Secrets
+
+* `AZCFG_KEYVAULT_NAME` - Name of Key Vault containing secrets.
+* `AZCFG_SECRETS_VERSIONS` - Secret names and versions when requiring specific secret versions.
+
+#### Settings
+
+* `AZCFG_APPCONFIGURATION_NAME` - Name of App Configuration containing settings.
+* `AZCFG_SETTINGS_LABEL` - Label for the intended settings.
+* `AZCFG_SETTINGS_LABELS` - Setting names and labels when requiring specific labels for specific settings.
 
 ### Required
 
@@ -366,19 +411,38 @@ func main() {
 }
 ```
 
+### Context and timeouts
+
+Every call to `Parse` or `parser.Parse` requires a `context.Context`. This is
+the main way of setting timeouts. However, the internal clients
+for fetching secrets and settings and their underlying HTTP client has
+a default timeout of 30 seconds. This can be configured with setting
+the `Timeout` field on the `Options` struct in an option function, or
+using the dedicated option function, `WithTimeout`.
+
+### Key Vault secret versions
+
+Secrets in Key Vault have versions associated with them. By default the latest version
+is retrieved.
+
+To target specific versions for specific secrets:
+
+- Set the secret names with their associated versions to the environment variable `AZCFG_SECRETS_VERSIONS` with format `secret1=version1,secret2=version2`.
+- Use the option function `WithSecretsVersions` and provide a `map[string]string` with the secret name as key and version as value.
+
 ### App Configuration setting labels
 
 Settings in App Configuration can have labels associated with them.
 
 To target a specific label for all settings:
 
-- Set the label to the environment variable `AZCFG_APPCONFIGURATION_LABEL`.
-- Use the option function `WithLabel`.
+- Set the label to the environment variable `AZCFG_SETTINGS_LABEL`.
+- Use the option function `WithSettingsLabel`.
 
 To target speciefic settings with specific labels:
 
-- Set the labels to the environment variable `AZCFG_APPCONFIGURATION_LABELS` with format: `setting1=label1,setting2=label2`.
-- Use the option function `WithLabels` and provide a `map[string]string` with the setting as key and label as value.
+- Set the setting names with their associated labels to the environment variable `AZCFG_SETTINGS_LABELS` with format: `setting1=label1,setting2=label2`.
+- Use the option function `WithSettingsLabels` and provide a `map[string]string` with the setting name as key and label as value.
 
 ### Authentication
 
@@ -404,8 +468,14 @@ In addition to this it should work on:
 - Azure Virtual Machines (since it makes use of the IMDS endpoint like Azure Container Instances)
 - Azure App Services (since it makes us of the same endpoint as Azure Functions)
 
-For more advanced scenarios like Azure Stack or Service Fabric see the section about using [`authopts`](./authopts/)
-together with [`azidentity`](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity).
+**Note**: Sometimes it can take some time for the IMDS endpoint to start up on Azure Container instances, resulting
+in authentication failures. If these issues occur, either:
+
+* Set the environment variable `AZCFG_MANAGED_IDENTITY_IMDS_DIAL_TIMEOUT` with a longer [duration string](https://pkg.go.dev/time#ParseDuration), example: `5s`.
+* Use the option `WithManagedIdentityIMDSDialTimeout` with a `time.Duration`.
+
+For more advanced scenarios for managed identities like Azure Stack or Service Fabric see the section
+about using [`authopts`](./authopts/) together with [`azidentity`](https://pkg.go.dev/github.com/Azure/azure-sdk-for-go/sdk/azidentity).
 
 ##### Authentication with environment variables
 

internal/identity/azure_cli_credential.go

@@ -58,7 +58,7 @@ func (c *AzureCLICredential) Token(ctx context.Context, options ...auth.TokenOpt
 	return *c.tokens[opts.Scope], nil
 }
 
-// cliToken retreives a token from the Azure CLI.
+// cliToken retrieves a token from the Azure CLI.
 var cliToken = func(scope string) (auth.Token, error) {
 	var command, flag, dir string
 	if runtime.GOOS == "windows" {

internal/secret/options.go

@@ -5,16 +5,24 @@ import (
 
 	"github.com/KarlGW/azcfg/azure/cloud"
 	"github.com/KarlGW/azcfg/internal/httpr"
+	"github.com/KarlGW/azcfg/internal/request"
 )
 
+// WithHTTPClient sets the HTTP client for secret retrieval.
+func WithHTTPClient(client request.Client) ClientOption {
+	return func(c *Client) {
+		c.c = client
+	}
+}
+
 // WithConcurrency sets the concurrency for secret retrieval.
 func WithConcurrency(n int) ClientOption {
 	return func(c *Client) {
 		c.concurrency = n
 	}
 }
 
-// WithTimeout sets timeout for secret retreival.
+// WithTimeout sets timeout for secret retrieval.
 func WithTimeout(d time.Duration) ClientOption {
 	return func(c *Client) {
 		c.timeout = d
@@ -38,3 +46,21 @@ func WithCloud(c cloud.Cloud) ClientOption {
 		c.cloud = cl
 	}
 }
+
+// WithClientVersions sets secret versions on the secret client based on the provided
+// map. The key of the map should be the secret name, and the value
+// should be the version.
+func WithClientVersions(versions map[string]string) ClientOption {
+	return func(c *Client) {
+		c.versions = versions
+	}
+}
+
+// WithVersions sets secret versions on the secret requests based on the provided
+// map. The key of the map should be the secret name, and the value
+// should be the version. Overrides the versions set on the client.
+func WithVersions(versions map[string]string) Option {
+	return func(o *Options) {
+		o.Versions = versions
+	}
+}

internal/secret/secret.go

@@ -46,6 +46,7 @@ type Client struct {
 	baseURL     string
 	vault       string
 	userAgent   string
+	versions    map[string]string
 	retryPolicy httpr.RetryPolicy
 	concurrency int
 	timeout     time.Duration
@@ -92,7 +93,9 @@ func NewClient(vault string, cred auth.Credential, options ...ClientOption) (*Cl
 }
 
 // Options for client operations.
-type Options struct{}
+type Options struct {
+	Versions map[string]string
+}
 
 // Option is a function that sets options for client operations.
 type Option func(o *Options)
@@ -104,11 +107,20 @@ func (c Client) GetSecrets(ctx context.Context, names []string, options ...Optio
 
 // Get a secret.
 func (c Client) Get(ctx context.Context, name string, options ...Option) (Secret, error) {
-	opts := Options{}
+	opts := Options{
+		Versions: c.versions,
+	}
 	for _, option := range options {
 		option(&opts)
 	}
 
+	if len(opts.Versions) > 0 {
+		version, ok := opts.Versions[name]
+		if ok {
+			name = name + "/" + version
+		}
+	}
+
 	u := fmt.Sprintf("%s/%s?api-version=%s", c.baseURL, name, apiVersion)
 	token, err := c.cred.Token(ctx, auth.WithScope(c.scope))
 	if err != nil {
@@ -155,7 +167,7 @@ func (c *Client) SetVault(vault string) {
 	c.vault = vault
 }
 
-// secretResults contains results from retreiving secrets. Should
+// secretResults contains results from retrieving secrets. Should
 // be used with a channel for handling results and errors.
 type secretResult struct {
 	secret Secret

internal/secret/secret_test.go

@@ -6,7 +6,7 @@ import (
 	"errors"
 	"io"
 	"net/http"
-	"path"
+	"strings"
 	"testing"
 	"time"
 
@@ -71,6 +71,7 @@ func TestClient_GetSecrets(t *testing.T) {
 		name  string
 		input struct {
 			names   []string
+			options []Option
 			bodies  map[string][]byte
 			timeout time.Duration
 			err     error
@@ -82,6 +83,7 @@ func TestClient_GetSecrets(t *testing.T) {
 			name: "get secrets",
 			input: struct {
 				names   []string
+				options []Option
 				bodies  map[string][]byte
 				timeout time.Duration
 				err     error
@@ -101,10 +103,42 @@ func TestClient_GetSecrets(t *testing.T) {
 			},
 			wantErr: nil,
 		},
+		{
+			name: "get secrets - with versions",
+			input: struct {
+				names   []string
+				options []Option
+				bodies  map[string][]byte
+				timeout time.Duration
+				err     error
+			}{
+				names: []string{"secret-a", "secret-b", "secret-c"},
+				options: []Option{
+					WithVersions(map[string]string{
+						"secret-a": "version-a",
+						"secret-b": "version-b",
+						"secret-c": "version-c",
+					}),
+				},
+				bodies: map[string][]byte{
+					"secret-a/version-a": []byte(`{"value":"a"}`),
+					"secret-b/version-b": []byte(`{"value":"b"}`),
+					"secret-c/version-c": []byte(`{"value":"c"}`),
+				},
+				timeout: 100 * time.Millisecond,
+			},
+			want: map[string]Secret{
+				"secret-a": {Value: "a"},
+				"secret-b": {Value: "b"},
+				"secret-c": {Value: "c"},
+			},
+			wantErr: nil,
+		},
 		{
 			name: "secret not found",
 			input: struct {
 				names   []string
+				options []Option
 				bodies  map[string][]byte
 				timeout time.Duration
 				err     error
@@ -127,6 +161,7 @@ func TestClient_GetSecrets(t *testing.T) {
 			name: "get secrets - context deadline exceeded",
 			input: struct {
 				names   []string
+				options []Option
 				bodies  map[string][]byte
 				timeout time.Duration
 				err     error
@@ -145,6 +180,7 @@ func TestClient_GetSecrets(t *testing.T) {
 			name: "server error",
 			input: struct {
 				names   []string
+				options []Option
 				bodies  map[string][]byte
 				timeout time.Duration
 				err     error
@@ -168,6 +204,7 @@ func TestClient_GetSecrets(t *testing.T) {
 			name: "request error",
 			input: struct {
 				names   []string
+				options []Option
 				bodies  map[string][]byte
 				timeout time.Duration
 				err     error
@@ -194,7 +231,7 @@ func TestClient_GetSecrets(t *testing.T) {
 			ctx, cancel := context.WithTimeout(context.Background(), test.input.timeout)
 			defer cancel()
 
-			got, gotErr := client.GetSecrets(ctx, test.input.names)
+			got, gotErr := client.GetSecrets(ctx, test.input.names, test.input.options...)
 
 			if diff := cmp.Diff(test.want, got); diff != "" {
 				t.Errorf("GetSecrets() = unexpected result (-want +got)\n%s\n", diff)
@@ -391,7 +428,13 @@ func (c mockHttpClient) Do(req *http.Request) (*http.Response, error) {
 		return nil, c.err
 	}
 
-	name := path.Base(req.URL.Path)
+	parts := strings.Split(req.URL.Path, "/")
+	var name string
+	if len(parts) > 3 {
+		name = strings.Join(parts[2:], "/")
+	} else {
+		name = parts[2]
+	}
 	b, ok := c.bodies[name]
 	if !ok {
 		return &http.Response{