Aws secretsmanager additions (#6381)

* Add SecretKey to AWS SecretsManager TriggerAuthentication spec Signed-off-by: Nick Richardson <nicholas.richardson@akasa.com> * Update CHANGELOG Signed-off-by: Nick Richardson <nicholas.richardson@akasa.com> * Fix linting issue Signed-off-by: Nick Richardson <nicholas.richardson@akasa.com> * Update E2E tests Signed-off-by: Nick Richardson <nicholas.richardson@akasa.com> * Update E2E tests Signed-off-by: Nick Richardson <nicholas.richardson@akasa.com> * Update E2E tests Signed-off-by: Nick Richardson <nicholas.richardson@akasa.com> * fixed mixedxe tab/space issue Signed-off-by: michael pechner <mike.pechner@akasa.com> * reverted back to exportloopref Signed-off-by: michael pechner <mike.pechner@akasa.com> * remoace tab with spaces Signed-off-by: michael pechner <mike.pechner@akasa.com> * Needed the hlper added in here Signed-off-by: michael pechner <mike.pechner@akasa.com> * only changed awssecreetmanager_test.go to see if my approach is correct. Will remove REMOVETestAwsSecretManagerJSONFormat and change aws_secret_manager_pod_identity.go once I have changed this file as expected. Signed-off-by: michael pechner <mike.pechner@akasa.com> * nd keeps removing // Local imports . "github.com/kedacore/keda/v2/tests/helper" Signed-off-by: michael pechner <mike.pechner@akasa.com> * Not sure what the correct way to import the local helper is. Signed-off-by: michael pechner <mike.pechner@akasa.com> * Trying blank import. Signed-off-by: michael pechner <mike.pechner@akasa.com> * fix path "github.com/kedacore/keda/v2/tests/helper" Signed-off-by: michael pechner <mike.pechner@akasa.com> * added call to GetRandomNumber() just to keep goimport from removeing the the helper. Fixed type Ti -> T Signed-off-by: michael pechner <mike.pechner@akasa.com> * really am a noob. AwsSecretManager() does not return anything, so fixed the calling test Signed-off-by: michael pechner <mike.pechner@akasa.com> * another noob error. added test code back and making AwsSecretmanager() return nil Signed-off-by: michael pechner <mike.pechner@akasa.com> * really am gonna make every dumb mistake. Signed-off-by: michael pechner <mike.pechner@akasa.com> * fixed return value Signed-off-by: michael pechner <mike.pechner@akasa.com> * thought I fixed that. Signed-off-by: michael pechner <mike.pechner@akasa.com> * figured goland would have pointed these out Signed-off-by: michael pechner <mike.pechner@akasa.com> * removed duplicated function. Tests not run with useJSONSecretFormat true and false Signed-off-by: michael pechner <mike.pechner@akasa.com> * no excuse Signed-off-by: michael pechner <mike.pechner@akasa.com> * fixed . "github.com/kedacore/keda/v2/tests/helper" igoland linter keeps removng it. Signed-off-by: michael pechner <mike.pechner@akasa.com> * go fmt needs for 1 empty line Signed-off-by: michael pechner <mike.pechner@akasa.com> * In both tests deleteAWSSecret() is already coded to immediately delete the secret. Instead of tryinfg to further randomize the secretname, just let the code do what it should. Added a poll to wait on the secret to be removed. Hoping 2 minniutes is more than enough. Signed-off-by: michael pechner <mike.pechner@akasa.com> * In both tests deleteAWSSecret() is already coded to immediately delete the secret. Instead of trying to further randomize the secretname, just let the code do what it should. Added a poll to wait on the secret to be removed. Should happen within a few seconds. But we are talking AWS. 5 minutes really should be more then enough. Signed-off-by: michael pechner <mike.pechner@akasa.com> * forgot the import Signed-off-by: michael pechner <mike.pechner@akasa.com> * format Signed-off-by: michael pechner <mike.pechner@akasa.com> * Sorry. did not realize the needed imcludes for the wait code. Signed-off-by: michael pechner <mike.pechner@akasa.com> * compiles clean. found types.ResourceNotFoundException Signed-off-by: michael pechner <mike.pechner@akasa.com> * swapped wait.PollImmediate for wait.PollUntilContextTimeout Signed-off-by: michael pechner <mike.pechner@akasa.com> * forgot to fmt 1 Signed-off-by: michael pechner <mike.pechner@akasa.com> * as requested. removed wait on secret delete. made sure we set a new secret name for each run Signed-off-by: michael pechner <mike.pechner@akasa.com> * removed useJSONSecretFormat from test struct. Signed-off-by: michael pechner <mike.pechner@akasa.com> * missed a spot where useJSONSecretFormat was set Signed-off-by: michael pechner <mike.pechner@akasa.com> --------- Signed-off-by: Nick Richardson <nicholas.richardson@akasa.com> Signed-off-by: michael pechner <mike.pechner@akasa.com> Signed-off-by: Michael D Pechner - Akasa <mike.pechner@akasa.com> Co-authored-by: Nick Richardson <nicholas.richardson@akasa.com> Co-authored-by: Nick Richardson <89413781+nrichardson-akasa@users.noreply.github.com> Co-authored-by: Zbynek Roubalik <zroubalik@gmail.com>
kedacore · Feb 6, 2025 · ca83efd · ca83efd
1 parent c036742
commit ca83efd
Show file tree

Hide file tree

Showing 8 changed files with 214 additions and 20 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -73,9 +73,12 @@ Here is an overview of all new **experimental** features:
 
 ### Improvements
 
+- General: Add SecretKey to AWS SecretsManager TriggerAuthentication to allow parsing JSON / Key/Value Pairs in secrets (#5940)
+- TODO ([#XXX](https://github.com/kedacore/keda/issues/XXX))
 - **IBMMQ Scaler**: Handling StatusNotFound in IBMMQ scaler ([#6472](https://github.com/kedacore/keda/pull/6472))
 - **RabbitMQ Scaler**: Support use of the ‘vhostName’ parameter in the ‘TriggerAuthentication’ resource ([#6369](https://github.com/kedacore/keda/issues/6369))
 
+
 ### Fixes
 
 - **General**: Centralize and improve automaxprocs configuration with proper structured logging ([#5970](https://github.com/kedacore/keda/issues/5970))

diff --git a/apis/keda/v1alpha1/triggerauthentication_types.go b/apis/keda/v1alpha1/triggerauthentication_types.go
@@ -376,6 +376,8 @@ type AwsSecretManagerSecret struct {
 	VersionID string `json:"versionId,omitempty"`
 	// +optional
 	VersionStage string `json:"versionStage,omitempty"`
+	// +optional
+	SecretKey string `json:"secretKey,omitempty"`
 }
 
 func init() {

diff --git a/config/crd/bases/keda.sh_clustertriggerauthentications.yaml b/config/crd/bases/keda.sh_clustertriggerauthentications.yaml
@@ -183,6 +183,8 @@ spec:
                           type: string
                         parameter:
                           type: string
+                        secretKey:
+                          type: string
                         versionId:
                           type: string
                         versionStage:

diff --git a/config/crd/bases/keda.sh_triggerauthentications.yaml b/config/crd/bases/keda.sh_triggerauthentications.yaml
@@ -182,6 +182,8 @@ spec:
                           type: string
                         parameter:
                           type: string
+                        secretKey:
+                          type: string
                         versionId:
                           type: string
                         versionStage:

diff --git a/pkg/scaling/resolver/aws_secretmanager_handler.go b/pkg/scaling/resolver/aws_secretmanager_handler.go
@@ -18,6 +18,7 @@ package resolver
 
 import (
 	"context"
+	"encoding/json"
 	"fmt"
 
 	"github.com/aws/aws-sdk-go-v2/aws"
@@ -43,9 +44,9 @@ func NewAwsSecretManagerHandler(a *kedav1alpha1.AwsSecretManager) *AwsSecretMana
 	}
 }
 
-// Read fetches the secret value from AWS Secret Manager using the provided secret name, version ID(optional), and version stage(optional).
+// Read fetches the secret value from AWS Secret Manager using the provided secret name, version ID(optional), version stage(optional), and secretKey(optional).
 // It returns the secret value as a string.
-func (ash *AwsSecretManagerHandler) Read(ctx context.Context, logger logr.Logger, secretName, versionID, versionStage string) (string, error) {
+func (ash *AwsSecretManagerHandler) Read(ctx context.Context, logger logr.Logger, secretName, versionID, versionStage string, secretKey string) (string, error) {
 	input := &secretsmanager.GetSecretValueInput{
 		SecretId: aws.String(secretName),
 	}
@@ -60,6 +61,27 @@ func (ash *AwsSecretManagerHandler) Read(ctx context.Context, logger logr.Logger
 		logger.Error(err, "Error getting credentials")
 		return "", err
 	}
+	if secretKey != "" {
+		// Parse the secret string as JSON
+		var secretMap map[string]interface{}
+		err = json.Unmarshal([]byte(*result.SecretString), &secretMap)
+		if err != nil {
+			logger.Error(err, "Error parsing secret string as JSON")
+			return "", err
+		}
+
+		// Check if the specified secret key exists
+		if val, ok := secretMap[secretKey]; ok {
+			// Convert the value to a string and return it
+			if strVal, isString := val.(string); isString {
+				return strVal, nil
+			}
+			logger.Error(nil, "SecretKey value is not a string")
+			return "", fmt.Errorf("SecretKey value is not a string")
+		}
+		logger.Error(nil, "SecretKey Not Found")
+		return "", fmt.Errorf("SecretKey Not Found")
+	}
 	return *result.SecretString, nil
 }
 

diff --git a/pkg/scaling/resolver/scale_resolvers.go b/pkg/scaling/resolver/scale_resolvers.go
@@ -333,10 +333,10 @@ func resolveAuthRef(ctx context.Context, client client.Client, logger logr.Logge
 					logger.Error(err, "error authenticating to Aws Secret Manager", "triggerAuthRef.Name", triggerAuthRef.Name)
 				} else {
 					for _, secret := range triggerAuthSpec.AwsSecretManager.Secrets {
-						res, err := awsSecretManagerHandler.Read(ctx, logger, secret.Name, secret.VersionID, secret.VersionStage)
+						res, err := awsSecretManagerHandler.Read(ctx, logger, secret.Name, secret.VersionID, secret.VersionStage, secret.SecretKey)
 						if err != nil {
 							logger.Error(err, "error trying to read secret from Aws Secret Manager", "triggerAuthRef.Name", triggerAuthRef.Name,
-								"secret.Name", secret.Name, "secret.Version", secret.VersionID, "secret.VersionStage", secret.VersionStage)
+								"secret.Name", secret.Name, "secret.Version", secret.VersionID, "secret.VersionStage", secret.VersionStage, "secret.SecretKey", secret.SecretKey)
 						} else {
 							result[secret.Parameter] = res
 						}

diff --git a/tests/secret-providers/aws_secretmanager/aws_secretmanager_test.go b/tests/secret-providers/aws_secretmanager/aws_secretmanager_test.go
@@ -4,12 +4,15 @@
 package aws_secret_manager_test
 
 import (
+	// Standard imports
 	"context"
 	"encoding/base64"
+	"encoding/json"
 	"fmt"
 	"os"
 	"testing"
 
+	// Third-party imports
 	"github.com/aws/aws-sdk-go-v2/aws"
 	"github.com/aws/aws-sdk-go-v2/config"
 	"github.com/aws/aws-sdk-go-v2/credentials"
@@ -25,6 +28,9 @@ import (
 // Load environment variables from .env file
 var _ = godotenv.Load("../../.env")
 
+// makes sure helper is not removed
+var _ = GetRandomNumber()
+
 const (
 	testName = "aws-secret-manager-test"
 )
@@ -151,6 +157,31 @@ spec:
       name: {{.SecretManagerSecretName}}
 `
 
+	triggerAuthenticationSecretKeyTemplate = `apiVersion: keda.sh/v1alpha1
+kind: TriggerAuthentication
+metadata:
+  name: {{.TriggerAuthenticationName}}
+  namespace: {{.TestNamespace}}
+spec:
+  awsSecretManager:
+    credentials:
+      accessKey:
+        valueFrom:
+          secretKeyRef:
+            name: {{.AwsCredentialsSecretName}}
+            key: AWS_ACCESS_KEY_ID
+      accessSecretKey:
+        valueFrom:
+          secretKeyRef:
+            name: {{.AwsCredentialsSecretName}}
+            key: AWS_SECRET_ACCESS_KEY
+    region: {{.AwsRegion}}
+    secrets:
+    - parameter: connection
+      name: {{.SecretManagerSecretName}}
+      secretKey: connectionString
+`
+
 	scaledObjectTemplate = `apiVersion: keda.sh/v1alpha1
 kind: ScaledObject
 metadata:
@@ -257,11 +288,39 @@ spec:
 )
 
 func TestAwsSecretManager(t *testing.T) {
+	// Run the test twice with two different flag values
+	flags := []bool{true, false}
+
+	for _, useJSONSecretFormat := range flags {
+		// Define a subtest for each flag value
+
+		t.Run(getTestNameForFlag(useJSONSecretFormat), func(t *testing.T) {
+			err := AwsSecretManager(t, useJSONSecretFormat)
+			if err != nil {
+				t.Errorf("AwsSecretManager(%v) failed: %v", getTestNameForFlag(useJSONSecretFormat), err)
+			}
+		})
+	}
+}
+
+// Helper to get dynamic test names based on the flag
+func getTestNameForFlag(flag bool) string {
+	if flag {
+		return "WithFlagTrue"
+	}
+	return "WithFlagFalse"
+}
+
+func AwsSecretManager(t *testing.T, useJSONSecretFormat bool) error {
 	require.NotEmpty(t, awsAccessKeyID, "TF_AWS_ACCESS_KEY env variable is required for AWS Secret Manager test")
 	require.NotEmpty(t, awsSecretAccessKey, "TF_AWS_SECRET_KEY env variable is required for AWS Secret Manager test")
 
-	// Create the secret in GCP
-	err := createAWSSecret(t)
+	// Resetting here since we need a unique value before each time this test function is called
+	secretManagerSecretName = fmt.Sprintf("connectionString-%d", GetRandomNumber())
+	data.SecretManagerSecretName = secretManagerSecretName
+
+	// Create the secret in AWS
+	err := createAWSSecret(t, useJSONSecretFormat)
 	assert.NoErrorf(t, err, "cannot create AWS Secret Manager secret - %s", err)
 
 	// Create kubernetes resources for PostgreSQL server
@@ -280,7 +339,7 @@ func TestAwsSecretManager(t *testing.T) {
 	assert.True(t, ok, "executing a command on PostreSQL Pod should work; Output: %s, ErrorOutput: %s, Error: %s", out, errOut, err)
 
 	// Create kubernetes resources for testing
-	data, templates := getTemplateData()
+	data, templates := getTemplateData(useJSONSecretFormat)
 
 	KubectlApplyMultipleWithTemplate(t, data, templates)
 	assert.True(t, WaitForDeploymentReplicaReadyCount(t, kc, deploymentName, testNamespace, minReplicaCount, 60, 3),
@@ -292,9 +351,10 @@ func TestAwsSecretManager(t *testing.T) {
 	KubectlDeleteMultipleWithTemplate(t, data, templates)
 	DeleteKubernetesResources(t, testNamespace, data, postgreSQLtemplates)
 
-	// Delete the secret in GCP
+	// Delete the secret in AWS
 	err = deleteAWSSecret(t)
 	assert.NoErrorf(t, err, "cannot delete AWS Secret Manager secret - %s", err)
+	return nil
 }
 
 var data = templateData{
@@ -324,12 +384,19 @@ func getPostgreSQLTemplateData() (templateData, []Template) {
 	}
 }
 
-func getTemplateData() (templateData, []Template) {
+func getTemplateData(useJSONFormat bool) (templateData, []Template) {
+	var triggerConfig string
+	if useJSONFormat {
+		triggerConfig = triggerAuthenticationSecretKeyTemplate
+	} else {
+		triggerConfig = triggerAuthenticationTemplate
+	}
+
 	return data, []Template{
 		{Name: "secretTemplate", Config: secretTemplate},
 		{Name: "awsCredentialsSecretTemplate", Config: awsCredentialsSecretTemplate},
 		{Name: "deploymentTemplate", Config: deploymentTemplate},
-		{Name: "triggerAuthenticationTemplate", Config: triggerAuthenticationTemplate},
+		{Name: "triggerAuthenticationTemplate", Config: triggerConfig},
 		{Name: "scaledObjectTemplate", Config: scaledObjectTemplate},
 	}
 }
@@ -342,7 +409,7 @@ func testScaleOut(t *testing.T, kc *kubernetes.Clientset, data templateData) {
 		"replica count should be %d after 3 minutes", maxReplicaCount)
 }
 
-func createAWSSecret(t *testing.T) error {
+func createAWSSecret(t *testing.T, useJSONFormat bool) error {
 	ctx := context.Background()
 
 	// Create AWS configuration
@@ -360,7 +427,22 @@ func createAWSSecret(t *testing.T) error {
 	client := secretsmanager.NewFromConfig(cfg)
 
 	// Create the secret value
-	secretString := postgreSQLConnectionString
+	var secretString string
+	if useJSONFormat {
+		secretObject := map[string]string{
+			"connectionString": postgreSQLConnectionString,
+		}
+		// Convert the map to a JSON string
+		jsonData, err := json.Marshal(secretObject)
+		if err != nil {
+			return fmt.Errorf("Error converting to JSON: %w", err)
+		}
+
+		// Print the JSON string
+		secretString = string(jsonData)
+	} else {
+		secretString = postgreSQLConnectionString
+	}
 	_, err = client.CreateSecret(ctx, &secretsmanager.CreateSecretInput{
 		Name:         &secretManagerSecretName,
 		SecretString: &secretString,