Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Load secrets via ESO for eks build clusters #4943

Closed
wants to merge 1 commit into from
Closed

Load secrets via ESO for eks build clusters #4943

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
186 changes: 111 additions & 75 deletions infra/aws/terraform/prow-build-cluster/resources/test-pods/externalsecrets.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,78 +4,114 @@
# NOTE !!!
# THIS CLUSTER USES EXTERNAL SECRETS OPERATOR INSTEAD OF KUBERNETES EXTERNAL SECRETS.
# The CRDs are different from other prow clusters. Sample guide that explains the diff https://wiki.cac.washington.edu/display/MCI/Transition+from+KES+to+ESO
# ---
# apiVersion: kubernetes-client.io/v1
# kind: ExternalSecret
# metadata:
# name: service-account # The name of the Kubernetes Secret
# namespace: test-pods
# spec:
# backendType: gcpSecretsManager
# projectId: k8s-infra-prow-build
# data:
# - key: prow-build-service-account # The name of the GSM Secret
# name: service-account.json # The key to write to in the Kubernetes Secret
# version: latest # The version of the GSM Secret
# ---
# apiVersion: kubernetes-client.io/v1
# kind: ExternalSecret
# metadata:
# name: ssh-key-secret # The name of the Kubernetes Secret
# namespace: test-pods
# spec:
# backendType: gcpSecretsManager
# projectId: k8s-infra-prow-build
# data:
# - key: prow-build-ssh-key-secret-ssh-public # The name of the GSM Secret
# name: ssh-public # The key to write to in the Kubernetes Secret
# version: latest # The version of the GSM Secret
# - key: prow-build-ssh-key-secret-ssh-private # The name of the GSM Secret
# name: ssh-private # The key to write to in the Kubernetes Secret
# version: latest # The version of the GSM Secret
# ---
# apiVersion: kubernetes-client.io/v1
# kind: ExternalSecret
# metadata:
# name: aws-credentials-768319786644 # The name of the Kubernetes Secret
# namespace: test-pods
# spec:
# backendType: gcpSecretsManager
# projectId: k8s-infra-prow-build
# data:
# - key: gke_k8s-prow-builds_us-central1-f_prow__test-pods__aws-credentials-768319786644 # The name of the GSM Secret
# name: credentials # The key to write to in the Kubernetes Secret
# version: latest # The version of the GSM Secret
# property: credentials
# ---
# apiVersion: kubernetes-client.io/v1
# kind: ExternalSecret
# metadata:
# name: aws-credentials-607362164682 # The name of the Kubernetes Secret
# namespace: test-pods
# spec:
# backendType: gcpSecretsManager
# projectId: k8s-infra-prow-build
# data:
# - key: gke_k8s-prow-builds_us-central1-f_prow__test-pods__aws-credentials-607362164682 # The name of the GSM Secret
# name: credentials # The key to write to in the Kubernetes Secret
# version: latest # The version of the GSM Secret
# property: credentials
# ---
# apiVersion: kubernetes-client.io/v1
# kind: ExternalSecret
# metadata:
# name: aws-ssh-key-secret # The name of the Kubernetes Secret
# namespace: test-pods
# spec:
# backendType: gcpSecretsManager
# projectId: k8s-infra-prow-build
# data:
# - key: gke_k8s-prow-builds_us-central1-f_prow__test-pods__aws-ssh-key-secret # The name of the GSM Secret
# name: aws-ssh-private # The key to write to in the Kubernetes Secret
# version: latest # The version of the GSM Secret
# property: aws-ssh-private
# - key: gke_k8s-prow-builds_us-central1-f_prow__test-pods__aws-ssh-key-secret # The name of the GSM Secret
# name: aws-ssh-public # The key to write to in the Kubernetes Secret
# version: latest # The version of the GSM Secret
# property: aws-ssh-public
---
apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only want to sync secrets in a specific namespace. we should scope this to the namespace.

metadata:
name: k8s-infra-prow-build
spec:
provider:
gcpsm:
projectID: k8s-infra-prow-build
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: service-account
namespace: test-pods
spec:
secretStoreRef:
kind: ClusterSecretStore
name: k8s-infra-prow-build
target:
name: service-account # The name of the Kubernetes Secret
creationPolicy: Owner
data:
- secretKey: service-account.json # The key to write to in the Kubernetes Secret
remoteRef:
key: prow-build-service-account # The name of the GSM Secret
version: latest # The version of the GSM Secret
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: ssh-key-secret
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need this SSH key. This is needed for e2e tests on GCP. we may need to create a new key pair on EC2.

namespace: test-pods
spec:
secretStoreRef:
kind: ClusterSecretStore
name: k8s-infra-prow-build
target:
name: ssh-key-secret # The name of the Kubernetes Secret
creationPolicy: Owner
data:
- secretKey: ssh-public # The key to write to in the Kubernetes Secret
remoteRef:
key: prow-build-ssh-key-secret-ssh-public # The name of the GSM Secret
version: latest # The version of the GSM Secret
- secretKey: ssh-private # The key to write to in the Kubernetes Secret
remoteRef:
key: prow-build-ssh-key-secret-ssh-private # The name of the GSM Secret
version: latest # The version of the GSM Secret
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: aws-credentials-768319786644
namespace: test-pods
spec:
secretStoreRef:
kind: ClusterSecretStore
name: k8s-infra-prow-build
target:
name: aws-credentials-768319786644 # The name of the Kubernetes Secret
creationPolicy: Owner
data:
- secretKey: credentials # The key to write to in the Kubernetes Secret
remoteRef:
key: gke_k8s-prow-builds_us-central1-f_prow__test-pods__aws-credentials-768319786644 # The name of the GSM Secret
version: latest # The version of the GSM Secret
property: credentials # The version of the GSM Secret
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: aws-credentials-607362164682
namespace: test-pods
spec:
secretStoreRef:
kind: ClusterSecretStore
name: k8s-infra-prow-build
target:
name: aws-credentials-607362164682 # The name of the Kubernetes Secret
creationPolicy: Owner
data:
- secretKey: credentials # The key to write to in the Kubernetes Secret
remoteRef:
key: gke_k8s-prow-builds_us-central1-f_prow__test-pods__aws-credentials-607362164682 # The name of the GSM Secret
version: latest # The version of the GSM Secret
property: credentials # The version of the GSM Secret
---
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: aws-ssh-key-secret
namespace: test-pods
spec:
secretStoreRef:
kind: ClusterSecretStore
name: k8s-infra-prow-build
target:
name: aws-ssh-key-secret # The name of the Kubernetes Secret
creationPolicy: Owner
data:
- secretKey: aws-ssh-private # The key to write to in the Kubernetes Secret
remoteRef:
key: gke_k8s-prow-builds_us-central1-f_prow__test-pods__aws-ssh-key-secret # The name of the GSM Secret
version: latest # The version of the GSM Secret
property: aws-ssh-private
- secretKey: aws-ssh-public # The key to write to in the Kubernetes Secret
remoteRef:
key: gke_k8s-prow-builds_us-central1-f_prow__test-pods__aws-ssh-key-secret # The name of the GSM Secret
version: latest # The version of the GSM Secret
property: aws-ssh-public
2 changes: 1 addition & 1 deletion ...a/aws/terraform/prow-build-cluster/resources/test-pods/test-pods-poddisruptionbudget.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
#
# It is still possible to delete the pods via a normal delete call. See https://kubernetes.io/docs/tasks/administer-cluster/safely-drain-node/#the-eviction-api

apiVersion: policy/v1beta1
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: prow-pods
Expand Down