[Blueprints, 02-at-scale]: Use Secrets to pass terraform variables to Casc (part I)

.gitignore

@@ -42,10 +42,6 @@ terraform.rc
 terraform*.output
 kubeconfig*.yaml
 
-# Secrets file
-*secrets*
-!secrets*example
-
 # Bankend file
 backend*.tf
 

README.md

@@ -95,7 +95,7 @@ The CloudBees CI add-on uses `helms release` for its resources definition, makin
 | cert_arn | AWS Certificate Manager (ACM) certificate for Amazon Resource Names (ARN). | `string` | n/a | yes |
 | hosted_zone | Amazon Route 53 hosted zone name. | `string` | n/a | yes |
 | trial_license | CloudBees CI trial license details for evaluation. | `map(string)` | n/a | yes |
-| create_k8s_secrets | Create the Kubernetes secret cbci-secrets. It can be consumed by CloudBees CasC. | `bool` | `false` | no |
+| create_k8s_secrets | Create the Kubernetes secret cbci-secrets. It can be consumed by CloudBees CasC at Operation Center level. | `bool` | `false` | no |
 | helm_config | CloudBees CI Helm chart configuration. | `any` | <pre>{<br>  "values": [<br>    ""<br>  ]<br>}</pre> | no |
 | k8s_secrets | Secrets file .yml as a string containing the secrets names:values. It is required when create_k8s_secrets is enabled. | `string` | `"secrets-values.yml"` | no |
 | prometheus_target | Creates a service monitor to discover the CloudBees CI Prometheus target dynamically. It is designed to be enabled with the AWS EKS Terraform Addon Kube Prometheus Stack. | `bool` | `false` | no |

blueprints/02-at-scale/.auto.tfvars.example

@@ -1,12 +1,15 @@
-hosted_zone = "acme.domain.com" # Required. Route 53 Hosted Zone to host CloudBees CI subdomains records.
+hosted_zone = "acme.domain.com" # Required. Route 53 Hosted Zone to host CloudBees CI subdomains records. Replace Default values by your own.
 
-trial_license = { # Required. CloudBees CI Trial license details for evaluation.
+trial_license = { # Required. CloudBees CI Trial license details for evaluation. Replace Default values by your own.
   first_name  = "Foo"
   last_name  = "Bar"
   email = "foo.bar@acme.com"
   company = "Acme Inc."
 }
 
+gh_user = "exampleUser" # Required. Default values can be used for demo.
+gh_token = "ExampleToken1234" # Required. Default values can be used for demo.
+
 # tags = {  # Optional. Tags for the resources created. Default set to empty. Shared among all.
 #   "cb-owner" : "team-services"
 #   "cb-user" : "demo-user"

blueprints/02-at-scale/README.md

@@ -58,7 +58,8 @@ Once you have familiarized yourself with [CloudBees CI blueprint add-on: Get sta
 |------|-------------|------|---------|:--------:|
 | hosted_zone | Amazon Route 53 hosted zone. CloudBees CI applications are configured to use subdomains in this hosted zone. | `string` | n/a | yes |
 | trial_license | CloudBees CI trial license details for evaluation. | `map(string)` | n/a | yes |
-| secret_file | Relative location of the Secret file to be converted into a Kubernetes Secret. secrets-values.yml is not stored in the repository but it is generated as copy from secrets-values.yml.example. | `string` | `"k8s/secrets-values.yml"` | no |
+| gh_token | GitHub Token for CloudBees Operation Center credential GH-User-token that is created via Casc. | `string` | `"ExampleToken1234"` | no |
+| gh_user | GitHub User for CloudBees Operation Center credential GH-User-token that is created via Casc. | `string` | `"exampleUser"` | no |
 | suffix | Unique suffix to assign to all resources. When adding the suffix, changes are required in CloudBees CI for the validation phase. | `string` | `""` | no |
 | tags | Tags to apply to resources. | `map(string)` | `{}` | no |
 
@@ -109,47 +110,6 @@ In addition to the minimum required settings explained in [Get started - Deploy]
 > [!TIP]
 > The `deploy` phase can be orchestrated via the companion [Makefile](../../Makefile).
 
-### Create the secrets file
-
-You must create your secrets file by copying the contents of [secrets-values.yml.example](k8s/secrets-values.yml.example) to `secrets-values.yml`. The content of this file will be served as [Kubernetes secrets](https://github.com/jenkinsci/configuration-as-code-plugin/blob/master/docs/features/secrets.adoc#kubernetes-secrets) that can be consumed by CasC.
-
-> [!IMPORTANT]
-> Do not update parametrized values in the `secrets-values.yml` file. These values are automatically replaced by Terraform during the deployment phase (e.g. `sec_ldapPassword: ${ldap_password}` ).
-
-### Update Amazon S3 bucket settings
-
-Since the optional Terraform variable `suffix` is used for this blueprint, you must update the Amazon S3 bucket name for CloudBees CI controllers and the Amazon S3 bucket for the backup controller cluster operations. To update the Amazon S3 bucket name, you have the following options:
-
- - [Option 1: Update Amazon S3 bucket name using CasC](#option-1-update-amazon-s3-bucket-name-using-casc)
- - [Option 2: Update Amazon S3 bucket name using the CloudBees CI UI](#option-2-update-amazon-s3-bucket-name-using-the-cloudbees-ci-ui)
-
-#### Option 1: Update Amazon S3 bucket name using CasC
-
->[!IMPORTANT]
-> This option can only be used before the blueprint has been deployed.
-
-1. Create a fork of the [cloudbees/terraform-aws-cloudbees-ci-eks-addon](https://github.com/cloudbees/casc-cloudbees-ci-eks-addon) GitHub repository to your GitHub organization and make any necessary edits to the controller CasC bundle.
-   - Update `cbci_s3` in the [casc/mc/parent/variables/variables.yaml](casc/mc/parent/variables/variables.yaml) file, including your custom prefix.
-   - Update `scm_casc_mc_store` in the [casc/oc/variables/variables.yaml](casc/oc/variables/variables.yaml) file and `bucketName` in the [casc/oc/items/items-admin-jobs-folder.yaml](casc/oc/items/items-admin-jobs-folder.yaml) file.
-2. Commit and push your changes to the forked repository in your organization.
-3. In the [k8s/cbci-values.yml](k8s/cbci-values.yml) Helm file, update the `OperationsCenter.CasC.Retriever.scmRepo` field based on the files in this blueprint.
-4. Save the file and issue the `terraform apply` command.
-
-#### Option 2: Update Amazon S3 bucket name using the CloudBees CI UI
-
-> [!IMPORTANT]
-> - This option can only be used after the blueprint is deployed.
-> - If using CasC, the declarative definition overrides any configuration updates that are made in the UI the next time the controller is restarted.
-
-1. Sign in to the CloudBees CI controller UI as a user with **Administer** privileges.
-2. Navigate to **Manage Jenkins > AWS > Amazon S3 Bucket Access settings**, update the **S3 Bucket Name**, and select **Save**.
-3. Sign in to the CloudBees CI operations center UI as a user with **Administer** privileges.
-   Note that access to back up jobs is restricted to admin users via role-based access control (RBAC).
-4. From the operations center dashboard, select **All** to view all folders on the operations center.  
-5. Navigate to the **admin** folder, and then select the **backup-all-controllers** Cluster Operations job.
-6. From the left pane, select **Configure**.
-7. Update the **S3 Bucket Name**, and then select **Save**.
-
 ## Validate
 
 Once the blueprint has been deployed, you can validate it.

blueprints/02-at-scale/casc/mc/parent/jcasc/main.yaml

@@ -30,7 +30,7 @@ unclassified:
       endpoint: ${ot_endpoint}
 aws:
   s3:
-    container: ${cbci_s3}
+    container: "${sec_s3bucketName}"
     disableSessionToken: false
     prefix: "cbci/"
     useHttp: false

blueprints/02-at-scale/casc/mc/parent/variables/variables.yaml

@@ -1,4 +1,2 @@
 variables:
   - ot_endpoint: "http://tempo.kube-prometheus-stack.svc.cluster.local:4317"
-  # You must update the variable in case a variable suffix is provided in the .tf files (for example, cbci-bp02-<YOUR-SUFFIX>-s3).
-  - cbci_s3: cbci-bp02-s3

blueprints/02-at-scale/casc/oc/items/items-admin-jobs-folder.yaml

@@ -82,9 +82,7 @@ items:
                   safeDelaySeconds: 0
                   store:
                     s3Store:
-                      #TODO: Use variables
-                      #bucketName: ${cbciS3}
-                      bucketName: "cbci-bp02-s3"
+                      bucketName: "${sec_s3bucketName}"
                       bucketFolder: cbci/backup
                       region: us-east-1
               itemSource:

blueprints/02-at-scale/casc/oc/items/items-root.yaml

@@ -26,12 +26,25 @@ items:
           kind: "StatefulSet"
           spec:
             template:
-                spec:
-                  tolerations:
-                  - key: "dedicated"
-                    operator: "Equal"
-                    value: "cb-apps"
-                    effect: "NoSchedule"
+              spec:
+                tolerations:
+                - key: "dedicated"
+                  operator: "Equal"
+                  value: "cb-apps"
+                  effect: "NoSchedule"
+                volumes:
+                - name: cbci-secrets
+                  secret:
+                    secretName: cbci-secrets
+                containers:
+                - name: jenkins
+                  env:
+                  - name: SECRETS
+                    value: "/var/run/secrets/cbci"
+                  volumeMounts:
+                  - name: cbci-secrets
+                    mountPath: "/var/run/secrets/cbci"
+                    readOnly: true
           ---
           apiVersion: "v1"
           kind: "Service"
@@ -40,7 +53,7 @@ items:
               "cloudbees.prometheus": "true"
     properties:
       - configurationAsCode:
-          bundle: "ldap/none-ha"
+          bundle: "s3/none-ha"
   # Casc, HA
   - kind: managedController
     name: team-c-ha
@@ -60,12 +73,25 @@ items:
           kind: "Deployment"
           spec:
             template:
-                spec:
-                  tolerations:
-                  - key: "dedicated"
-                    operator: "Equal"
-                    value: "cb-apps"
-                    effect: "NoSchedule"
+              spec:
+                tolerations:
+                - key: "dedicated"
+                  operator: "Equal"
+                  value: "cb-apps"
+                  effect: "NoSchedule"
+                volumes:
+                - name: cbci-secrets
+                  secret:
+                    secretName: cbci-secrets
+                containers:
+                - name: jenkins
+                  env:
+                  - name: SECRETS
+                    value: "/var/run/secrets/cbci"
+                  volumeMounts:
+                  - name: cbci-secrets
+                    mountPath: "/var/run/secrets/cbci"
+                    readOnly: true
           ---
           apiVersion: "v1"
           kind: "Service"
@@ -74,4 +100,4 @@ items:
               "cloudbees.prometheus": "true"
     properties:
       - configurationAsCode:
-          bundle: "ldap/ha"
+          bundle: "s3/ha"

blueprints/02-at-scale/casc/oc/jcasc/main.yaml

@@ -1,7 +1,9 @@
 jenkins:
-  systemMessage: ${message}
+  systemMessage: "Test ${sec_s3bucketName}" #Replace by ${message}
   numExecutors: 0
 unclassified:
+  cascItemsConfiguration:
+      variableInterpolationEnabledForAdmin: true
   bundleStorageService:
     activated: true
     activeBundle:

blueprints/02-at-scale/casc/oc/variables/variables.yaml

@@ -1,10 +1,8 @@
 variables:
   - message: "Welcome to the CloudBees CI blueprint add-on: At scale!"
   - scmCascMmStore: "https://github.com/cloudbees/terraform-aws-cloudbees-ci-eks-addon.git"
-  - cascBranch: ldap
+  - cascBranch: s3
   - ldapManagerDN: "cn=admin,dc=acme,dc=org"
   - ldapRootDN: "dc=acme,dc=org"
   - ldapServer: "openldap-stack.auth.svc.cluster.local"
   - ldapUserSearch: "cn={0}"
-  #Issue #70
-  #- cbciS3: "cbci-bp02-s3"

blueprints/02-at-scale/k8s/cbci-values.yml

@@ -19,7 +19,7 @@ OperationsCenter:
     Retriever:
       Enabled: true
       scmRepo: https://github.com/cloudbees/terraform-aws-cloudbees-ci-eks-addon.git
-      scmBranch: ldap
+      scmBranch: s3
       scmBundlePath: blueprints/02-at-scale/casc/oc
       scmPollingInterval: PT20M
 Persistence:

blueprints/02-at-scale/k8s/secrets-values.yml

@@ -0,0 +1,8 @@
+# Copyright (c) CloudBees, Inc.
+
+# IMPORTANT: Secrets must be parametrized.
+
+sec_globalPassword: ${global_password}
+sec_s3bucketName: ${s3bucketName}
+sec_githubUser: ${githubUser}
+sec_githubToken: ${githubToken}

blueprints/02-at-scale/main.tf

@@ -107,8 +107,11 @@ module "eks_blueprints_addon_cbci" {
   }
 
   create_k8s_secrets = true
-  k8s_secrets = templatefile(var.secret_file, {
+  k8s_secrets = templatefile("k8s/secrets-values.yml", {
     global_password = local.global_password
+    s3bucketName    = local.bucket_name
+    githubUser      = var.gh_user
+    githubToken     = var.gh_token
   })
 
   prometheus_target = true

blueprints/02-at-scale/variables.tf

@@ -25,8 +25,14 @@ variable "suffix" {
   }
 }
 
-variable "secret_file" {
-  description = "Relative location of the Secret file to be converted into a Kubernetes Secret. secrets-values.yml is not stored in the repository but it is generated as copy from secrets-values.yml.example."
-  default     = "k8s/secrets-values.yml"
+variable "gh_user" {
+  description = "GitHub User for CloudBees Operation Center credential GH-User-token that is created via Casc."
+  default     = "exampleUser"
+  type        = string
+}
+
+variable "gh_token" {
+  description = "GitHub Token for CloudBees Operation Center credential GH-User-token that is created via Casc."
+  default     = "ExampleToken1234"
   type        = string
 }

blueprints/helpers.sh

@@ -170,7 +170,7 @@ set-kube-env () {
 set-casc-branch () {
   local branch=$1
   sed -i "s|scmBranch: .*|scmBranch: $branch|g" "$SCRIPTDIR/02-at-scale/k8s/cbci-values.yml"
-  sed -i "s|casc_branch: .*|casc_branch: $branch|g" "$SCRIPTDIR/02-at-scale/casc/oc/variables/variables.yaml"
+  sed -i "s|cascBranch: .*|cascBranch: $branch|g" "$SCRIPTDIR/02-at-scale/casc/oc/variables/variables.yaml"
   sed -i "s|bundle: \".*/none-ha\"|bundle: \"$branch/none-ha\"|g" "$SCRIPTDIR/02-at-scale/casc/oc/items/items-root.yaml"
   sed -i "s|bundle: \".*/ha\"|bundle: \"$branch/ha\"|g" "$SCRIPTDIR/02-at-scale/casc/oc/items/items-root.yaml"
 }

variables.tf

@@ -36,7 +36,7 @@ variable "trial_license" {
 }
 
 variable "create_k8s_secrets" {
-  description = "Create the Kubernetes secret cbci-secrets. It can be consumed by CloudBees CasC."
+  description = "Create the Kubernetes secret cbci-secrets. It can be consumed by CloudBees CasC at Operation Center level."
   default     = false
   type        = bool
 }