add valkey instructions (#117)

* add valkey connection string to controller

* add agents guide

* add instructions for connecting to agents cluster

* add instructions for connecting to agents cluster

* add instructions for connecting to agents cluster

* add instructions for connecting to agents cluster

* add instructions for connecting to agents cluster

AGENTS-GUIDE.md

@@ -0,0 +1,70 @@
+# Getting started with Nx Agents
+
+We recommend deploying Nx Agents onto a new cluster, but you can also deploy onto your existing NxCloud cluster under a different namespace.
+
+### Install Valkey
+
+Valkey is an in-memory key-value store (like Redis) that is used by the workflow controller to hold temporary state.
+
+1. Create a secret similar to `agents-guide/agents-secrets.yml` and set your valkey password in there.
+   - Important: the key `valkey-password` itself shouldn't be changed, only its value 
+   - You do not need to set the secret values for the S3 bucket yet. They are there as an example. Please refer to `charts/nx-agents/values.yaml` for info on what Agent storage options we support.
+2. Apply the secret: `kubectl apply -f agents-secrets.yml`
+3. Now let's deploy Valkey:
+    ```bash
+    helm repo add bitnami https://charts.bitnami.com/bitnami
+    helm install valkey bitnami/valkey --set auth.usePassword=true --set auth.existingSecret=nx-cloud-agents-secret
+    ```
+
+### Deploy the Agents chart onto your cluster
+
+```bash
+helm repo add nx-cloud https://nrwl.github.io/nx-cloud-helm
+helm repo update nx-cloud
+helm upgrade --install nx-agents nx-cloud/nx-agents --values=nx-agents.yml
+```
+
+Use the values file in `charts/nx-agents/values.yaml` as an example.
+
+Note on storage: 
+1. The Agents need a storage bucket for storing logs as well as cached folders (such as `node_modules`)
+2. You do not need to use S3, we also support Azure Blob Storage and GCloud buckets
+
+### Connect NxCloud to your Nx Agents deployment
+
+These are the options you can use to configure how NxCloud connects to your Nx Agents cluster.
+
+Depending on how you deployed your Nx Agents cluster (which namespace you used, whether it was in the same or a different cluster etc.) you might need
+to use different combinations of the below values.
+
+Set these in your NxCloud `values.yaml` file:
+
+```yaml
+nxCloudWorkflows:
+  enabled: true
+  port: 9000
+  name: 'nx-cloud-workflow-controller-service'
+  workflowsNamespace: 'nx-cloud-workflows'
+
+  # If externalName is left unset, the applications will look for ane existing service with the name defined
+  # by `nxCloudWorkflows.name` in the namespace `nxCloudWorkflows.workflowsNamespace`. Use this option if you are
+  # also running the nx-agents chart in the same cluster as this nx-cloud chart
+  #
+  # If externalName is set, an additional service will be created with the name `nxCloudWorkflows.name`
+  # in the global namespace of this chart, and applications will use that service to connect to the workflow controller.
+  # Use this option if your nx-agents are running in a different cluster than this nx-cloud chart
+  externalName: ''
+
+  # If you find that an externalName service is not working as expected, you can set this to true to create a headless service
+  # which will create an endpoint group as an alternative. Please continue to set `externalName` to the IP address
+  # you wish to direct traffic to as we will use it to populate the endpoint slice.
+  headless: false
+```
+
+Please see `charts/nx-cloud/values.yaml` for up to date docs on the above options.
+
+Finally, push the updates to your NxCloud cluster:
+
+```bash
+helm upgrade --install nx-cloud nx-cloud/nx-cloud --values=nx-cloud-values.yml
+```

agents-guide/agents-secrets.yml

@@ -0,0 +1,9 @@
+apiVersion: v1
+kind: Secret
+metadata:
+  name: nx-cloud-agents-secret
+type: Opaque
+stringData:
+  AWS_S3_ACCESS_KEY_ID: ''
+  AWS_S3_SECRET_ACCESS_KEY: ''
+  valkey-password: ''

charts/nx-agents/Chart.yaml

@@ -2,7 +2,7 @@ apiVersion: v2
 name: nx-agents
 description: Nx Cloud Agents Helm Chart
 type: application
-version: 1.2.0
+version: 1.2.1
 maintainers:
   - name: nx
     url: "https://nx.app/"

charts/nx-agents/templates/deployment.yaml

@@ -91,12 +91,21 @@ spec:
             {{- toYaml .Values.controller.deployment.envFrom | nindent 12}}
           {{- end }}
           env:
+            - name: VALKEY_CONNECTION_STRING
+              value: valkey://valkey-master.{{ .Values.global.namespace }}.svc.cluster.local:6379
           {{- if gt (len .Values.controller.deployment.env) 0 }}
             {{- toYaml .Values.controller.deployment.env | nindent 12 }}
-          {{- end }}          
+          {{- end }}
           {{- with .Values.secret }}
-          {{- if .awsS3AccessKeyId }}
           {{- if .name }}
+          {{- if .valkeyPassword }}
+            - name: VALKEY_PASSWORD
+              valueFrom:
+                secretKeyRef:
+                  name: {{ .name }}
+                  key: {{ .valkeyPassword }}
+          {{- end }}
+          {{- if .awsS3AccessKeyId }}
             - name: AWS_S3_ACCESS_KEY_ID
               valueFrom:
                 secretKeyRef:

charts/nx-agents/values.yaml

@@ -128,9 +128,12 @@ daemonset:
 # If set, ensure to provide the proper ENV mappings that aws client libararies expect such as AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
 # for both the controller and executor. Additionally point the controller to the correct bucket and region with the NX_CLOUD_AWS_BUCKET variable
 secret:
+  # the name of the secret needs to match the name of the secret provided to valkey (see AGENTS-GUIDE.md)
   name: ''
   awsS3AccessKeyId: ''
   awsS3SecretAccessKey: ''
+  # this needs to remain as is. It is used by the controller to load the password for valkey from the provided secret.
+  valkeyPassword: ''
 
 
 # If you would like to manage additional resources with this helm chart you can add additional manifests here.