Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Variable substitution only partly working on Jobs #1782

Closed
stravag opened this issue Nov 12, 2019 · 3 comments
Closed

Variable substitution only partly working on Jobs #1782

stravag opened this issue Nov 12, 2019 · 3 comments

Comments

@stravag
Copy link

stravag commented Nov 12, 2019

We are using kustomize bundled with kubectl and have some issues with variable replacement.

We have a Job and Deployment both make use of initContainers and in all we'd like to use kustomize vars that should automatically apply a prefix.

When applying kustomize, for the Deployment the variables get correctly substituted and prefixed.

Deployment definition

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  ...
  template:
    ...
    spec:
      containers:
        - name: nginx
          env:
            - name: BACKEND_HOST
              value: $(BACKEND_SERVICE_NAME)
      initContainers:
        - name: init-nginx-wait-backend
          image: busybox:1.31
          command: ['sh', '-c', 'until nslookup $(BACKEND_SERVICE_NAME); do sleep 2; done;']

Deployment Kustomize substitution

apiVersion: apps/v1
kind: Deployment
...
      containers:
      - env:
        - name: BACKEND_HOST
          value: test-backend-service
      initContainers:
      - command:
        - sh
        - -c
        - until nslookup test-backend-service; do echo waiting for backend; sleep
          2; done;
        env:
        - name: IAM_HOST
          value: test-iam-service
        image: busybox:1.31
        name: init-nginx-wait-backend

As expected $(BACKEND_SERVICE_NAME) gets substituted with: test-backend-service in both the command and the env var of both container and initContainer 👍

The same thing with the Job

Job definition

apiVersion: batch/v1
kind: Job
metadata:
  name: some-job
spec:
  template:
    ...
    spec:
      containers:
          env:
            - name: HOST
              value: $(SERVICE_NAME)
      initContainers:
        - name: init-test
          env:
            - name: HOST
              value: $(SERVICE_NAME)
          command: ['sh', '-c', 'printenv; echo $(SERVICE_NAME); echo $HOST']
  backoffLimit: 0

Job Kustomize substitution

apiVersion: batch/v1
kind: Job
metadata:
  labels:
    env: test
  name: test-some-job
spec:
  template:
    ...
    spec:
      containers:
        env:
        - name: HOST
          value: test-service
      initContainers:
      - command:
        - sh
        - -c
        - printenv; echo $(SERVICE_NAME); echo $IAM_HOST
        env:
        - name: HOST
          value: $(SERVICE_NAME)
        name: init-test

The same setup for Jobs. First as exptected $(SERVICE_NAME) gets substituted with test-service but only the variable in the container section! Everything under initContainers remains as is. If I change the type from Job to Deployment the substitution works as expected.

Is this a known issue?

My current workaround is to to patch the values with an overlay instead of using the variables.

@haiyanmeng
Copy link
Contributor

I have trouble to reproduce your issue. Can you share more info on how setup:

  1. kustomize version
  2. your kustomization.yaml file

I tested your example, and kustomize works well.

$ cat job.yaml 
apiVersion: batch/v1
kind: Job
metadata:
  name: nginx-job
spec:
  template:
    spec:
      containers:
        - name: nginx
          env:
            - name: BACKEND_HOST
              value: $(BACKEND_SERVICE_NAME)
      initContainers:
        - name: init-nginx-wait-backend
          image: busybox:1.31
          command: ['sh', '-c', 'until nslookup $(BACKEND_SERVICE_NAME); do sleep 2; done;']

$ cat kustomization.yaml 
resources:
- job.yaml

vars:
- name: BACKEND_SERVICE_NAME
  objref:
    kind: Job
    name: nginx-job
    apiVersion: batch/v1

$ kustomize build 
apiVersion: batch/v1
kind: Job
metadata:
  name: nginx-job
spec:
  template:
    spec:
      containers:
      - env:
        - name: BACKEND_HOST
          value: nginx-job
        name: nginx
      initContainers:
      - command:
        - sh
        - -c
        - until nslookup nginx-job; do sleep 2; done;
        image: busybox:1.31
        name: init-nginx-wait-backend

$ kustomize version
{Version:kustomize/v3.4.0 GitCommit:2c9635967a2b1469d605a91a1d040bd27c73ca7d BuildDate:2019-11-12T05:00:57Z GoOs:linux GoArch:amd64}

@stravag
Copy link
Author

stravag commented Nov 20, 2019

Hi thanks for the reply. As I mentioned we're using the kustomize integrated in kubectl

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-19T13:57:45Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"darwin/amd64"}

Which I believe according to the README.md in this repo is still v.2.0.3.

What's the suggested approach, use the standalone kustomize?

@monopole
Copy link
Contributor

monopole commented Jan 9, 2020

The kubectl version lags significantly - suggest trying the standalone.

@monopole monopole closed this as completed Jan 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants