BUG: Kubefile CMD can not get ENV values with `${VERSION}` format #3853

willzhang · 2023-09-06T08:36:51Z

Sealos Version

v4.3.3

How to reproduce the bug?

condition1:

when i use ${VERSION} in CMD

FROM scratch
ENV VERSION=v1.0
CMD ["echo VERSION=${VERSION}"]

CMD can not get ENV values

root@node1:~# sealos run registry.cn-shenzhen.aliyuncs.com/cnmirror/test:v1.0
2023-09-06T16:33:32 info start to install app in this cluster
2023-09-06T16:33:32 info Executing SyncStatusAndCheck Pipeline in InstallProcessor
2023-09-06T16:33:32 info Executing ConfirmOverrideApps Pipeline in InstallProcessor
2023-09-06T16:33:32 info are you sure to override these following apps? 
registry.cn-shenzhen.aliyuncs.com/cnmirror/test:v1.0
✔ Yes [y/yes], No [n/no]: y█
2023-09-06T16:33:34 info Executing PreProcess Pipeline in InstallProcessor
2023-09-06T16:33:34 info Executing pipeline MountRootfs in InstallProcessor.
2023-09-06T16:33:34 info Executing pipeline MirrorRegistry in InstallProcessor.
2023-09-06T16:33:34 info Executing UpgradeIfNeed Pipeline in InstallProcessor
VERSION=
2023-09-06T16:33:34 info succeeded install app in this cluster
2023-09-06T16:33:34 info no nodes that need to be scaled
2023-09-06T16:33:34 info 
      ___           ___           ___           ___       ___           ___
     /\  \         /\  \         /\  \         /\__\     /\  \         /\  \
    /::\  \       /::\  \       /::\  \       /:/  /    /::\  \       /::\  \
   /:/\ \  \     /:/\:\  \     /:/\:\  \     /:/  /    /:/\:\  \     /:/\ \  \
  _\:\~\ \  \   /::\~\:\  \   /::\~\:\  \   /:/  /    /:/  \:\  \   _\:\~\ \  \
 /\ \:\ \ \__\ /:/\:\ \:\__\ /:/\:\ \:\__\ /:/__/    /:/__/ \:\__\ /\ \:\ \ \__\
 \:\ \:\ \/__/ \:\~\:\ \/__/ \/__\:\/:/  / \:\  \    \:\  \ /:/  / \:\ \:\ \/__/
  \:\ \:\__\    \:\ \:\__\        \::/  /   \:\  \    \:\  /:/  /   \:\ \:\__\
   \:\/:/  /     \:\ \/__/        /:/  /     \:\  \    \:\/:/  /     \:\/:/  /
    \::/  /       \:\__\         /:/  /       \:\__\    \::/  /       \::/  /
     \/__/         \/__/         \/__/         \/__/     \/__/         \/__/

                  Website: https://www.sealos.io/
                  Address: github.com/labring/sealos
                  Version: 4.3.3-d27f4fbc

condition2:

when i use $(VERSION) in CMD

FROM scratch
ENV VERSION=v1.0
CMD ["echo VERSION=$(VERSION)"]

CMD can get ENV values

root@node1:~# sealos run registry.cn-shenzhen.aliyuncs.com/cnmirror/test:v1.0
2023-09-06T16:35:36 info start to install app in this cluster
2023-09-06T16:35:36 info Executing SyncStatusAndCheck Pipeline in InstallProcessor
2023-09-06T16:35:36 info Executing ConfirmOverrideApps Pipeline in InstallProcessor
2023-09-06T16:35:36 info are you sure to override these following apps? 
registry.cn-shenzhen.aliyuncs.com/cnmirror/test:v1.0
Yes [y/yes], No [n/no]: y
2023-09-06T16:35:38 info Executing PreProcess Pipeline in InstallProcessor
2023-09-06T16:35:38 info Executing pipeline MountRootfs in InstallProcessor.
2023-09-06T16:35:38 info Executing pipeline MirrorRegistry in InstallProcessor.
2023-09-06T16:35:38 info Executing UpgradeIfNeed Pipeline in InstallProcessor
VERSION=v1.0
2023-09-06T16:35:38 info succeeded install app in this cluster
2023-09-06T16:35:38 info no nodes that need to be scaled
2023-09-06T16:35:38 info 
      ___           ___           ___           ___       ___           ___
     /\  \         /\  \         /\  \         /\__\     /\  \         /\  \
    /::\  \       /::\  \       /::\  \       /:/  /    /::\  \       /::\  \
   /:/\ \  \     /:/\:\  \     /:/\:\  \     /:/  /    /:/\:\  \     /:/\ \  \
  _\:\~\ \  \   /::\~\:\  \   /::\~\:\  \   /:/  /    /:/  \:\  \   _\:\~\ \  \
 /\ \:\ \ \__\ /:/\:\ \:\__\ /:/\:\ \:\__\ /:/__/    /:/__/ \:\__\ /\ \:\ \ \__\
 \:\ \:\ \/__/ \:\~\:\ \/__/ \/__\:\/:/  / \:\  \    \:\  \ /:/  / \:\ \:\ \/__/
  \:\ \:\__\    \:\ \:\__\        \::/  /   \:\  \    \:\  /:/  /   \:\ \:\__\
   \:\/:/  /     \:\ \/__/        /:/  /     \:\  \    \:\/:/  /     \:\/:/  /
    \::/  /       \:\__\         /:/  /       \:\__\    \::/  /       \::/  /
     \/__/         \/__/         \/__/         \/__/     \/__/         \/__/

                  Website: https://www.sealos.io/
                  Address: github.com/labring/sealos
                  Version: 4.3.3-d27f4fbc

What is the expected behavior?

when i use ${VERSION} in CMD, i hope CMD can get ENV values.

What do you see instead?

when i use ${VERSION} in CMD, CMD can not get ENV values , i must use $(VERSION) format.

Operating environment

- Sealos version:
- Docker version:
- Kubernetes version:
- Operating system:
- Runtime environment:
- Cluster size:
- Additional information:

and It is work to use sealos run - e to pass variables in these two condition.

Additional information

related issue: Kubefile CMD not support Standard environment variable format

The text was updated successfully, but these errors were encountered:

cuisongliu · 2023-09-06T15:40:58Z

CMD ["echo VERSION=${VERSION}"] this type docker engine is not support. but $() is support .

I think this is not bug, is docker using spec.

cuisongliu · 2023-09-07T02:30:01Z

sealos-ci-robot · 2023-09-07T02:30:13Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

willzhang · 2023-09-07T02:43:34Z

not work

sample dockerfile is ok

root@ubuntu:/data/dockerfile# ls
Dockerfile
root@ubuntu:/data/dockerfile# cat Dockerfile 
FROM alpine:latest

ENV MY_VARIABLE=default_value

CMD echo "The value of MY_VARIABLE is $MY_VARIABLE"


root@ubuntu:/data/dockerfile# docker run test
The value of MY_VARIABLE is default_value
root@ubuntu:/data/dockerfile#

but Kubefile not work

root@ubuntu:/data/cluster-image/test# cat Kubefile 
FROM scratch

ENV MY_VARIABLE=default_value

CMD echo "The value of MY_VARIABLE is $MY_VARIABLE"


root@node1:~# sealos run registry.cn-shenzhen.aliyuncs.com/cnmirror/test:v1.0
2023-09-07T10:48:22 info start to install app in this cluster
2023-09-07T10:48:22 info Executing SyncStatusAndCheck Pipeline in InstallProcessor
2023-09-07T10:48:22 info Executing ConfirmOverrideApps Pipeline in InstallProcessor
2023-09-07T10:48:22 info are you sure to override these following apps? 
✔ Yes [y/yes], No [n/no]: y█
2023-09-07T10:48:24 info Executing PreProcess Pipeline in InstallProcessor
2023-09-07T10:48:24 info Executing pipeline MountRootfs in InstallProcessor.
2023-09-07T10:48:24 info Executing pipeline MirrorRegistry in InstallProcessor.
2023-09-07T10:48:24 info Executing UpgradeIfNeed Pipeline in InstallProcessor
The value of MY_VARIABLE is 
2023-09-07T10:48:24 info succeeded install app in this cluster
2023-09-07T10:48:24 info no nodes that need to be scaled
2023-09-07T10:48:24 info 
      ___           ___           ___           ___       ___           ___
     /\  \         /\  \         /\  \         /\__\     /\  \         /\  \
    /::\  \       /::\  \       /::\  \       /:/  /    /::\  \       /::\  \
   /:/\ \  \     /:/\:\  \     /:/\:\  \     /:/  /    /:/\:\  \     /:/\ \  \
  _\:\~\ \  \   /::\~\:\  \   /::\~\:\  \   /:/  /    /:/  \:\  \   _\:\~\ \  \
 /\ \:\ \ \__\ /:/\:\ \:\__\ /:/\:\ \:\__\ /:/__/    /:/__/ \:\__\ /\ \:\ \ \__\
 \:\ \:\ \/__/ \:\~\:\ \/__/ \/__\:\/:/  / \:\  \    \:\  \ /:/  / \:\ \:\ \/__/
  \:\ \:\__\    \:\ \:\__\        \::/  /   \:\  \    \:\  /:/  /   \:\ \:\__\
   \:\/:/  /     \:\ \/__/        /:/  /     \:\  \    \:\/:/  /     \:\/:/  /
    \::/  /       \:\__\         /:/  /       \:\__\    \::/  /       \::/  /
     \/__/         \/__/         \/__/         \/__/     \/__/         \/__/

                  Website: https://www.sealos.io/
                  Address: github.com/labring/sealos
                  Version: 4.3.3-d27f4fbc

docker inspect

root@ubuntu:/data/dockerfile# docker inspect test
[
    {
        "Id": "sha256:51ea13928bd78e82a4ba3940f88bd232e7e9d412fb23024c0af7005731de92f5",
        "RepoTags": [
            "test:latest"
        ],
        "RepoDigests": [],
        "Parent": "",
        "Comment": "buildkit.dockerfile.v0",
        "Created": "2023-08-07T19:20:20.894140623Z",
        "Container": "",
        "ContainerConfig": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": null,
            "Cmd": null,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "DockerVersion": "",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "MY_VARIABLE=default_value"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "echo \"The value of MY_VARIABLE is $MY_VARIABLE\""
            ],
            "ArgsEscaped": true,
            "Image": "",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 7335363,
        "VirtualSize": 7335363,
        "GraphDriver": {
            "Data": {
                "MergedDir": "/var/lib/docker/overlay2/220af1a94871cdee5df90d023df4ebbdb78e8f649f53c5f9461d48da181e8193/merged",
                "UpperDir": "/var/lib/docker/overlay2/220af1a94871cdee5df90d023df4ebbdb78e8f649f53c5f9461d48da181e8193/diff",
                "WorkDir": "/var/lib/docker/overlay2/220af1a94871cdee5df90d023df4ebbdb78e8f649f53c5f9461d48da181e8193/work"
            },
            "Name": "overlay2"
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:4693057ce2364720d39e57e85a5b8e0bd9ac3573716237736d6470ec5b7b7230"
            ]
        },
        "Metadata": {
            "LastTagTime": "0001-01-01T00:00:00Z"
        }
    }
]

sealos inspect, the env and cmd are same as docker

root@ubuntu:/data/cluster-image/test# sealos inspect registry.cn-shenzhen.aliyuncs.com/cnmirror/test:v1.0
{
    "Name": "registry.cn-shenzhen.aliyuncs.com/cnmirror/test:v1.0",
    "FromImageDigest": "sha256:5ed32ca869e745c89b532b4c73baf8e0b4cf6b97a815afd048afe5704f76c8d3",
    "FromImageID": "61249c2a2af99b0b5d209a33fda5adc9b5e74ac5a7fdc2160a8dc970cec9bb5d",
    "OCIv1": {
        "created": "2023-09-07T02:48:09.262424504Z",
        "architecture": "amd64",
        "os": "linux",
        "config": {
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "MY_VARIABLE=default_value"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "echo \"The value of MY_VARIABLE is $MY_VARIABLE\""
            ],
            "Labels": {
                "io.buildah.version": "1.30.0"
            }
        },
        "rootfs": {
            "type": "layers",
            "diff_ids": [
                "sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef"
            ]
        },
        "history": [
            {
                "created": "2023-09-07T02:48:09.260320923Z",
                "created_by": "/bin/sh -c #(nop) ENV MY_VARIABLE=default_value",
                "empty_layer": true
            },
            {
                "created": "2023-09-07T02:48:09.263153537Z",
                "created_by": "/bin/sh -c #(nop) CMD echo \"The value of MY_VARIABLE is $MY_VARIABLE\""
            }
        ]
    }
}

sealos-ci-robot · 2023-09-07T02:43:47Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

not work

willzhang · 2023-09-07T02:55:51Z

$ sealos run test --debug
2023-09-07T10:53:22 info Executing ConfirmOverrideApps Pipeline in InstallProcessor
2023-09-07T10:53:22 info are you sure to override these following apps? 
registry.cn-shenzhen.aliyuncs.com/cnmirror/test:v1.0
Yes [y/yes], No [n/no]: y
2023-09-07T10:53:31 info Executing PreProcess Pipeline in InstallProcessor
2023-09-07T10:53:31 debug images 61249c2a2af99b0b5d209a33fda5adc9b5e74ac5a7fdc2160a8dc970cec9bb5d are pulled
2023-09-07T10:53:31 debug parse reference 61249c2a2af99b0b5d209a33fda5adc9b5e74ac5a7fdc2160a8dc970cec9bb5d with transport containers-storage
2023-09-07T10:53:31 debug trying to override app registry.cn-shenzhen.aliyuncs.com/cnmirror/test:v1.0
2023-09-07T10:53:31 debug Pull Policy for pull [missing]
2023-09-07T10:53:31 debug parse reference 61249c2a2af99b0b5d209a33fda5adc9b5e74ac5a7fdc2160a8dc970cec9bb5d with transport containers-storage
2023-09-07T10:53:31 info Executing pipeline MountRootfs in InstallProcessor.
2023-09-07T10:53:31 debug render env dir: /var/lib/containers/storage/overlay/cc40194b204cc46d67c29d469bc14529de9228a96f2944670f3de1a803f277d9/merged/etc
2023-09-07T10:53:31 debug render env dir: /var/lib/containers/storage/overlay/cc40194b204cc46d67c29d469bc14529de9228a96f2944670f3de1a803f277d9/merged/scripts
2023-09-07T10:53:31 debug render env dir: /var/lib/containers/storage/overlay/cc40194b204cc46d67c29d469bc14529de9228a96f2944670f3de1a803f277d9/merged/manifests
2023-09-07T10:53:31 debug send mount image, target: 192.168.72.40, image: registry.cn-shenzhen.aliyuncs.com/cnmirror/test:v1.0, type: application
2023-09-07T10:53:31 debug not trying to parse private key file cause it's not exists
2023-09-07T10:53:31 debug local 192.168.72.40 copy files src /var/lib/containers/storage/overlay/cc40194b204cc46d67c29d469bc14529de9228a96f2944670f3de1a803f277d9/merged to dst /var/lib/sealos/data/default/applications/default-f0ur1w4v/workdir
2023-09-07T10:53:31 info Executing pipeline MirrorRegistry in InstallProcessor.
2023-09-07T10:53:31 debug registry nodes is: [192.168.72.40]
2023-09-07T10:53:31 info Executing UpgradeIfNeed Pipeline in InstallProcessor
2023-09-07T10:53:31 debug not trying to parse private key file cause it's not exists
2023-09-07T10:53:31 debug start to run command `cd /var/lib/sealos/data/default/applications/default-f0ur1w4v/workdir && echo "The value of MY_VARIABLE is $MY_VARIABLE"` via exec
The value of MY_VARIABLE is 
2023-09-07T10:53:31 info succeeded install app in this cluster
2023-09-07T10:53:31 info no nodes that need to be scaled
......


  - cmd:
    - echo "The value of MY_VARIABLE is $MY_VARIABLE"
    env:
      MY_VARIABLE: default_value
    imageName: registry.cn-shenzhen.aliyuncs.com/cnmirror/test:v1.0
    labels:
      io.buildah.version: 1.30.0
    mountPoint: /var/lib/containers/storage/overlay/cc40194b204cc46d67c29d469bc14529de9228a96f2944670f3de1a803f277d9/merged
    name: default-f0ur1w4v
    type: application

willzhang · 2023-09-07T04:22:29Z

I have been searching for a long time, but there is no documentation indicating that $(VARIABLE) is a dockerfile syntax. After testing, it has been found that native dockerfiles do not support this format.

I only find this ,it's for kubernetes

https://kubernetes.io/docs/tasks/inject-data-application/define-interdependent-environment-variables/

So where exactly does this format come from ?

LZiHaN · 2023-09-08T08:10:14Z

/assign

willzhang added the kind/bug Something isn't working label Sep 6, 2023

willzhang changed the title ~~BUG: Kubefile CMD can not get ENV values~~ BUG: Kubefile CMD can not get ENV values with ${VERSION} format Sep 6, 2023

cuisongliu closed this as completed Sep 7, 2023

cuisongliu reopened this Sep 7, 2023

cuisongliu assigned LZiHaN Sep 8, 2023

LZiHaN mentioned this issue Sep 13, 2023

feature: kubefile CMD support ENV values with "${...}" and "$..." format #3921

Merged

cuisongliu closed this as completed Sep 17, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: Kubefile CMD can not get ENV values with `${VERSION}` format #3853

BUG: Kubefile CMD can not get ENV values with `${VERSION}` format #3853

willzhang commented Sep 6, 2023 •

edited

Loading

cuisongliu commented Sep 6, 2023

cuisongliu commented Sep 7, 2023

sealos-ci-robot commented Sep 7, 2023

willzhang commented Sep 7, 2023 •

edited

Loading

sealos-ci-robot commented Sep 7, 2023

willzhang commented Sep 7, 2023 •

edited

Loading

willzhang commented Sep 7, 2023 •

edited

Loading

LZiHaN commented Sep 8, 2023

BUG: Kubefile CMD can not get ENV values with ${VERSION} format #3853

BUG: Kubefile CMD can not get ENV values with ${VERSION} format #3853

Comments

willzhang commented Sep 6, 2023 • edited Loading

Sealos Version

How to reproduce the bug?

What is the expected behavior?

What do you see instead?

Operating environment

Additional information

cuisongliu commented Sep 6, 2023

cuisongliu commented Sep 7, 2023

sealos-ci-robot commented Sep 7, 2023

willzhang commented Sep 7, 2023 • edited Loading

sealos-ci-robot commented Sep 7, 2023

willzhang commented Sep 7, 2023 • edited Loading

willzhang commented Sep 7, 2023 • edited Loading

LZiHaN commented Sep 8, 2023

BUG: Kubefile CMD can not get ENV values with `${VERSION}` format #3853

BUG: Kubefile CMD can not get ENV values with `${VERSION}` format #3853

willzhang commented Sep 6, 2023 •

edited

Loading

willzhang commented Sep 7, 2023 •

edited

Loading

willzhang commented Sep 7, 2023 •

edited

Loading

willzhang commented Sep 7, 2023 •

edited

Loading