Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with variable value beginning with a pipe #1454

Closed
icmcnamara opened this issue Dec 10, 2024 · 2 comments · Fixed by #1457
Closed

Issue with variable value beginning with a pipe #1454

icmcnamara opened this issue Dec 10, 2024 · 2 comments · Fixed by #1457
Labels
bug Something isn't working

Comments

@icmcnamara
Copy link

icmcnamara commented Dec 10, 2024

Minimal .gitlab-ci.yml illustrating the issue

job:
  variables:
    JOB_VAR_BEGINNING_WITH_PIPE_01: '|1|job_var_example'
    JOB_VAR_BEGINNING_WITH_PIPE_02: "|2|job_var_example"
    JOB_VAR_BEGINNING_WITH_PIPE_03: >
      |3|job_var_example
    JOB_VAR_BEGINNING_WITH_PIPE_04: |
      |4|job_var_example
  script:
    - |
      echo $JOB_VAR_BEGINNING_WITH_PIPE_01;
      echo $JOB_VAR_BEGINNING_WITH_PIPE_02;
      echo $JOB_VAR_BEGINNING_WITH_PIPE_03;
      echo $JOB_VAR_BEGINNING_WITH_PIPE_04;
      echo $LOCAL_VAR_BEGINNING_WITH_PIPE_01;
      echo $LOCAL_VAR_BEGINNING_WITH_PIPE_02;
      echo $LOCAL_VAR_BEGINNING_WITH_PIPE_03;
      echo $LOCAL_VAR_BEGINNING_WITH_PIPE_04;
      echo $GLOBAL_VAR_BEGINNING_WITH_PIPE_01;
      echo $GLOBAL_VAR_BEGINNING_WITH_PIPE_02;
      echo $GLOBAL_VAR_BEGINNING_WITH_PIPE_03;
      echo $GLOBAL_VAR_BEGINNING_WITH_PIPE_04;
    - echo $(<$GLOBAL_VAR_BEGINNING_WITH_PIPE_04)

.gitlab-ci-local-variables.yml in local directory

LOCAL_VAR_BEGINNING_WITH_PIPE_01: '|1|local_var_example'
LOCAL_VAR_BEGINNING_WITH_PIPE_02: "|2|local_var_example"
LOCAL_VAR_BEGINNING_WITH_PIPE_03: >
  |3|local_var_example
LOCAL_VAR_BEGINNING_WITH_PIPE_04: |
  |4|local_var_example

$HOME/.gitlab-ci-local/variables.yml

global:
  GLOBAL_VAR_BEGINNING_WITH_PIPE_01: '|1|global_var_example'
  GLOBAL_VAR_BEGINNING_WITH_PIPE_02: "|2|global_var_example"
  GLOBAL_VAR_BEGINNING_WITH_PIPE_03: >
    |3|global_var_example
  GLOBAL_VAR_BEGINNING_WITH_PIPE_04: |
    |4|global_var_example

Expected behavior
gitlab-ci-local should run and echo the following:

job $ echo $JOB_VAR_BEGINNING_WITH_PIPE_01; # collapsed multi-line command
job > |1|job_var_example
job > |2|job_var_example
job > |3|job_var_example
job > |4|job_var_example
job > |1|local_var_example
job > |2|local_var_example
job > |3|local_var_example
job > |4|local_var_example
job > |1|global_var_example
job > |2|global_var_example
job > |3|global_var_example
job > |4|global_var_example

Current behavior

GitLab CI Local appears to be interpreting the non-job variables as file paths.

$ gitlab-ci-local
parsing and downloads finished in 629 ms.
json schema validated in 174 ms
job starting shell (test)
job $ echo $JOB_VAR_BEGINNING_WITH_PIPE_01; # collapsed multi-line command
job > |1|job_var_example
job > |2|job_var_example
job > |3|job_var_example
job > |4|job_var_example
job > /tmp/gitlab-ci-local-file-variables-[GITLAB_PROJECT_PATH]-928424/LOCAL_VAR_BEGINNING_WITH_PIPE_01
job > /tmp/gitlab-ci-local-file-variables-[GITLAB_PROJECT_PATH]-928424/LOCAL_VAR_BEGINNING_WITH_PIPE_02
job > /tmp/gitlab-ci-local-file-variables-[GITLAB_PROJECT_PATH]-928424/LOCAL_VAR_BEGINNING_WITH_PIPE_03
job > /tmp/gitlab-ci-local-file-variables-[GITLAB_PROJECT_PATH]-928424/LOCAL_VAR_BEGINNING_WITH_PIPE_04
job > /tmp/gitlab-ci-local-file-variables-[GITLAB_PROJECT_PATH]-928424/GLOBAL_VAR_BEGINNING_WITH_PIPE_01
job > /tmp/gitlab-ci-local-file-variables-[GITLAB_PROJECT_PATH]-928424/GLOBAL_VAR_BEGINNING_WITH_PIPE_02
job > /tmp/gitlab-ci-local-file-variables-[GITLAB_PROJECT_PATH]-928424/GLOBAL_VAR_BEGINNING_WITH_PIPE_03
job > /tmp/gitlab-ci-local-file-variables-[GITLAB_PROJECT_PATH]-928424/GLOBAL_VAR_BEGINNING_WITH_PIPE_04
job finished in 117 ms

Host information
MacBook Pro running Sonoma 14.7.1 (23H222)
gitlab-ci-local 4.56.0

Containerd binary
Docker version 26.0.0, build 2ae903e

Additional context
Inside our GitLab CI workflow we have some know_hosts files being stored as raw text. Attempting to duplicate these variables anywhere but inside a job's variables results in the above. Attempting to read a file from the locations listed results in a file not found message.

Minimal .gitlab-ci.yml showing off file not found

job:
  variables:
    JOB_VAR_BEGINNING_WITH_PIPE_01: '|1|job_var_example'
    JOB_VAR_BEGINNING_WITH_PIPE_02: "|2|job_var_example"
    JOB_VAR_BEGINNING_WITH_PIPE_03: >
      |3|job_var_example
    JOB_VAR_BEGINNING_WITH_PIPE_04: |
      |4|job_var_example
  script:
    - echo $GLOBAL_VAR_BEGINNING_WITH_PIPE_04;
    - echo $(<$GLOBAL_VAR_BEGINNING_WITH_PIPE_04)
$ gitlab-ci-local --cleanup
parsing and downloads finished in 602 ms.
json schema validated in 159 ms
job starting shell (test)
job $ echo $GLOBAL_VAR_BEGINNING_WITH_PIPE_04;
job > /tmp/gitlab-ci-local-file-variables-[PROJECT_DIR]-245031/GLOBAL_VAR_BEGINNING_WITH_PIPE_04
job $ echo $(<$GLOBAL_VAR_BEGINNING_WITH_PIPE_04)
job > warn: GLOBAL_VAR_BEGINNING_WITH_PIPE_04 is pointing to invalid path
job finished in 98 ms
@icmcnamara
Copy link
Author

Possibly of interest, passing in via the command line seems to work. A good workaround for me at this point.

$ gitlab-ci-local --variable CLI_VAR_01='|1|cli_var'
parsing and downloads finished in 528 ms.
json schema validated in 176 ms
job starting shell (test)
job $ echo $CLI_VAR_01;
job > |1|cli_var
job finished in 112 ms

@ANGkeith ANGkeith added the bug Something isn't working label Dec 13, 2024
@ANGkeith
Copy link
Collaborator

ANGkeith commented Dec 13, 2024

Good spot on this bug, quite unlucky of you to encounter it 😅

you can consider this slightly cleaner workaround

# .gitlab-ci-local-variables.yml in local directory
# https://github.com/firecow/gitlab-ci-local?tab=readme-ov-file#env-format
LOCAL_VAR_BEGINNING_WITH_PIPE_01='|1|local_var_example'
LOCAL_VAR_BEGINNING_WITH_PIPE_02="|2|local_var_example"
LOCAL_VAR_BEGINNING_WITH_PIPE_03=|3|local_var_example
LOCAL_VAR_BEGINNING_WITH_PIPE_04=|4|local_var_example

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants