Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify that - is replaced by _ when changing meta keys into environment variables #15359

Closed
Kamilcuk opened this issue Nov 22, 2022 · 2 comments · Fixed by #19501
Closed

Specify that - is replaced by _ when changing meta keys into environment variables #15359

Kamilcuk opened this issue Nov 22, 2022 · 2 comments · Fixed by #19501
Labels
good first issue stage/accepted Confirmed, and intend to work on. No timeline committment though. theme/docs Documentation issues and enhancements theme/jobspec

Comments

@Kamilcuk
Copy link
Contributor

On the documentation page https://developer.hashicorp.com/nomad/docs/runtime/environment#meta it is not explained that meta keys have replaced - for _ when transformed into environment variables. Although variables with - in the name can't be accessed with unix shell, they are technically perfectly valid environment variable names. I tried to follow the source code https://github.com/hashicorp/nomad/blob/main/client/taskenv/env.go#L115 but with no success.

Proposal

Please specify in the documentation that meta keys containing - are replaced by _ when transformed into environment variables.

Please also specify if other characters are replaced? Do all non-alphanumeric characters in meta keys are replaced by underscores?

Use-cases

Currently, this behavior I believe would count as "unspecified". Or it is specified in the documentation, but I have not found it, that's on me.

Attempted Solutions

job "test_meta_dash" {
  datacenters = [
    "wee-dev",
  ]
  type = "service"
  namespace = "dev"
  meta {
    key-key = "value"
  }
  group "test" {
    task "test" {
      driver = "docker"
      config {
        image = "busybox"
        command = "sh"
        args = [
          "-xc",
          "env | grep NOMAD_META_"
        ]
      }
    }
  }
}

outputs:

NOMAD_META_KEY_KEY=value
NOMAD_META_key_key=value

The environment variable is NOMAD_META_key_key not NOMAD_META_key-key.

@tgross
Copy link
Member

tgross commented Nov 22, 2022

Hi @Kamilcuk! This definitely needs better documentation. We should probably also update the docs for the meta block. I'll mark this for fixing but if you're interested in opening a PR with the fix I'd be happy to review it as well.

In the meantime, the code that calls this is in taskenv/env.go#L611-L622, and it "cleans" any of the Nomad-owned environment variable paths. That CleanEnvVar function removes anything that isn't in [A-Za-z0-9_\.]. This was introduced in #2405

@tgross tgross added theme/docs Documentation issues and enhancements theme/jobspec and removed type/enhancement labels Nov 22, 2022
@tgross tgross added stage/accepted Confirmed, and intend to work on. No timeline committment though. good first issue labels Nov 22, 2022
@Kamilcuk
Copy link
Contributor Author

Amazing, thank you for code links and clarification.

tgross added a commit that referenced this issue Dec 15, 2023
The keys of `meta` fields have all characters outside of `[A-Za-z0-9_.]`
replaced by underscores when we create `NOMAD_META` environment variables. Make
sure this replacement is documented.

Fixes: #15359
tgross added a commit that referenced this issue Dec 15, 2023
The keys of `meta` fields have all characters outside of `[A-Za-z0-9_.]`
replaced by underscores when we create `NOMAD_META` environment variables. Make
sure this replacement is documented.

Fixes: #15359
nvanthao pushed a commit to nvanthao/nomad that referenced this issue Mar 1, 2024
…19501)

The keys of `meta` fields have all characters outside of `[A-Za-z0-9_.]`
replaced by underscores when we create `NOMAD_META` environment variables. Make
sure this replacement is documented.

Fixes: hashicorp#15359
nvanthao pushed a commit to nvanthao/nomad that referenced this issue Mar 1, 2024
…19501)

The keys of `meta` fields have all characters outside of `[A-Za-z0-9_.]`
replaced by underscores when we create `NOMAD_META` environment variables. Make
sure this replacement is documented.

Fixes: hashicorp#15359
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue stage/accepted Confirmed, and intend to work on. No timeline committment though. theme/docs Documentation issues and enhancements theme/jobspec
Projects
Development

Successfully merging a pull request may close this issue.

2 participants