git artifact protocol not working as expected

[cberescu]

Nomad version

Nomad v1.3.1 (2b054e3)

Operating system and Environment details

Ubuntu 20.04.4 LTS

Issue

When using multiple artifacts that have the same destination, and one of the artifacts is using the git module/protocol the allocation always fails with error :/usr/bin/git exited with 128: fatal: not a git repository (or any of the parent directories): .git

Reproduction steps

driver = "docker"
		artifact {
			source      = "git::https://domain.com/somerepository.git"
			destination = "../alloc/data/repo"

		}
		artifact {
			source      = "https://domain.com/somefile.json"
			destination = "../alloc/data/repo/config.json"
			mode		= "file"
		}

Expected Result

For the repository to be cloned. This worked on v1.2.6

[tgross]

Hi @cberescu!

The go-getter library that we use for artifact fetching was upgraded in Nomad 1.3.1 in 3968509 to patch a number of security vulnerabilities (ref #13057), so that's my first suspicion as to the problem.

I don't see any obvious place where the go-getter code has changed that could be causing this problem. And if we take a look at recent git getter history there's not much there other than the new timeouts either.

That being said, it's certainly possible the library is more strict now and I'm just missing how. Can you see if it might work if you remove the .git suffix from your URL?

[cberescu]

Hi @tgross , i have tried removing the .git and did not work.

So, as i said the problem is that i download multiple artifacts to the same destination and the git module doesn't like for anything to be in the destination directory (must be empty).

From what i could gather there may be one of the following that has changed:

1. the go-getter has changed and is not fetching the repository if the destination is not empty
2. the artifacts have become async in the new 1.3.x version and they are not fetched/processed in the order from the hcl file , in this case the second artifact that fetches content and puts it in the same destination as the git artifact completes first making the directory non empty , witch go-getter doesn't like and throws the error .

[tgross]

My apologies @cberescu, I did some sloppy reading of your carefully-written issue description and totally missed that the destinations collided. 😊

As of Nomad 1.3.0 we download artifacts in parallel: #11531 The workaround for your use case would be to download the config file via artifact and then move it into position as a template. If you take a look at the Filesystem docs, template rendering gets done after artifacts are downloaded. So something along the lines of the following:

artifact {
	source      = "git::https://domain.com/somerepository.git"
	destination = "../alloc/data/repo"
}

artifact {
	source      = "https://domain.com/somefile.json"
	destination = "local/config.json"
}

template {
	source      = "local/config.json"
	destination = "../alloc/data/repo/config.json"
}

[cberescu]

Thanks a lot @tgross, it seems i missed that in the changelog. Works great as you recommended. You're a lifesaver!

[cberescu]

Just one more question , what do you recommend to do in case an artifact is an archive? The template solution clearly will not work.

[tgross]

You could use a prestart task to move the archive contents. But for something that simple, I usually would wrap my main application in a script that moves the archive contents at start.

For example, the job below is running against a local artifact server on port 15101. The artifact block downloads and unpacks the archive. The template block defines the top-level script to move the unpacked contents to where the application expects them. And then the script execs into the httpd application so that PID1 in the container is still httpd and not /bin/sh. The only real advantage of having the script in the template block is that you can use it in cases where you don't "own" the container image (ex. something downloaded off the public Docker Registry).

locals {
  auth_header = base64encode("artifacts:supersecret")
}

job "example" {
  datacenters = ["dc1"]

  group "web" {

    network {
      mode = "bridge"
      port "www" {
        to = 8001
      }
    }

    task "httpd" {

      artifact {
        source      = "http://localhost:15101/website.tar.gz"
        destination = "local/temp"
        headers {
          Authorization = "Basic ${local.auth_header}"
        }
      }

      template {
        data        = <<EOT
#!/usr/bin/env sh
mkdir /srv
mv local/temp/* /srv

exec httpd -v -f -p 8001 -h /srv
EOT
        destination = "local/run.sh"
      }

      driver = "docker"

      config {
        image   = "busybox:1"
        command = "/bin/sh"
        args    = ["local/run.sh"]
        ports   = ["www"]
      }

      resources {
        cpu    = 128
        memory = 128
      }

    }
  }
}

[cberescu]

Thanks for the answer, will go with the prestart option most likely , i can structure things a little better this way.

Thanks again .

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.