-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
git artifact protocol not working as expected #13579
Comments
Hi @cberescu! The go-getter library that we use for artifact fetching was upgraded in Nomad 1.3.1 in 3968509 to patch a number of security vulnerabilities (ref #13057), so that's my first suspicion as to the problem. I don't see any obvious place where the go-getter code has changed that could be causing this problem. And if we take a look at recent git getter history there's not much there other than the new timeouts either. That being said, it's certainly possible the library is more strict now and I'm just missing how. Can you see if it might work if you remove the |
Hi @tgross , i have tried removing the .git and did not work. So, as i said the problem is that i download multiple artifacts to the same destination and the git module doesn't like for anything to be in the destination directory (must be empty). From what i could gather there may be one of the following that has changed:
|
My apologies @cberescu, I did some sloppy reading of your carefully-written issue description and totally missed that the destinations collided. 😊 As of Nomad 1.3.0 we download artifacts in parallel: #11531 The workaround for your use case would be to download the config file via artifact {
source = "git::https://domain.com/somerepository.git"
destination = "../alloc/data/repo"
}
artifact {
source = "https://domain.com/somefile.json"
destination = "local/config.json"
}
template {
source = "local/config.json"
destination = "../alloc/data/repo/config.json"
} |
Thanks a lot @tgross, it seems i missed that in the changelog. Works great as you recommended. You're a lifesaver! |
Just one more question , what do you recommend to do in case an artifact is an archive? The template solution clearly will not work. |
You could use a prestart task to move the archive contents. But for something that simple, I usually would wrap my main application in a script that moves the archive contents at start. For example, the job below is running against a local artifact server on port 15101. The locals {
auth_header = base64encode("artifacts:supersecret")
}
job "example" {
datacenters = ["dc1"]
group "web" {
network {
mode = "bridge"
port "www" {
to = 8001
}
}
task "httpd" {
artifact {
source = "http://localhost:15101/website.tar.gz"
destination = "local/temp"
headers {
Authorization = "Basic ${local.auth_header}"
}
}
template {
data = <<EOT
#!/usr/bin/env sh
mkdir /srv
mv local/temp/* /srv
exec httpd -v -f -p 8001 -h /srv
EOT
destination = "local/run.sh"
}
driver = "docker"
config {
image = "busybox:1"
command = "/bin/sh"
args = ["local/run.sh"]
ports = ["www"]
}
resources {
cpu = 128
memory = 128
}
}
}
} |
Thanks for the answer, will go with the prestart option most likely , i can structure things a little better this way. Thanks again . |
I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. |
Nomad version
Nomad v1.3.1 (2b054e3)
Operating system and Environment details
Ubuntu 20.04.4 LTS
Issue
When using multiple artifacts that have the same destination, and one of the artifacts is using the git module/protocol the allocation always fails with error :
/usr/bin/git exited with 128: fatal: not a git repository (or any of the parent directories): .git
Reproduction steps
Expected Result
For the repository to be cloned. This worked on v1.2.6
The text was updated successfully, but these errors were encountered: