-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to get nomad config/get template function_denylist option #11923
Comments
nomad process is not overriding/can't get config from /etc/nomad/config.json...
I'm also have /etc/nomad/config.json with my options, when i started nomad, it's launching, but i can't get final config at /var/log/messages. At other versions - i do. Can you please asap fix this issue? |
Hi @bubejur and thanks for raising this issue. Could you provide the Nomad version you upgraded from in order to help investigation as well as any potential configuration options you're using on the Nomad client/servers which could impact templating? |
Hi. Current version: 1.2.4. I also tried all versions from 1.1.6 to 1.2.4... |
example 1.2.0 is printing config:
1.2.4 is not. I did debug mode for nomad agent. |
Hi @bubejur. Could you please provide the following pieces of information to help understand the problem you are facing. Also to confirm, Nomad 1.2.0 you do not see the template rendering failure, but in 1.2.4 you do?
Thanks in advance. |
Also to confirm, Nomad 1.2.0 you do not see the template rendering failure, but in 1.2.4 you do? Yeap.
jobspec:
|
Hi @bubejur & @ngmlabs. Thanks for reporting and confirming this issue. I've confirmed a similar issue in Consul Template config parsing ( #11902 ). If it turns out to be the same issue, and I suspect it is, I'll likely merge this issue into that. Thanks again. Your contributions were a big help in finding this. |
Confirmed that this is a duplicate of #11902. Closing in favor of the original issue. |
Hi @bubejur. I'm sorry to hear you are still having issues after 1.2.5. I'm re-opening this issue for investigation, and will let you know if I see any work arounds for you so you don't have to wait for another release. What's confusing to me though, is that I don't see you actually calling any functions in your jobspec. Any chance you know where a function is being called? Is it possibly something embedded in the |
fyi i can see full config now in debug mode:
/worker-mpi-resolver/local/platformConfig/nl3.tmpl:
|
@bubejur I see the issue. I apologize for any inconvenience this causes you. I'll get a PR up as soon as I can. |
@DerekStrickland thank you for ur time, it will be great if u can fix it |
@DerekStrickland Hi! Do you have any updates? |
HI @bubejur, Sorry for the delay. I've set aside time to work on this Friday. Once it lands in main, you'll be able to pull, build, and test. I'd love to get your confirmation before we cut the next release. I'll link this issue to the PR so you that you are notified when it lands, and can test it out if you have time. Thanks for your help! Derek |
Great, thanks! Sure, i'll help you with some checks. See u on Friday. |
Thanks a lot! Finally got binary - going to test it now! |
Still unlucky :-(
|
@bubejur what happens if you put "template": {
"function_denylist": []
} |
@DerekStrickland tried this way, still getting error... |
All right. Thanks for the update. I'll let you know when I have new bits to share. |
@DerekStrickland hi! any updates? |
Hi @bubejur! I'm sorry no. I haven't been able to get back to it this week yet, but I'm looking for some schedule time because I know it's not ideal. I haven't forgotten about you I promise 😄 |
I've got some new code up for you to pull build and test. I just ran this on our test cluster and the template worked. Also, I see this in the final config in the logs. "FunctionDenylist": [
""
], |
@DerekStrickland still nothing...
|
Hi @bubejur We're getting closer I think. Looking at the commit hash in this output you included
It looks like you haven't built with the latest changes. You should see the following if you are on the latest commit for this branch.
Here are the steps I'm taking to make sure I am testing the new code. I just want to make sure we are both testing the same way. Forgive me if I am stating obvious things. I just don't want to assume anything.
Now that I have the right binary built my process is the following. NOTE: I am running a development cluster for testing so this is safe for me. If you are testing this fix with a non-development cluster you will have to adjust your steps to avoid an outage.
I am no longer seeing "plugin" in the function deny list with these steps and the sample config I pulled from your previous comments on this issue. Let me know if that helps at all. Also, if you have problems deleting the data directory on the clients because the alloc dir or it's subdirs are mounted, you can try this following script. It also assumes the data dir is
|
@bubejur I just remembered you included a |
same on Nomad v1.2.6-dev (89b4f3c)
but i didn't restart and update server side, will do it later in the evening... |
Thanks. Also really important to delete the data directories or you might end up with a copy of the job that has the template already rendered. If that still fails for any reason, could you post your full client config minus any secrets? That would be really helpful. Thanks for working through this with me! |
Yes, i read it and almost deleted any client info before nomad update. Here is client config:
|
Was that a typo? You have |
Also, it's worth pointing out, that you don't actually want an empty string in the array. We've discussed it internally, and it should have rejected the empty string as invalid config, but since it hasn't in the past, we won't change that for now so as not to break backward compatibility. Here's an example f you don't want to deny any functions. "template": {
"function_denylist": []
} |
Nomad v1.2.6-dev (89b4f3c) @DerekStrickland Victory! |
How can i get new rpm with this fix in your repo? |
Once it's merged into main it will be in the next release. We don't produce packages except for official releases. Sorry for the delay |
@DerekStrickland can you also somehow add our ips to whitelist? 88.218.113.16 88.218.112.16 178.57.66.244 - it's not Russian ips. We are from luxembourg, amsterdam and finland. |
@tgross maybe you can tell me a date for new 1.2.7 release? so excited about it! |
@bubejur unfortunately we can't give out dates for releases. I can tell you the whole team is actively working on 1.3.0 (which will have backports to 1.2.7). |
I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues. |
@notnoop @tgross hi guys! I made an update for 1.2.4 but got another issue with consul templating:
Nomad side:
Originally posted by @bubejur in #11547 (comment)
The text was updated successfully, but these errors were encountered: