-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve logging on Connect proxy error #12389
Comments
Heya @mikenomitch ! I tried a couple things to surface more information about the nature of bootstrap errors, and ran into d476780, released in The conundrum is that on the Nomad side, we have little/no control over the output of the bootstrap command, and trying to parse it out into something that fits well in a Task Event seems pretty fraught. For example, to induce an error I added a non-existent flag to the
and a change similar to your suggestion yielded the entire "Usage" output in the error, which gets emitted as a task event that takes up the whole page in Nomad UI. example output comparisonbefore, currently very little info:
after, notice the side scroll:
🤣 We could filter out On the Consul side, feasibly hashicorp/consul#9724 could improve the situation. It would also just be nice if we didn't need to run consul CLI for this bootstrap process. |
Hi again @mikenomitch :) I chatted with the team about this some more, and I'm wondering how this sounds to you. Would it help if we added some documentation describing the process laid out in d476780 and included a link to that doc in the error that gets surfaced as a task event? So in addition to
the message presented to the user could include a link like
How does that strike you? |
@gulducat and I had a chat about this offline. Given that the error messages from Consul aren't great to surface (see above comment). We probably aren't going to add the code change as is. @gulducat is going to document the steps laid out here so this will be more avoidable in the future, then we'll close this out. If an endpoint is added for envoy bootstrapping, then we can switch to that and surface that error for an eventual fix. |
Proposal
When there is an error creating the bootstrap configuration for Connect proxy sidecar, the error is not properly surfaced. Currently there is no indication of what happened when Envoy fails to start allocations
Use-cases
Ideally Nomad could pass this error through to the end user so they could better diagnose and fix the issue.
For instance, an ACL mismatch in Consul and/or notifications on expired Certificates would be logged/surfaced.
Attempted Solutions
The following code is used to surface the bug in a local Nomad fork:
The text was updated successfully, but these errors were encountered: