[feature]: cli job run retry on "Error fetching deployment"

[resmo]

Proposal

Add an option --retry (default 3) for monitoring the job run status.

Use-cases

We run a reverse proxy as a cetralized entry point for our devs to the nomad API. The proxy itself runs on nomad.

If our CI re-deploys that proxy accessing the API through that proxy, even with canary and rolling update it might be the cli errors out with "Error fetching deployment".

We added a retry on error in our CI, but we wished the CLI job monitoring would just retry a couple of times by itself e.g. 3 times with a delay of a second.

Attempted Solutions

canary and rolling update

[Amier3]

Hey @resmo

This would be a great feature to have so thanks for making this, we'll address this internally and get back to you with our thoughts 👍. Would you be willing to raise a PR for this?

[resmo]

Thanks @Amier3

I am afraid, I don't have much experience in golang.

[bubejur]

Hi all! @DerekStrickland do you have any news about that?

[bubejur]

@DerekStrickland @tgross Hi! Any news?

[tgross]

@bubejur we'll update issues when we're working on them. This isn't currently on our immediate roadmap.

[kaspergrubbe]

This would be helpful for us too! Thanks for suggesting this change.

[vkrebs-wktaa]

We upgraded our CI pipeline to use nomad cli 1.5.5 (from 1.4.4).
With 1.4.4 we could simply call nomad job run <jobfile>.
With 1.5.5 we always get the error "Error fetching deployment" after a couple of seconds.
I guess we have to implement a retry logic, beacuse the pipeline always fails now.

So ... this feature would be much appreciated.

[josh-m-sharpe]

Doubling down on @vkrebs-wktaa comment. I just upgraded from 1.3.x to 1.5.6 and now I see this error almost every time I nomad job run. Seems like a major regression.

[lgfa29]

Hi all 👋

Just noting here that, while investigating #17320, I noticed that the deployment monitor was not outputting the actual error that happened, which makes it hard to understand what problem is happening, so I've opened #17348 to improve this.

A retry mechanism would indeed be helpful, so I'm keeping this one opened.

[Lord-Y]

Any news about this?

[resmo]

As this software no longer has an osi approved license, closing...