Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

batch jobs causes error message in stats_hook #6073

Closed
42wim opened this issue Aug 5, 2019 · 9 comments · Fixed by #8028
Closed

batch jobs causes error message in stats_hook #6073

42wim opened this issue Aug 5, 2019 · 9 comments · Fixed by #8028

Comments

@42wim
Copy link
Contributor

42wim commented Aug 5, 2019

Nomad version

0.9.4

Issue

Batch jobs causes ERROR message in stats_hook (even when they exit clean)

Aug 05 17:13:18 q-cloud nomad[125584]: 2019-08-05T17:13:18.444+0200 [ERROR] client.alloc_runner.task_runner.task_hook.stats_hook: failed to start stats collection for task with unrecoverable error: alloc_id=58b67839-0704-081e-b05c-3e39898d8461 task=batch-job error="container stopped"
Aug 05 17:13:18 q-cloud nomad[125584]: 2019-08-05T17:13:18.470+0200 [INFO ] client.alloc_runner.task_runner: not restarting task: alloc_id=58b67839-0704-081e-b05c-3e39898d8461 task=batch-job reason="Restart unnecessary as task terminated successfully"

@42wim
Copy link
Contributor Author

42wim commented Sep 28, 2019

same on 0.9.5

@msvbhat
Copy link
Contributor

msvbhat commented Oct 7, 2019

I'm seeing same error messages in 0.9.5. From what I can tell, it looks like a harmless message and may be it should not be logged as error?

@tgross
Copy link
Member

tgross commented Oct 7, 2019

It looks like this happens for all tasks, but it's just a lot more noticeable for batch jobs because they terminate as part of their regular operation. I agree with you @msvbhat that this is most likely over-zealous logging.

@erhlee-bird
Copy link

Confirming that this still happens.

  • Nomad version: v0.10.2
  • Parameterized Batch Jobs
  • Docker Driver

@tgross
Copy link
Member

tgross commented Jan 10, 2020

Thanks @erhlee-bird. It definitely is a case where we're logging something that doesn't really need to be logged, especially as ERROR because it's going to happy literally every time an alloc stops. I've marked this issue for cleanup, but happy to take a PR if you have time to contribute too!

@tgross tgross added this to the unscheduled milestone Jan 10, 2020
@stale
Copy link

stale bot commented Apr 9, 2020

Hey there

Since this issue hasn't had any activity in a while - we're going to automatically close it in 30 days. If you're still seeing this issue with the latest version of Nomad, please respond here and we'll keep this open and take another look at this.

Thanks!

@tgross
Copy link
Member

tgross commented Apr 9, 2020

I still want to fix this error message and haven't gotten around to it. I think adding the enhancement label will get stalebot to chill.

@tgross
Copy link
Member

tgross commented May 20, 2020

Sorry about the delay on such a trivial item. Will ship in 0.11.3.

@tgross tgross modified the milestones: unscheduled , 0.11.3 May 20, 2020
@github-actions
Copy link

github-actions bot commented Nov 7, 2022

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 7, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants