Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

busy runner pod is deleted by recreate feature #136

Open
4 tasks
masa213f opened this issue Jan 31, 2022 · 0 comments
Open
4 tasks

busy runner pod is deleted by recreate feature #136

masa213f opened this issue Jan 31, 2022 · 0 comments

Comments

@masa213f
Copy link
Contributor

What

If the timing is bad, the recreate feature will delete busy runner pods.

2022-01-31 10:27:48    {"level":"info","ts":1643592468.2755053,"logger":"controllers.RunnerManager","msg":"deleted runner pod that exceeded recreate deadline","runnerpool":"<Namespace>/<RunnerPool>","pod":"<Namespace>/<RunnerPod>"}
2022-01-31 10:29:48    {"level":"error","ts":1643592588.5469356,"logger":"controllers.RunnerManager","msg":"failed to remove runner","runnerpool":"<Namespace>/<RunnerPool>","runner":"<Runner>","runner_id":59,"error":"DELETE https://api.github.com/repos/***/***/actions/runners/NN: 422 Bad request - Runner \"<Runner>\" is still running a job\" []","stacktrace":"github.com/cybozu-go/meows/controllers.(*manageProcess).run\n\tgit.luolix.top/cybozu-go/meows/controllers/runner_manager.go:256\ngit.luolix.top/cybozu-go/meows/controllers.(*manageProcess).start.func1\n\tgit.luolix.top/cybozu-go/meows/controllers/runner_manager.go:223\ngit.luolix.top/cybozu-go/well.(*Environment).Go.func1\n\tgit.luolix.top/cybozu-go/well@v1.10.0/env.go:133"}
2022-01-31 10:29:48    {"level":"error","ts":1643592588.5470736,"logger":"controllers.RunnerManager","msg":"failed to run a runner manager process","runnerpool":"<Namespace>/<RunnerPool>","error":"DELETE https://api.github.com/repos/***/***/actions/runners/NN: 422 Bad request - Runner \"<Runner>\" is still running a job\" []","stacktrace":"github.com/cybozu-go/well.(*Environment).Go.func1\n\tgit.luolix.top/cybozu-go/well@v1.10.0/env.go:133"}

How

Describe how to address the issue.

Checklist

  • Finish implementation of the issue
  • Test all functions
  • Have enough logs to trace activities
  • Notify developers of necessary actions
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant