Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Alerting Http Client #671

Closed
alexandreLamarre opened this issue Oct 15, 2022 · 3 comments · Fixed by #667 or #663
Closed

Fix Alerting Http Client #671

alexandreLamarre opened this issue Oct 15, 2022 · 3 comments · Fixed by #667 or #663
Assignees
Labels
alerting bug Something isn't working
Milestone

Comments

@alexandreLamarre
Copy link
Contributor

alexandreLamarre commented Oct 15, 2022

  • Fix the HTTP alert client struct
  • Fix the controller backoff logic
  • Make API pipelines when needing to call multiple async apis that can depend on each other
@alexandreLamarre
Copy link
Contributor Author

Made the required enhancements, they are now used by alerting.endpoint.proto / alerting.condition.proto

@alexandreLamarre
Copy link
Contributor Author

TODO, need to fix :

  • the internal trigger api in api.go still uses the old way of using the client, need to fix that
  • the tests use the old api client, so they either will fail or failed to compile

@alexandreLamarre
Copy link
Contributor Author

alexandreLamarre commented Oct 17, 2022

Still can't fix the pre-existing "race condition" with /test api that triggers an alert with the given credentials where it may not trigger the alert

The crux of the issue is comparing the output of /api/v2/status to the newly applied config in the at the end of the AlertManager reload API pipeline

(The new client is more robust to failure but exacerbates the stuttering that can occur during reload)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment