Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reverse Notification for health check failures, when used as a side car for an orchestrator? #9045

Closed
rmohta opened this issue Nov 15, 2019 · 2 comments

Comments

@rmohta
Copy link

rmohta commented Nov 15, 2019

New Feature Request
Envoy being a popular choice as a side-car in microservice architecture - I was wondering if it's possible to send reverse notification to an app when a pod/instance fails health check and is removed from the cluster? Is it possible to achieve this via extension?

I'll expand with below scenarios below

Scenario: When Envoy is used as a side car for an Orchestrator (like Netflix Conductor)

Orchestrator and envoy are deployed as a single pod in Kubernetes. Envoy is configured to perform health check of services (all the services are deployed in same k8s cluster). In a situation, a particular core service starts failing envoy health check (or k8s readiness probe), then the platform might want the orchestrator to take itself out of rotation for the load-balancer (F5 or AWS Load Balancer).

A workaround I use, is to read failed health check logs + poll Envoy admin API for cluster health and act upon it.

What I would expect?

When envoy has less than N healthy connections for a cluster, it would call a POST endpoint (could do gRPC too) with details

  • cluster info + any other metadata we have configured
  • number of healthy connections envoy has
  • min number of configured healthy connections
  • total number of connections envoy has (healthy + unhealthy)
  • Last Health Check Failure Time

With this info, the destination server can act upon it.

@mattklein123
Copy link
Member

I think this is now a dup of #8970?

@rmohta
Copy link
Author

rmohta commented Nov 16, 2019

Agree. Will close this issue, and follow #8970
Thank You @mattklein123!

@rmohta rmohta closed this as completed Nov 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants