Eureka will disable expiration forever in some cases?

[neoremind]

If any time, the heartbeat from all clients, a.k.a renewals falls below a percentage which by default configured to 85% in 15min, the server stops expiring instances. This will happen usually by either kill -9 on client or network unreachable.
For example, there are 50 endpoints, and eureka expects 100 heartbeats per min.
If due to some reason, the heartbeats drops to 80 which is below 85, and eureka will disable expiration, so there are 10 endpoints (should be 20 heartbeats) unavailable.
But what if this is a permanent state? People just kill -9 on the 10 endpoints to make them offline? Eureka server will hold these unavailable endpoints for as long as possible because expiration is disabled. Is this case make sense?

[holy12345]

Hi @neoremind
As far as i know , When the eureka client Down it will sent a message to Eureka Server, so Eureka Server can know its down, also Eureka server can recalculate numbers. If you use kill -9 The Client will not send message to Eureka Server. So at this time The Eureka Server not recalculate numbers.

But Eureka server have a task in background, it will evict client which not sent heartbeat.

If you have any question, Maybe can we use Chinese to communication : )

I hope i can help you!

thanks again

[neoremind]

@holy12345 The background task is to evict those ones do not send heartbeats as regular basis, but if expiration is disabled, the background task is not able to perform, so those unavailable endpoints will stay if eureka does not enable expiration.

[qiangdavidliu]

Hi. Thanks for the questions, let me try to address these.

1. How does eureka clients 'end' their registration lifecycle?

• the eureka protocol requires clients to execute an explicit unregister action when they are permanently going away. For example, in the provided java client, this is done in the shutdown() method.
• any clients that fails 3 consecutive heartbeats is considered to have an unclean termination, and will be evicted by the background eviction process.

2. What happens when too many unclean terminations happen and trigger self preservations?

• this is typically a rare case and we expect that this happen because of some failure mode. The expectation here is that once recovery has happened, the number of heartbeating clients will go back to be above the expected threshold and the servers will automatically recover from self preservation state.
• if however this is expected (or after recovery the number of heartbeating clients does not go back to be above the threshold), you can force the servers to be out of self preservation mode by temporarily disabling self preservation. We would expect that in these cases, a human operation is available to evaluate the situation and take the appropriate action.

[neoremind]

@qiangdavidliu Thanks for your answer. What you explained makes sense. Abnormal offline instances should be considered as failure endpoints and expected to be back, normal offline is what Eureka expects clients to do.

In our system, we make self preservation to be dynamic configurable, so that we can change the behavior in runtime.