-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement raft multipler flag #8082
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -141,6 +141,25 @@ server { | |
features and is typically not required as the agent internally knows the | ||
latest version, but may be useful in some upgrade scenarios. | ||
|
||
- `raft_multiplier` `(int: 1)` - An integer multiplier used by Nomad servers to | ||
scale key Raft timing parameters. Omitting this value or setting it to 0 uses | ||
default timing described below. Lower values are used to tighten timing and | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "described below" - we don't describe it below. I don't think this documentation from Consul works well for us because they set their If it's ok that Nomad cannot be tuned to be more sensitive (we're really locking ourselves in for the long term with this design), and I think that's ok, then maybe the docs should read:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks for catching this. I'll follow up. To clarify, Nomad's raft_multiplier=1 matches Consul's raft_multiplier=1 - i.e. nomad's minimum is consul's minimum. When Consul introduced raft_multipler flag in 0.7, they additionally changed the default timeouts to make it looser to accommodate dev instances - here, I didn't change the default, hence it stays at 1 - the equivalent of pre-0.12 values. |
||
increase sensitivity while higher values relax timings and reduce sensitivity. | ||
Tuning this affects the time it takes Nomad to detect leader failures and to | ||
perform leader elections, at the expense of requiring more network and CPU | ||
resources for better performance. The maximum allowed value is 10. | ||
|
||
By default, Nomad will use the highest-performance timing, currently equivalent | ||
to setting this to a value of 1. Increasing the timings makes leader election | ||
less likely during periods of networking issues or resource starvation. Since | ||
leader elections pause Nomad's normal work, it may be beneficial for slow or | ||
unreliable networks to wait longer before electing a new leader. The tradeoff | ||
when raising this value is that during network partitions or other events | ||
(server crash) where a leader is lost, Nomad will not elect a new leader for | ||
a longer period of time than the default. The ['nomad.nomad.leader.barrier' and | ||
`nomad.raft.leader.lastContact` metrics](/docs/telemetry/metrics is a good | ||
indicator of how often leader elections occur and raft latency. | ||
|
||
- `redundancy_zone` `(string: "")` - (Enterprise-only) Specifies the redundancy | ||
zone that this server will be a part of for Autopilot management. For more | ||
information, see the [Autopilot Guide](https://learn.hashicorp.com/nomad/operating-nomad/autopilot). | ||
|
@@ -275,4 +294,4 @@ server { | |
[encryption]: https://learn.hashicorp.com/nomad/transport-security/gossip-encryption 'Nomad Encryption Overview' | ||
[server-join]: /docs/configuration/server_join 'Server Join' | ||
[update-scheduler-config]: /api-docs/operator#update-scheduler-configuration 'Scheduler Config' | ||
[bootstrapping a cluster]: /docs/faq#bootstrapping | ||
[bootstrapping a cluster]: /docs/faq#bootstrapping |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just checking my understanding: we've already parsed ElectionTimeout from the config, so that an operator can combine a specific ElectionTimeout and the multiplier, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Almost but not quite.
ElectionTimeout
value is set earlier from the raft library hardcoded value. WithoutRaftMultipler
, operators don't have direct way to manipulate the value in any way.