Nomad Client Node Simulator
To install and use you must have Go 1.19+ and run:
go install -v github.com/schmichael/nomad-nodesim@latest
nomad-nodesim -help
See nodesim.nomad
for an example jobspec for running nodesim in a Nomad
cluster using Docker.
See terraform
for some of the worst Packer and Terraform code you have ever
seen. Should provision a working Nomad cluster (in AWS us-west-2) to run
nodesim.nomad
.
- Disable limits! Otherwise you won't scale far: https://developer.hashicorp.com/nomad/docs/configuration#limits
- If Consul is running it may try to perform 1 health check per virtual node against the real nodes which... can get a little intense.
To simulate clusters orders of magnitude larger than the number of physical machines.
Also a useful exploration of how pluggable the client is.
- Hundreds of clients per process.
- Client restarting.
Features that aren't support today but could be:
- Replace
Config.RPCHandler
with a simulated network stack - Server mode
- Add universal "?node_id=..." HTTP API support upstream to route the RPCs to individual Client instances.
Features that probably can't be supported:
- Consul - Consul assumes a Consul agent per Nomad agent per Machine.
The nomad-nodesim
application supports providing configuration parameters via CLI flags and a
config file in HCL format.
-
alloc-runner-type
("sim"
) - Defines what alloc-runnernomad-nodesim
will use. It supports either the basic simulated runner included within this application, or the "real" one pulled directly from Nomad. Supports"sim"
or"real"
. -
config
(""
) - The path to a config file to load. -
log-include-location
(false
) - Whether thenomad-nodesim
logs should include file and line information. -
log-json
(false
) - Whether thenomad-nodesim
logs should be output in JSON format. -
log-level
("debug"
) - The verbosity level of thenomad-nodesim
logs. This currently supports"trace"
,"debug"
,"info"
,"warn"
, and"error"
. -
node-name-prefix
(""
) - The prefix used to name each Nomad client. When not supplied, this will be generated using a concatenation ofnode-
and a generated short UUID such asnode-8a48e733
. -
node-num
(1
) - The number of Nomad clients that will be started within thenomad-nodesim
application. -
server-addr
(["127.0.0.1:4647"]
) This flag can be supplied multiple times and specifies the Nomad server RPC addresses that the Nomad clients should use for registration and connectivity. -
work-dir
(""
) - The working directory for the application. Each nodesim client will store its data directory within this. When not supplied, this will be placed within the directory it is run from and will be prefixed withnomad-nodesim-
.
-
node_name_prefix
(""
) - The prefix used to name each Nomad client. When not supplied, this will be generated using a concatenation ofnode-
and a generated short UUID such asnode-8a48e733
. -
node_num
(1
) - The number of Nomad clients that will be started within the nomad-nodesim application. -
server_addr
(["127.0.0.1:4647"]
) Specifies a list of Nomad server RPC addresses that the Nomad clients should use for registration and connectivity. -
work_dir
(""
) - The working directory for the application. Each nodesim client will store its data directory within this. When not supplied, this will be placed within the directory it is run from and will be prefixed withnomad-nodesim-
.
-
include-location
(false
) - Whether thenomad-nodesim
logs should include file and line information. -
json
(false
) - Whether thenomad-nodesim
logs should be output in JSON format. -
level
("debug"
) - The verbosity level of thenomad-nodesim
logs. This currently supports"trace"
,"debug"
,"info"
,"warn"
, and"error"
.
-
datacenter
- ("global"
) - Specifies the data center of thenomad-nodesim
clients. -
node_class
- (""
) - Specifies an arbitrary string used to logically groupnomad-nodesim
clients. -
node_pool
- ("default"
) - Specifies the node pool in whichnomad-nodesim
clients are registered. -
region
- ("dc1"
) - Specifies the region of thenomad-nodesim
clients. -
resources
- (block) - The CPU and Memory configuration that will be given to the simulated node.-
cpu_compute
- (10_000
) - The CPU value that the simulated node will be configured with and will represent the total allocatable CPU of the client. -
memory_mb
- (10_000
) - The memory MB value that the simulated node will be configured with and will represent the total allocatable memory of the client.
-
-
options
- ("map[string]string"
) - Specifies a key-value mapping of internal configuration fornomad-nodesim
clients, such as for driver configuration.
The example below demonstrates setting config parameter within a configuration file.
work_dir = "/tmp/nomad-nodesim/"
node_name_prefix = "node-8a48e733"
server_addr = ["127.0.0.1:4647"]
node_num = 100
log {
level = "info"
json = true
include_location = true
}
node {
region = "kent-1"
datacenter = "fav"
node_pool = "default"
node_class = "high_mem"
options = {
"fingerprint.denylist" = "env_aws,env_gce,env_azure,env_digitalocean"
}
}