Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Connect scheduling #10700

Closed
shoenig opened this issue Jun 3, 2021 · 4 comments · Fixed by #10702
Closed

Improve Connect scheduling #10700

shoenig opened this issue Jun 3, 2021 · 4 comments · Fixed by #10702
Assignees
Labels
stage/accepted Confirmed, and intend to work on. No timeline committment though. theme/consul/connect Consul Connect integration type/enhancement
Milestone

Comments

@shoenig
Copy link
Member

shoenig commented Jun 3, 2021

Currently Nomad only checks for the presense of Consul at a minimum version number before scheduling Connect enabled tasks.

In reality, Connect must be explicitly enabled on each Consul agent, and the gRPC port must be set to active the grpc listener for the agent. This is been a source of pain and confusion for users who are missing these config values in Consul, which in turn appears like Nomad problems when their Connect sidecars fail to make contact with Consul.

Using the consul.connect and consul.grpc node attributes introduced in #10699, add additional constraints for Connect tasks so they are only provisioned on nodes where Connect is actually enabled.

@shoenig shoenig added type/enhancement theme/consul/connect Consul Connect integration stage/accepted Confirmed, and intend to work on. No timeline committment though. labels Jun 3, 2021
@shoenig shoenig self-assigned this Jun 3, 2021
shoenig added a commit that referenced this issue Jun 3, 2021
This PR adds two additional constraints on Connect sidecar and gateway tasks,
making sure Nomad schedules them only onto nodes where Connect is actually
enabled on the Consul agent.

Consul requires `connect.enabled = true` and `ports.grpc = <number>` to be
explicitly set on agent configuration before Connect APIs will work. Until
now, Nomad would only validate a minimum version of Consul, which would cause
confusion for users who try to run Connect tasks on nodes where Consul is not
yet sufficiently configured. These contstraints prevent job scheduling on nodes
where Connect is not actually use-able.

Closes #10700
shoenig added a commit that referenced this issue Jun 3, 2021
This PR adds two additional constraints on Connect sidecar and gateway tasks,
making sure Nomad schedules them only onto nodes where Connect is actually
enabled on the Consul agent.

Consul requires `connect.enabled = true` and `ports.grpc = <number>` to be
explicitly set on agent configuration before Connect APIs will work. Until
now, Nomad would only validate a minimum version of Consul, which would cause
confusion for users who try to run Connect tasks on nodes where Consul is not
yet sufficiently configured. These contstraints prevent job scheduling on nodes
where Connect is not actually use-able.

Closes #10700
shoenig added a commit that referenced this issue Jun 3, 2021
This PR adds two additional constraints on Connect sidecar and gateway tasks,
making sure Nomad schedules them only onto nodes where Connect is actually
enabled on the Consul agent.

Consul requires `connect.enabled = true` and `ports.grpc = <number>` to be
explicitly set on agent configuration before Connect APIs will work. Until
now, Nomad would only validate a minimum version of Consul, which would cause
confusion for users who try to run Connect tasks on nodes where Consul is not
yet sufficiently configured. These contstraints prevent job scheduling on nodes
where Connect is not actually use-able.

Closes #10700
shoenig added a commit that referenced this issue Jun 3, 2021
This PR adds two additional constraints on Connect sidecar and gateway tasks,
making sure Nomad schedules them only onto nodes where Connect is actually
enabled on the Consul agent.

Consul requires `connect.enabled = true` and `ports.grpc = <number>` to be
explicitly set on agent configuration before Connect APIs will work. Until
now, Nomad would only validate a minimum version of Consul, which would cause
confusion for users who try to run Connect tasks on nodes where Consul is not
yet sufficiently configured. These contstraints prevent job scheduling on nodes
where Connect is not actually use-able.

Closes #10700
@kds-rune
Copy link

I know this is closed, but the consul documentation states the following:

"The first step to use Connect is to enable Connect for your Consul cluster. By default, Connect is disabled. Enabling Connect requires changing the configuration of only your Consul servers (not client agents). To enable Connect, add the following to a new or existing server configuration file."

I now had to reconfigure all clients with this setting in the consul client-configuration, otherwise they got filtered by nomad to connect.enabled-constraint. I couldn't find any reference in the consul-documentation stating this should be set on client nodes, so I am still a bit confused ;)

@tgross
Copy link
Member

tgross commented Jun 10, 2021

Hi @kds-rune! Let's open a new issue to discuss documentation concerns, or post on Discuss. Thanks!

@runeron
Copy link

runeron commented Jun 10, 2021

@tgross tgross added this to the 1.1.1 milestone Jun 15, 2021
@github-actions
Copy link

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
stage/accepted Confirmed, and intend to work on. No timeline committment though. theme/consul/connect Consul Connect integration type/enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants