Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change consul SI tokens to be local #8068

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .changelog/8068.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
```release-note:breaking-change
connect: Consul Service Identity ACL tokens automatically generated for Connect services are now
created as Local rather than Global tokens. Nomad clusters with Connect services making cross-Consul
datacenter requests will need to ensure their Consul agents are configured with anonymous ACL tokens
of sufficient node and service read permissions.
```

```release-note:breaking-change
The minimum Consul version supported by Nomad's Connect integration is now Consul v1.8.0.
```
1 change: 1 addition & 0 deletions nomad/consul.go
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,7 @@ func (c *consulACLsAPI) CreateToken(ctx context.Context, sir ServiceIdentityRequ
Description: sir.Description(),
ServiceIdentities: []*api.ACLServiceIdentity{{ServiceName: service}},
Namespace: sir.ConsulNamespace,
Local: true,
}

// Ensure we are under our rate limit.
Expand Down
4 changes: 2 additions & 2 deletions nomad/job_endpoint_hook_connect.go
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,11 @@ func connectGatewayDriverConfig(hostNetwork bool) map[string]interface{} {
// connectSidecarVersionConstraint is used when building the sidecar task to ensure
// the proper Consul version is used that supports the necessary Connect
// features. This includes bootstrapping envoy with a unix socket for Consul's
// gRPC xDS API.
// gRPC xDS API, and support for generating local service identity tokens.
func connectSidecarVersionConstraint() *structs.Constraint {
return &structs.Constraint{
LTarget: "${attr.consul.version}",
RTarget: ">= 1.6.0-beta1",
RTarget: ">= 1.8.0",
Operand: structs.ConstraintSemver,
}
}
Expand Down
26 changes: 22 additions & 4 deletions website/content/docs/integrations/consul-connect.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,6 @@ description: >-

# Consul Service Mesh

~> **Note:** This guide requires Nomad 0.10.0 or later and Consul 1.6.0 or
later.

~> **Note:** Nomad's service mesh integration requires Linux network namespaces.
Consul service mesh will not run on Windows or macOS.

Expand Down Expand Up @@ -58,7 +55,7 @@ The Consul service mesh integration with Nomad requires [Consul 1.6 or
later.](https://releases.hashicorp.com/consul/1.6.0/) The Consul agent can be
run in dev mode with the following command:

**Note**: Nomad's Consul service mesh integration requires Consul in your `$PATH`
~> **Note:** Nomad's Consul service mesh integration requires Consul in your `$PATH`

```shell-session
$ consul agent -dev
Expand Down Expand Up @@ -96,6 +93,23 @@ For JSON configurations:
}
```

#### Consul ACLs

~> **Note:** Starting in Nomad v1.3.0, Consul Service Identity ACL tokens automatically
generated by Nomad on behalf of Connect enabled services are now created in [`Local`]
rather than Global scope, and are no longer replicated globally.

To facilitate cross-Consul datacenter requests of Connect services registered by
Nomad, Consul agents will need to be configured with [default anonymous][anon_token]
ACL tokens with ACL policies of sufficient permissions to read service and node
metadata pertaining to those requests. This mechanism is described in Consul [#7414][consul_acl].
A typical Consul agent anonymous token may contain an ACL policy such as:

```hcl
service_prefix "" { policy = "read" }
node_prefix "" { policy = "read" }
```

### Nomad

Nomad must schedule onto a routable interface in order for the proxies to
Expand Down Expand Up @@ -326,11 +340,15 @@ dashes (`-`) are converted to underscores (`_`) in environment variables so

## Limitations

- The minimum Consul version to use Connect with Nomad is Consul v1.8.0.
- The `consul` binary must be present in Nomad's `$PATH` to run the Envoy
proxy sidecar on client nodes.
- Consul service mesh using network namespaces is only supported on Linux.
- Prior to Consul 1.9, the Envoy sidecar proxy will drop and stop accepting
connections while the Nomad agent is restarting.

[count-dashboard]: /img/count-dashboard.png
[consul_acl]: https://github.com/hashicorp/consul/issues/7414
[gh-9907]: https://github.com/hashicorp/nomad/issues/9907
[`Local`]: https://www.consul.io/docs/security/acl/acl-tokens#token-attributes
[anon_token]: https://www.consul.io/docs/security/acl/acl-tokens#special-purpose-tokens
21 changes: 21 additions & 0 deletions website/content/docs/upgrade/upgrade-specific.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,24 @@ connect {
}
```

#### Changes to Consul Connect Service Identity Tokens

Starting with Nomad 1.3.0, Consul Service Identity Tokens created automatically
by Nomad on behalf of Connect services will now be created as [`Local`] tokens. These
tokens will no longer be replicated globally. To facilitate cross-Consul datacenter
requests of Connect services registered by Nomad, Consul agents will need to be
configured with [default anonymous][anon_token] ACL tokens with ACL policies of
sufficient permissions to read service and node metadata pertaining to those
requests. This mechanism is described in Consul [#7414][consul_acl].
A typical Consul agent anonymous token may contain an ACL policy such as:

```hcl
service_prefix "" { policy = "read" }
node_prefix "" { policy = "read" }
```

The minimum version of Consul supported by Nomad's Connect integration is now Consul v1.8.0.

#### Linux Control Groups Version 2

Starting with Nomad 1.3.0, Linux systems configured to use [cgroups v2][cgroups2]
Expand Down Expand Up @@ -1376,3 +1394,6 @@ deleted and then Nomad 0.3.0 can be launched.
[cap_drop_exec]: /docs/drivers/exec#cap_drop
[`log_file`]: /docs/configuration#log_file
[Upgrading to Raft Protocol 3]: /docs/upgrade#upgrading-to-raft-protocol-3
[`Local`]: https://www.consul.io/docs/security/acl/acl-tokens#token-attributes
[anon_token]: https://www.consul.io/docs/security/acl/acl-tokens#special-purpose-tokens
[consul_acl]: https://github.com/hashicorp/consul/issues/7414