Enable ACL Client Token #1093

jmurret · 2022-03-10T22:35:28Z

This PR is best viewed Commit by Commit to see the description/purpose and what got added.

Changes proposed in this PR:

control-plane/subcommand/acl-init/command.go
- when using ACLs, retrieve the token secret from consul login rather than k8s secrets
- when external servers are configured, use them for the cases of partition set up
control-plane/subcommand/common/common.go
- modify ConsulLogin() to also return the token (consul core code does this as well), in addition to storing it in the tokenSinkFile, so that it can be used in ACL init to write out the
- modify ConsulLogin()to write the tokenSinkFile with 0644 permissions that was original used before ACLs refactor, in part to support [this bug fix] (https://github.com/hashicorp/consul-k8s/pull/248/files#diff-29d4f970a3786e8c721fe5472dc71f9f94482513a3e6042f03c7481e6cec2195L108).
  acl-config.json file that is also used in the k8s secrets workflow
control-plane/subcommand/connect-init/command.go - minor changes to support return type change toConsulLogin
control-plane/subcommand/partition-init/command.go - refactor touse existing tls-server-name flag.
control-plane/subcommand/server-acl-init/create_or_update.go - make change to create BindingRule if it does not exist rather than error when there are other existing BindingRules associated with the auth method
control-plane/subcommand/server-acl-init/command.go
- change -create-client-token flag to -client flag
- create consul policy and binding role rather than a k8s secret so it can be used to authenticate client-acl-init when it calls ConsulLogin().

How I've tested this PR:

Acceptance Tests
Unit Tests

How I expect reviewers to test this PR:
👀

Checklist:

Tests added
CHANGELOG entry added

hashicorp-cla · 2022-03-11T21:16:04Z

All committers have signed the CLA.

jmurret · 2022-03-14T17:13:23Z

charts/consul/templates/_helpers.tpl

@@ -48,7 +48,7 @@ as well as the global.name setting.
 {{- define "consul.serverTLSAltNames" -}}
 {{- $name := include "consul.fullname" . -}}
 {{- $ns := .Release.Namespace -}}
-{{ printf "localhost,%s-server,*.%s-server,*.%s-server.%s,*.%s-server.%s.svc,*.server.%s.%s" $name $name $name $ns $name $ns (.Values.global.datacenter ) (.Values.global.domain) }}{{ include "consul.serverAdditionalDNSSANs" . }}
+{{ printf "localhost,%s-server,*.%s-server,*.%s-server.%s,*.%s-server.%s.svc,%s-server.%s.svc,*.server.%s.%s" $name $name $name $ns $name $ns $name $ns (.Values.global.datacenter ) (.Values.global.domain) }}{{ include "consul.serverAdditionalDNSSANs" . }}


Modifying SANs in cert so client pod can talk to server load balancer.

should we also include %s-server.%s and %s-server for completeness?

Those are the 2nd and 4th entries above.

Oh good point! I didn't notice the 2nd entry. The 4th one though is *.%s-server.%s and I'm wondering about %s-server.%s (i.e. without a wildcard).

👍 Gotcha. Coming up.

Just pushed it up.

jmurret · 2022-03-14T17:13:40Z

charts/consul/templates/tls-init-job.yaml

@@ -78,6 +78,7 @@ spec:
                -additional-dnsname="*.{{ template "consul.fullname" . }}-server" \
                -additional-dnsname="*.{{ template "consul.fullname" . }}-server.${NAMESPACE}" \
                -additional-dnsname="*.{{ template "consul.fullname" . }}-server.${NAMESPACE}.svc" \
+                -additional-dnsname="{{ template "consul.fullname" . }}-server.${NAMESPACE}.svc" \


Modifying SANs in cert so client pod can talk to server load balancer.

jmurret · 2022-03-14T17:15:47Z

charts/consul/templates/client-daemonset.yaml

@@ -47,6 +48,7 @@ spec:
      annotations:
        {{- if .Values.global.secretsBackend.vault.enabled }}
        "vault.hashicorp.com/agent-inject": "true"
+        "vault.hashicorp.com/agent-init-first": "true"


Need vault agent injector to fun before client-acl-init initContainer.

jmurret · 2022-03-14T17:16:42Z

charts/consul/templates/client-daemonset.yaml

@@ -135,6 +140,13 @@ spec:
            items:
            - key: {{ default "tls.crt" .Values.global.tls.caCert.secretKey }}
              path: tls.crt
+          {{- else }}
+          emptyDir:


when using vault, the cert will come form a temp file, not a k8s secret.

jmurret · 2022-03-14T17:17:24Z

charts/consul/templates/client-daemonset.yaml

          env:
+            {{- if .Values.global.acls.manageSystemACLs }}


need this also for the logout command in the preStop above.

jmurret · 2022-03-14T17:20:06Z

charts/consul/templates/client-daemonset.yaml

+              {{- end }}
+              -log-level={{ default .Values.global.logLevel .Values.client.logLevel }} \
+              -log-json={{ .Values.global.logJSON }} \
+              {{- if .Values.externalServers.enabled }}


this follows prior art of partition-init-job and tls-init-job. In a partition scenario, the client container needs to talk directly to the servers in another DC and external servers will be configured. When configured, client-acl-init should use them.

jmurret · 2022-03-14T17:22:09Z

charts/consul/templates/partition-init-job.yaml

@@ -86,10 +86,10 @@ spec:
                {{- if .Values.global.tls.enabled }}
                -use-https \
                {{- if not .Values.externalServers.useSystemRoots }}
-                -consul-ca-cert=/consul/tls/ca/tls.crt \
+                -ca-file=/consul/tls/ca/tls.crt \


These two flags are already available in c.http so we will use those instead of having two similar, unnecessary flags.

jmurret · 2022-03-14T17:22:53Z

charts/consul/templates/server-acl-init-job.yaml

@@ -244,7 +244,7 @@ spec:
                {{- end }}

                {{- if not (or (and (ne (.Values.client.enabled | toString) "-") .Values.client.enabled) (and (eq (.Values.client.enabled | toString) "-") .Values.global.enabled)) }}
-                -create-client-token=false \
+                -client=false \


Changing flag to the new flag convention.

jmurret · 2022-03-14T17:23:21Z

charts/consul/test/unit/partition-init-job.bats

@@ -91,7 +91,7 @@ load _helpers
  actual=$(echo $command | jq -r '. | any(contains("-use-https"))' | tee /dev/stderr)
  [ "${actual}" = "true" ]

-  actual=$(echo $command | jq -r '. | any(contains("-consul-ca-cert=/consul/tls/ca/tls.crt"))' | tee /dev/stderr)


This flag already existed, so we will use it instead.

jmurret · 2022-03-14T17:25:17Z

control-plane/subcommand/acl-init/command.go

+			// For all of the next operations we'll need a Consul client.
+			serverAddr := fmt.Sprintf("%s:%d", serverAddresses[0], c.flagServerPort)
+			cfg.Address = serverAddr
+			cfg.Scheme = scheme


This whole added block is taken from prior art in partition-init command. basically an external server(s) has been passed in and we want to talk to that and not the default endpoint.

control-plane/subcommand/acl-init/command.go

jmurret · 2022-03-14T17:27:39Z

control-plane/subcommand/common/common.go

+			// Write out the resultant token file.
+			// Must be 0644 because this is written by the consul-k8s user but needs
+			// to be readable by the consul user
+			if err := WriteFileWithPerms(tokenSinkFile, tok.SecretID, 0644); err != nil {


0644 was original used before ACLs refactor, in part to support [this bug fix] (https://github.com/hashicorp/consul-k8s/pull/248/files#diff-29d4f970a3786e8c721fe5472dc71f9f94482513a3e6042f03c7481e6cec2195L108).

jmurret · 2022-03-14T17:28:50Z

control-plane/subcommand/common/common.go

@@ -88,15 +88,15 @@ func ValidateUnprivilegedPort(flagName, flagValue string) error {

 // ConsulLogin issues an ACL().Login to Consul and writes out the token to tokenSinkFile.
 // The logic of this is taken from the `consul login` command.
-func ConsulLogin(client *api.Client, cfg *api.Config, authMethodName, datacenter, namespace, bearerTokenFile, serviceAccountName, tokenSinkFile string, meta map[string]string, log hclog.Logger) error {
+func ConsulLogin(client *api.Client, cfg *api.Config, authMethodName, datacenter, namespace, bearerTokenFile, serviceAccountName, tokenSinkFile string, meta map[string]string, log hclog.Logger) (*api.ACLToken, error) {


change signature to return token so that it can be used directly from a ConsulLogin call and we don't have to use i/oto read it from the sink file if not necessary.

jmurret · 2022-03-14T17:29:30Z

control-plane/subcommand/connect-init/command.go

@@ -123,7 +123,7 @@ func (c *Command) Run(args []string) int {
 	if c.flagACLAuthMethod != "" {
 		// loginMeta is the default metadata that we pass to the consul login API.
 		loginMeta := map[string]string{"pod": fmt.Sprintf("%s/%s", c.flagPodNamespace, c.flagPodName)}
-		err = common.ConsulLogin(consulClient, cfg, c.flagACLAuthMethod, "", c.flagAuthMethodNamespace, c.flagBearerTokenFile, c.flagServiceAccountName, c.flagACLTokenSink, loginMeta, c.logger)
+		_, err = common.ConsulLogin(consulClient, cfg, c.flagACLAuthMethod, "", c.flagAuthMethodNamespace, c.flagBearerTokenFile, c.flagServiceAccountName, c.flagACLTokenSink, loginMeta, c.logger)


byproduct of refactoring ConsulLogin signature

jmurret · 2022-03-14T17:29:52Z

control-plane/subcommand/common/common_test.go

@@ -64,7 +64,7 @@ func TestConsulLogin(t *testing.T) {
 	log, err := Logger("INFO", false)
 	require.NoError(err)
 	client, cfg := startMockServer(t, &counter)
-	err = ConsulLogin(client, cfg, testAuthMethod, "dc1", "", bearerTokenFile, "", tokenFile, testPodMeta, log)
+	_, err = ConsulLogin(client, cfg, testAuthMethod, "dc1", "", bearerTokenFile, "", tokenFile, testPodMeta, log)


Everything in this file is a byproduct of refactoring ConsulLogin signature

jmurret · 2022-03-14T17:31:32Z

control-plane/subcommand/server-acl-init/command.go

 		agentRules, err := c.agentRules()
 		if err != nil {
 			c.log.Error("Error templating client agent rules", "err", err)
 			return 1
 		}

-		err = c.createLocalACL("client", agentRules, consulDC, primary, consulClient)
+		serviceAccountName := c.withPrefix("client")
+		err = c.createACLPolicyRoleAndBindingRule("client", agentRules, consulDC, primaryDC, false, primary, localComponentAuthMethodName, serviceAccountName, consulClient)


Set up the binding rule and policy that later client-acl-init will be authenticated with the BindingRule and get a token.

…are enabled

…ing datacenter logic because of breaking test.

…n its a local token, not global tokens.

…ConsulLogin to return secretID, error instead ok token, error.

* Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error.

* Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Enable snapshot agent configuration to be retrieved from vault * Adding CHANGELOG entry * Changing the decoding of snapshot agent config in vault to platform agnostic * Fixing the way we write the encoded vault secret out to a decoded json file * Decoding vault secret using consul template function on the vault annotation. Able to remove the bash that decodes the file and changes the extension. * Update CHANGELOG.md Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update charts/consul/values.yaml Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update charts/consul/values.yaml Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update charts/consul/values.yaml Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * PR Feedback - change client-snapshot-deployment to only have one vault role entry even when needing to set to vault roles * PR Feedback - when both snapshot agent and ca roles are specified in vault, it should get the sa role. * Simplifying conditional for vault role. Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com>

* Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error.

…Snapshot agent acceptance tests (#1125) * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Enable snapshot agent configuration to be retrieved from vault * Adding CHANGELOG entry * Changing the decoding of snapshot agent config in vault to platform agnostic * Fixing the way we write the encoded vault secret out to a decoded json file * Decoding vault secret using consul template function on the vault annotation. Able to remove the bash that decodes the file and changes the extension. * Adding an acceptance test for snapshot agent. It currently fails because of a bug with Consul where it does not recognize CONSUL_HTTP_TOKEN. Will need to refactor test to bootstrap, then create vault secret with embedded acl token, then helm upgrade to add snapshot agent. Then assert that a *.snap file is created. * Adding acceptance test for snapshot agent on vault. * renaming test and removing extra file * Move vault test helpers into framework folder so we can use it more easily from other folders. * Adding snapshot agent test for k8s secret * Adding ability to set initial_management token when using k8s secrets. Also working acceptance test for snapshot agent on k8s secrets. * Adding bats tests. Adding envvar for ACL_BOOTSTRAP_TOKEN. Removing volume and volume mounts for bootstrap token. * Adding CHANGELOG entry for ability to pre-set bootstrap ACL token * Fixing bats tests * Update acceptance/framework/consul/helm_cluster.go Co-authored-by: Thomas Eckert <teckert@hashicorp.com> * Fixing broken unit tests * Lowering snapshot interval from 1mto15s for tests * Update acceptance/framework/consul/helm_cluster.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Update acceptance/framework/vault/helpers.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Update acceptance/tests/snapshot-agent/snapshot_agent_vault_test.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * PR Feedback - clarify comments on Vault helper functions * PR Feedback - clarify comments on Vault helper functions * Modifying tests to not incidentally send an encoded file * Removing logging token in acceptance test code. Co-authored-by: Thomas Eckert <teckert@hashicorp.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>

* Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Enable terminating gateway policy to be generated via Auth Method * Filtering out failing portion of test for terminating gateway work * PR feedback.Fixing tests. Changing naming conventions for policy and roles for terminating gateways. * Changing terminating gateway to pass acl-init a -component-name flag in the form of terminating-gateway/RELEASE-NAME-consul-terminating - <component type>/<consul fullname>-<name> * Correcting serviceAccount used on deployment * Making all nameshavea-ingress-gateway * Enable ingress gateway policy to be generated via Auth Method * Making all names have a -ingress-gateway suffix * Removing duplicate test * Update acceptance/tests/ingress-gateway/ingress_gateway_namespaces_test.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com>

* Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error.

…inating and ingress gateways (#1120) * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Enable terminating gateway policy to be generated via Auth Method * Filtering out failing portion of test for terminating gateway work * PR feedback.Fixing tests. Changing naming conventions for policy and roles for terminating gateways. * Correcting serviceAccount used on deployment * Making all nameshavea-ingress-gateway * Enable ingress gateway policy to be generated via Auth Method * Making all names have a -ingress-gateway suffix * Removing duplicate test * Update acceptance/tests/ingress-gateway/ingress_gateway_namespaces_test.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Removing the gateway type suffix from the naming conventions for terminating and ingress gateways * Adding check for duplicate terminating gateways and ingress gateway names * Update charts/consul/templates/ingress-gateways-deployment.yaml Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com> * PR Feedback - adding the duplicate name found to the check failures for duplicate ingress or terminating gateway names * Fixing rebase conflict * Merge Conflict- duplicate test * Adding a 10s sleep to flakey snapshot agent tests that were not finding a snapshot in time. Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>

* Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error.

…e-configured bootstrap token as k8s secret. (#1128) * Use kube auth method to provision ACL token for the crd controller (#995) * Use a Consul Kubernetes Auth Method to issue consul-login to mint ACL tokens and consul-logout to clean them up for the CRD controller. Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Support storing bootstrap token in Vault (#1061) * Global auth method (#1075) • Update server-acl-init to create authmethods in the primary datacenter when the job is run in a secondary datacenter during federation. This authmethod allows us to issue logins for global policies. • Update the controller workflow in server-acl-init to use this global authmethod when run in a secondary DC. • Update the mesh-gateway acceptance tests to create proxy defaults in the secondary DC to test above behavior works successfully. • Updated logout to not pass in the partition flag as it is not required. • Update server acl init tests to migrate from require := require.New(t) to require.xyz(t, ...) patterns. * Refactor ConnectInject to use authmethods (#1076) Refactor connect-injector to use the new auth-method workflow when ACLs are enabled so that Kubernetes secrets are not used. * Sync token acl refactor (#1081) • Refactor sync-catalog to use the new auth-method workflow when ACLs are enabled so that Kubernetes secrets are not used. • Create a service account and rolebinding dedicated to the component authmethod so that it no longer piggybacks on the one used by the connect-inject authmethod. * rename the controller flag (#1089) * Refactor Consul API Gateway Controller to use AuthMethod workflow. (#1083) * Refactor Consul API Gateway Controller to use AuthMethod workflow. * Refactor snapshot agent to use new acl authmethod workflow (#1084) * refactor snapshot agent to use new acl authmethod workflow. * Refactor mesh-gateway ACL flow (#1085) * Refactor mesh-gateway ACL flow * Fix flakey server-acl-init tests with retries (#1095) * Fix flakey server-acl-init tests with retries * Adding retry for flakey server-acl-init enterprise test * adding missing retry module in server-acl-init enterprise tests * Update Binding Rule if it exists for the authmethod (#1094) * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * vault: add support for admin partitions (#1098) * Refactor common.Login (#1101) * convert function args to a struct * add some missing tests * move logic that is only relevant for connect out * Use bootstrap token from vault to validate exec'ing into consul server (#1116) Follow up on #1103 * Enable terminating gateways to use ACL Auth Method (#1102) * Enable terminating gateway policy to be generated via Auth Method * Filtering out failing portion of test for terminating gateway work * PR feedback.Fixing tests. Changing naming conventions for policy and roles for terminating gateways. * Update control-plane/subcommand/server-acl-init/command.go Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update control-plane/subcommand/server-acl-init/command.go Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Fixing enterprise tests * Changing terminating gateway to pass acl-init a -component-name flag in the form of terminating-gateway/RELEASE-NAME-consul-terminating - <component type>/<consul fullname>-<name> * fixing acceptance test to recognize that long lived tokens will not exist and we ahve to update the role. * Correcting serviceAccount used on deployment * Making all nameshavea-ingress-gateway * Update charts/consul/templates/terminating-gateways-deployment.yaml Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update control-plane/subcommand/server-acl-init/command.go Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update control-plane/subcommand/server-acl-init/command.go Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update control-plane/subcommand/server-acl-init/command.go Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Enable snapshot agent configuration to be retrieved from vault (#1113) * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Enable snapshot agent configuration to be retrieved from vault * Adding CHANGELOG entry * Changing the decoding of snapshot agent config in vault to platform agnostic * Fixing the way we write the encoded vault secret out to a decoded json file * Decoding vault secret using consul template function on the vault annotation. Able to remove the bash that decodes the file and changes the extension. * Update CHANGELOG.md Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update charts/consul/values.yaml Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update charts/consul/values.yaml Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update charts/consul/values.yaml Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * PR Feedback - change client-snapshot-deployment to only have one vault role entry even when needing to set to vault roles * PR Feedback - when both snapshot agent and ca roles are specified in vault, it should get the sa role. * Simplifying conditional for vault role. Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Ability to set initial_management token when using k8s secret store. Snapshot agent acceptance tests (#1125) * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Enable snapshot agent configuration to be retrieved from vault * Adding CHANGELOG entry * Changing the decoding of snapshot agent config in vault to platform agnostic * Fixing the way we write the encoded vault secret out to a decoded json file * Decoding vault secret using consul template function on the vault annotation. Able to remove the bash that decodes the file and changes the extension. * Adding an acceptance test for snapshot agent. It currently fails because of a bug with Consul where it does not recognize CONSUL_HTTP_TOKEN. Will need to refactor test to bootstrap, then create vault secret with embedded acl token, then helm upgrade to add snapshot agent. Then assert that a *.snap file is created. * Adding acceptance test for snapshot agent on vault. * renaming test and removing extra file * Move vault test helpers into framework folder so we can use it more easily from other folders. * Adding snapshot agent test for k8s secret * Adding ability to set initial_management token when using k8s secrets. Also working acceptance test for snapshot agent on k8s secrets. * Adding bats tests. Adding envvar for ACL_BOOTSTRAP_TOKEN. Removing volume and volume mounts for bootstrap token. * Adding CHANGELOG entry for ability to pre-set bootstrap ACL token * Fixing bats tests * Update acceptance/framework/consul/helm_cluster.go Co-authored-by: Thomas Eckert <teckert@hashicorp.com> * Fixing broken unit tests * Lowering snapshot interval from 1mto15s for tests * Update acceptance/framework/consul/helm_cluster.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Update acceptance/framework/vault/helpers.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Update acceptance/tests/snapshot-agent/snapshot_agent_vault_test.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * PR Feedback - clarify comments on Vault helper functions * PR Feedback - clarify comments on Vault helper functions * Modifying tests to not incidentally send an encoded file * Removing logging token in acceptance test code. Co-authored-by: Thomas Eckert <teckert@hashicorp.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Enable ingress gateways to use ACL Auth Method (#1118) * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Enable terminating gateway policy to be generated via Auth Method * Filtering out failing portion of test for terminating gateway work * PR feedback.Fixing tests. Changing naming conventions for policy and roles for terminating gateways. * Changing terminating gateway to pass acl-init a -component-name flag in the form of terminating-gateway/RELEASE-NAME-consul-terminating - <component type>/<consul fullname>-<name> * Correcting serviceAccount used on deployment * Making all nameshavea-ingress-gateway * Enable ingress gateway policy to be generated via Auth Method * Making all names have a -ingress-gateway suffix * Removing duplicate test * Update acceptance/tests/ingress-gateway/ingress_gateway_namespaces_test.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Removing the gateway type suffix from the naming conventions for terminating and ingress gateways (#1120) * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Enable terminating gateway policy to be generated via Auth Method * Filtering out failing portion of test for terminating gateway work * PR feedback.Fixing tests. Changing naming conventions for policy and roles for terminating gateways. * Correcting serviceAccount used on deployment * Making all nameshavea-ingress-gateway * Enable ingress gateway policy to be generated via Auth Method * Making all names have a -ingress-gateway suffix * Removing duplicate test * Update acceptance/tests/ingress-gateway/ingress_gateway_namespaces_test.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Removing the gateway type suffix from the naming conventions for terminating and ingress gateways * Adding check for duplicate terminating gateways and ingress gateway names * Update charts/consul/templates/ingress-gateways-deployment.yaml Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com> * PR Feedback - adding the duplicate name found to the check failures for duplicate ingress or terminating gateway names * Fixing rebase conflict * Merge Conflict- duplicate test * Adding a 10s sleep to flakey snapshot agent tests that were not finding a snapshot in time. Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com> Co-authored-by: Kyle Schochenmaier <kschoche@gmail.com> Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com> Co-authored-by: Thomas Eckert <teckert@hashicorp.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>

* Use kube auth method to provision ACL token for the crd controller (#995) * Use a Consul Kubernetes Auth Method to issue consul-login to mint ACL tokens and consul-logout to clean them up for the CRD controller. Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Support storing bootstrap token in Vault (#1061) * Global auth method (#1075) • Update server-acl-init to create authmethods in the primary datacenter when the job is run in a secondary datacenter during federation. This authmethod allows us to issue logins for global policies. • Update the controller workflow in server-acl-init to use this global authmethod when run in a secondary DC. • Update the mesh-gateway acceptance tests to create proxy defaults in the secondary DC to test above behavior works successfully. • Updated logout to not pass in the partition flag as it is not required. • Update server acl init tests to migrate from require := require.New(t) to require.xyz(t, ...) patterns. * Refactor ConnectInject to use authmethods (#1076) Refactor connect-injector to use the new auth-method workflow when ACLs are enabled so that Kubernetes secrets are not used. * Sync token acl refactor (#1081) • Refactor sync-catalog to use the new auth-method workflow when ACLs are enabled so that Kubernetes secrets are not used. • Create a service account and rolebinding dedicated to the component authmethod so that it no longer piggybacks on the one used by the connect-inject authmethod. * rename the controller flag (#1089) * Refactor Consul API Gateway Controller to use AuthMethod workflow. (#1083) * Refactor Consul API Gateway Controller to use AuthMethod workflow. * Refactor snapshot agent to use new acl authmethod workflow (#1084) * refactor snapshot agent to use new acl authmethod workflow. * Refactor mesh-gateway ACL flow (#1085) * Refactor mesh-gateway ACL flow * Fix flakey server-acl-init tests with retries (#1095) * Fix flakey server-acl-init tests with retries * Adding retry for flakey server-acl-init enterprise test * adding missing retry module in server-acl-init enterprise tests * Update Binding Rule if it exists for the authmethod (#1094) * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * vault: add support for admin partitions (#1098) * Refactor common.Login (#1101) * convert function args to a struct * add some missing tests * move logic that is only relevant for connect out * Use bootstrap token from vault to validate exec'ing into consul server (#1116) Follow up on #1103 * Enable terminating gateways to use ACL Auth Method (#1102) * Enable terminating gateway policy to be generated via Auth Method * Filtering out failing portion of test for terminating gateway work * PR feedback.Fixing tests. Changing naming conventions for policy and roles for terminating gateways. * Update control-plane/subcommand/server-acl-init/command.go Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update control-plane/subcommand/server-acl-init/command.go Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Fixing enterprise tests * Changing terminating gateway to pass acl-init a -component-name flag in the form of terminating-gateway/RELEASE-NAME-consul-terminating - <component type>/<consul fullname>-<name> * fixing acceptance test to recognize that long lived tokens will not exist and we ahve to update the role. * Correcting serviceAccount used on deployment * Making all nameshavea-ingress-gateway * Update charts/consul/templates/terminating-gateways-deployment.yaml Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update control-plane/subcommand/server-acl-init/command.go Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update control-plane/subcommand/server-acl-init/command.go Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update control-plane/subcommand/server-acl-init/command.go Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Enable snapshot agent configuration to be retrieved from vault (#1113) * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Enable snapshot agent configuration to be retrieved from vault * Adding CHANGELOG entry * Changing the decoding of snapshot agent config in vault to platform agnostic * Fixing the way we write the encoded vault secret out to a decoded json file * Decoding vault secret using consul template function on the vault annotation. Able to remove the bash that decodes the file and changes the extension. * Update CHANGELOG.md Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update charts/consul/values.yaml Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update charts/consul/values.yaml Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update charts/consul/values.yaml Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * PR Feedback - change client-snapshot-deployment to only have one vault role entry even when needing to set to vault roles * PR Feedback - when both snapshot agent and ca roles are specified in vault, it should get the sa role. * Simplifying conditional for vault role. Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Ability to set initial_management token when using k8s secret store. Snapshot agent acceptance tests (#1125) * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Enable snapshot agent configuration to be retrieved from vault * Adding CHANGELOG entry * Changing the decoding of snapshot agent config in vault to platform agnostic * Fixing the way we write the encoded vault secret out to a decoded json file * Decoding vault secret using consul template function on the vault annotation. Able to remove the bash that decodes the file and changes the extension. * Adding an acceptance test for snapshot agent. It currently fails because of a bug with Consul where it does not recognize CONSUL_HTTP_TOKEN. Will need to refactor test to bootstrap, then create vault secret with embedded acl token, then helm upgrade to add snapshot agent. Then assert that a *.snap file is created. * Adding acceptance test for snapshot agent on vault. * renaming test and removing extra file * Move vault test helpers into framework folder so we can use it more easily from other folders. * Adding snapshot agent test for k8s secret * Adding ability to set initial_management token when using k8s secrets. Also working acceptance test for snapshot agent on k8s secrets. * Adding bats tests. Adding envvar for ACL_BOOTSTRAP_TOKEN. Removing volume and volume mounts for bootstrap token. * Adding CHANGELOG entry for ability to pre-set bootstrap ACL token * Fixing bats tests * Update acceptance/framework/consul/helm_cluster.go Co-authored-by: Thomas Eckert <teckert@hashicorp.com> * Fixing broken unit tests * Lowering snapshot interval from 1mto15s for tests * Update acceptance/framework/consul/helm_cluster.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Update acceptance/framework/vault/helpers.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Update acceptance/tests/snapshot-agent/snapshot_agent_vault_test.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * PR Feedback - clarify comments on Vault helper functions * PR Feedback - clarify comments on Vault helper functions * Modifying tests to not incidentally send an encoded file * Removing logging token in acceptance test code. Co-authored-by: Thomas Eckert <teckert@hashicorp.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Enable ingress gateways to use ACL Auth Method (#1118) * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Enable terminating gateway policy to be generated via Auth Method * Filtering out failing portion of test for terminating gateway work * PR feedback.Fixing tests. Changing naming conventions for policy and roles for terminating gateways. * Changing terminating gateway to pass acl-init a -component-name flag in the form of terminating-gateway/RELEASE-NAME-consul-terminating - <component type>/<consul fullname>-<name> * Correcting serviceAccount used on deployment * Making all nameshavea-ingress-gateway * Enable ingress gateway policy to be generated via Auth Method * Making all names have a -ingress-gateway suffix * Removing duplicate test * Update acceptance/tests/ingress-gateway/ingress_gateway_namespaces_test.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Removing the gateway type suffix from the naming conventions for terminating and ingress gateways (#1120) * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Enable terminating gateway policy to be generated via Auth Method * Filtering out failing portion of test for terminating gateway work * PR feedback.Fixing tests. Changing naming conventions for policy and roles for terminating gateways. * Correcting serviceAccount used on deployment * Making all nameshavea-ingress-gateway * Enable ingress gateway policy to be generated via Auth Method * Making all names have a -ingress-gateway suffix * Removing duplicate test * Update acceptance/tests/ingress-gateway/ingress_gateway_namespaces_test.go Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> * Removing the gateway type suffix from the naming conventions for terminating and ingress gateways * Adding check for duplicate terminating gateways and ingress gateway names * Update charts/consul/templates/ingress-gateways-deployment.yaml Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com> * PR Feedback - adding the duplicate name found to the check failures for duplicate ingress or terminating gateway names * Fixing rebase conflict * Merge Conflict- duplicate test * Adding a 10s sleep to flakey snapshot agent tests that were not finding a snapshot in time. Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com> * Enable ACL Client Token (#1093) * Refactor ConsulLogin() to return the acltoken in addition to theerror. * Refactor createACLPolicyRoleAndBindingRule toappend datacenters for local tokens. Refactor updateOrCreateBindingRule to create binding rule if there are binding rules but this one does not exist * Rename -create-client-token flag to -client * set additional sans for consul server load balancer so that client will be able to use the certificate to talk to the load balancers rather than just an individual server. * Refactor server-acl-init command to create ACL Policy and Rule for client component so that client can call ConsulLogin and receive and ACL Token Call. * Enable client to talk to Consul Server to perform consul login. * Pass Auth Method to k8s al-init command. * Configure Consul address to be the Consul Server Load Balancer. * Configure CA Cert volume to be in memory rather than k8s secret when using vault. * Set consul/login volume and CONSUL_HTTP_TOKEN_FILE for use during logout. * Setup prestop command to perform consul logout. * Configure client-daemonset so that we can utilize the externalServers setting to configure clients to be able to call consul login on a server that is on a different partition. * Configuring partition-init to remove additional flags and use ones that already exist * adding missing comma * fix flakey tests by wrapping asserts in retries a la Iryna * Adding -use-https flag to client-daemonset.yaml when externalServers are enabled * Refactoring tests to cover client-acl-init changes * addressing PR comments * removing mounted tmpfs for consul-ca-cert when using vault and restoring datacenter logic because of breaking test. * addressing PR comments and only appending datacenters to a policy when its a local token, not global tokens. * completing additional dns names based on PR feedback * Do not ca-cert volume when using vault. * removing unused flagConsulCACert from partition-init command * PR Feedback. Removing unused envvars in acl-init container. changing ConsulLogin to return secretID, error instead ok token, error. * Updating changelog for ACLs work. * Update CHANGELOG.md Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update CHANGELOG.md Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Update CHANGELOG.md Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> * Adding note about old acl token cleanup. Adding note about configuring k8sAuthMethodHost in secondary datacenters when using mesh gateways in mesh federation. * Update CHANGELOG.md Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> Co-authored-by: Kyle Schochenmaier <kschoche@gmail.com> Co-authored-by: Iryna Shustava <ishustava@users.noreply.github.com> Co-authored-by: Ashwin Venkatesh <ashwin@hashicorp.com> Co-authored-by: Thomas Eckert <teckert@hashicorp.com> Co-authored-by: Nitya Dhanushkodi <nitya@hashicorp.com> Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com>

jmurret force-pushed the jm/acl-client branch 5 times, most recently from e62995f to f305e0e Compare March 11, 2022 21:16

jmurret force-pushed the acls-refactor-base-branch branch from 158bb3d to e41855a Compare March 11, 2022 21:57

jmurret force-pushed the jm/acl-client branch 6 times, most recently from 7a56bb8 to 818a5db Compare March 14, 2022 15:49

jmurret commented Mar 14, 2022

View reviewed changes

control-plane/subcommand/acl-init/command.go Show resolved Hide resolved

jmurret commented Mar 14, 2022

View reviewed changes

jmurret marked this pull request as ready for review March 14, 2022 17:50

jmurret added 10 commits March 15, 2022 14:32

fix flakey tests by wrapping asserts in retries a la Iryna

1cac8f7

Adding -use-https flag to client-daemonset.yaml when externalServers …

19d0064

…are enabled

Refactoring tests to cover client-acl-init changes

2c76ee6

addressing PR comments

eb41a15

removing mounted tmpfs for consul-ca-cert when using vault and restor…

14200d1

…ing datacenter logic because of breaking test.

addressing PR comments and only appending datacenters to a policy whe…

10c9b8c

…n its a local token, not global tokens.

completing additional dns names based on PR feedback

c690c57

Do not ca-cert volume when using vault.

653f819

removing unused flagConsulCACert from partition-init command

1fadcb4

PR Feedback. Removing unused envvars in acl-init container. changing …

b3a3ac9

…ConsulLogin to return secretID, error instead ok token, error.

jmurret force-pushed the jm/acl-client branch from 764ba08 to b3a3ac9 Compare March 15, 2022 20:33

jmurret merged commit 1476ecf into acls-refactor-base-branch Mar 15, 2022

jmurret deleted the jm/acl-client branch March 15, 2022 21:44

jmurret added the area/acls Related to ACLs label Mar 24, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable ACL Client Token #1093

Enable ACL Client Token #1093

jmurret commented Mar 10, 2022 •

edited

Loading

hashicorp-cla commented Mar 11, 2022 •

edited

Loading

jmurret Mar 14, 2022

ishustava Mar 14, 2022

jmurret Mar 15, 2022

ishustava Mar 15, 2022

jmurret Mar 15, 2022

jmurret Mar 15, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

jmurret Mar 14, 2022

Enable ACL Client Token #1093

Enable ACL Client Token #1093

Conversation

jmurret commented Mar 10, 2022 • edited Loading

hashicorp-cla commented Mar 11, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jmurret commented Mar 10, 2022 •

edited

Loading

hashicorp-cla commented Mar 11, 2022 •

edited

Loading