Add support for consul EnableTagOverride

[bogdanalbei]

At the moment the Consul's EnableTagOverride flag not being set by Nomad, so the default(false) is always used. In certain cases it is useful for services outside Nomad to update the tags of the Consul services, however for that to happen EnableTagOverride has to be set to true.
This change enables setting EnableTagOverride at a service level.
An older request describing this can be found here #2057

[hashicorp-cla]

All committers have signed the CLA.

[preetapan]

@bogdanalbei please fix linting errors using gofmt - https://travis-ci.org/hashicorp/nomad/jobs/541187845

[endocrimes]

nomad/command/agent/consul/client.go

Lines 91 to 98 in 168e3e9

	func agentServiceUpdateRequired(reg *api.AgentServiceRegistration, svc *api.AgentService) bool {
	return !(reg.Kind == svc.Kind &&
	reg.ID == svc.ID &&
	reg.Port == svc.Port &&
	reg.Address == svc.Address &&
	reg.Name == svc.Service &&
	reflect.DeepEqual(reg.Tags, svc.Tags))
	}

will also need updating to ensure that the AgentService tags are a superset of the service registration definition - otherwise Nomad will revert them every time it reconciles with Consul.

There may also be potential issues if other state causes the service to be re-registered (that could drop unmanaged tags), such as when promoting canary deployments.

[bogdanalbei]

you're right @endocrimes, I updated the code such that if EnableTagOverride is set, then the Consul tags are being kept and not overridden. I tested the change end to end and it works as expected.
Below are the test steps:

1. POST http://localhost:4646/v1/jobs

    "Job": {
        "ID": "example",
        "Name": "example",
        "Type": "service",
        "Priority": 50,
        "Datacenters": [
            "dc1"
        ],
        "TaskGroups": [{
            "Name": "cache",
            "Count": 1,
            "Tasks": [{
                "Name": "redis",
                "Driver": "docker",
                "User": "",
                "Config": {
                    "image": "redis:3.2",
                    "port_map": [{
                        "db": 6379
                    }]
                },
                "Services": [{
                    "Id": "",
                    "Name": "redis-cache",
                    "Tags": [
                        "global",
                        "cache"
                    ],
                    "PortLabel": "db",
                    "AddressMode": "",
                    "EnableTagOverride": true,
                    "Checks": [{
                        "Id": "",
                        "Name": "alive",
                        "Type": "tcp",
                        "Command": "",
                        "Args": null,
                        "Path": "",
                        "Protocol": "",
                        "PortLabel": "",
                        "Interval": 10000000000,
                        "Timeout": 2000000000,
                        "InitialStatus": "",
                        "TLSSkipVerify": false
                    }]
                }],
                "Resources": {
                    "CPU": 500,
                    "MemoryMB": 256,
                    "Networks": [{
                        "Device": "",
                        "CIDR": "",
                        "IP": "",
                        "MBits": 10,
                        "DynamicPorts": [{
                            "Label": "db",
                            "Value": 0
                        }]
                    }]
                },
                "Leader": false
            }],
            "RestartPolicy": {
                "Interval": 300000000000,
                "Attempts": 10,
                "Delay": 25000000000,
                "Mode": "delay"
            },
            "ReschedulePolicy": {
                "Attempts": 10,
                "Delay": 30000000000,
                "DelayFunction": "exponential",
                "Interval": 36000000000000,
                "MaxDelay": 3600000000000,
                "Unlimited": false
            },
            "EphemeralDisk": {
                "SizeMB": 300
            }
        }],
        "Update": {
            "MaxParallel": 1,
            "MinHealthyTime": 10000000000,
            "HealthyDeadline": 180000000000,
            "AutoRevert": false,
            "Canary": 0
        }
    }
}

2. GET http://localhost:8500/v1/catalog/service/redis-cache

3. PUT httplocalhost:8500/v1/agent/service/register

  "ID": "_nomad-task-c3d9c00e-0f06-de15-b53a-2e5e31ab67f8-redis-redis-cache-true",
  "Name": "redis-cache",
  "Tags": [
	"updated",
        "tags"
  ]
}

4. GET http://localhost:8500/v1/catalog/service/redis-cache - the updated tags shouldn't be reverted by Nomad

[dcarbone]

It would be really great if this could be merged in.

I have an application that needs to dynamically add and remove fabio urlprefix tags from its definition during runtime. At the moment this means I have to manage it in-process / implement some weird double-registration nonsense.

Would be great if I could offload the service lifecycle to nomad, and merely perform the updates in process.

edit - jira markup has ruined me.

[endocrimes]

@bogdanalbei I've targetted this for ~0.10.0, but I'd like to hold off merging till after #6197 merges. If you could rebase after that I'll take a look again 👍

[shoenig]

Thanks for the PR, @bogdanalbei! I started looking into this and realized there's some non-obvious complexity around the stateful house-keeping between Nomad's view of a service definition and what is maintained by each Consul agent. In short, we need to keep track of why a service is being synchronized with the agent, so that we can switch on that knowledge and make the right changes accordingly. There's a more detailed explanation in #7106

I'll go ahead and close this out; the ability to configure service.enable_tag_override will be shipping with v0.11

[github-actions]

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.