Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

consul/connect: in-place update service definition when connect upstreams are modified #10776

Merged
merged 1 commit into from
Jun 17, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions command/agent/consul/service_client.go
Original file line number Diff line number Diff line change
Expand Up @@ -248,9 +248,74 @@ func sidecarTagsDifferent(parent, wanted, sidecar []string) bool {
return tagsDifferent(wanted, sidecar)
}

// proxyUpstreamsDifferent determines if the sidecar_service.proxy.upstreams
// configurations are different between the desired sidecar service state, and
// the actual sidecar service state currently registered in Consul.
func proxyUpstreamsDifferent(wanted *api.AgentServiceConnect, sidecar *api.AgentServiceConnectProxyConfig) bool {
// There is similar code that already does this in Nomad's API package,
// however here we are operating on Consul API package structs, and they do not
// provide such helper functions.

getProxyUpstreams := func(pc *api.AgentServiceConnectProxyConfig) []api.Upstream {
switch {
case pc == nil:
return nil
case len(pc.Upstreams) == 0:
return nil
default:
return pc.Upstreams
}
}

getConnectUpstreams := func(sc *api.AgentServiceConnect) []api.Upstream {
switch {
case sc.SidecarService.Proxy == nil:
return nil
case len(sc.SidecarService.Proxy.Upstreams) == 0:
return nil
default:
return sc.SidecarService.Proxy.Upstreams
}
}

upstreamsDifferent := func(a, b []api.Upstream) bool {
if len(a) != len(b) {
return true
}

for i := 0; i < len(a); i++ {
A := a[i]
B := b[i]
switch {
case A.Datacenter != B.Datacenter:
return true
case A.DestinationName != B.DestinationName:
return true
case A.LocalBindAddress != B.LocalBindAddress:
return true
case A.LocalBindPort != B.LocalBindPort:
return true
case A.MeshGateway.Mode != B.MeshGateway.Mode:
return true
case !reflect.DeepEqual(A.Config, B.Config):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this big switch/case is a performance optimization to avoid doing reflect.DeepEqual on the api.Upstream if we can avoid it, but from a maintenance perspective this seems like it's brittle. If we upgrade the Consul library and they add new fields this check will silently break.

(Alternately, is there maybe a way we can write a unit test that automatically gives us non-default values for all fields so that we have a breaking test on upgrade? I don't know of one off the top of my head but that would be really cool.)

Copy link
Member Author

@shoenig shoenig Jun 17, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you just gave me an excuse to play around with https://blog.golang.org/fuzz-beta 🙂

Using reflect.DeepEqual is similarly dangerous, for example it always returns false if a function is added to the struct fields:
https://play.golang.org/p/Yfu4d2RdTKd

Really thought the only correct thing to do here would be fore Consul to provide equality methods in their api package - they are the ones who define what the thing is and know how it compares.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally agreed there.

return true
}
}
return false
}

return upstreamsDifferent(
getConnectUpstreams(wanted),
getProxyUpstreams(sidecar),
)
}

// connectSidecarDifferent returns true if Nomad expects there to be a sidecar
// hanging off the desired parent service definition on the Consul side, and does
// not match with what Consul has.
//
// This is used to determine if the connect sidecar service registration should be
// updated - potentially (but not necessarily) in-place.
func connectSidecarDifferent(wanted *api.AgentServiceRegistration, sidecar *api.AgentService) bool {
if wanted.Connect != nil && wanted.Connect.SidecarService != nil {
if sidecar == nil {
Expand All @@ -262,6 +327,11 @@ func connectSidecarDifferent(wanted *api.AgentServiceRegistration, sidecar *api.
// tags on the nomad definition have been modified
return true
}

if proxyUpstreamsDifferent(wanted.Connect, sidecar.Proxy) {
// proxy upstreams on the nomad definition have been modified
return true
}
}

// Either Nomad does not expect there to be a sidecar_service, or there is
Expand Down
Loading