advertise-client-urls does not take effect from config file in 3.1.1

[markshuttle]

Am launching etcd using --config-file with a config that includes:

listen-peer-urls: 'http://192.168.12.3:2380'
listen-client-urls: 'http://192.168.12.3:2379,http://localhost:2379'
advertise-client-urls: 'http://192.168.12.3:2379'

On startup, I see this:

etcd[26411]: the server is already initialized as member before, starting as etcd member...
etcd[26411]: advertising using detected default host "XX.XX.XX.XX"
etcd[26411]: listening for peers on http://192.168.12.3:2380
etcd[26411]: listening for client requests on 192.168.12.3:2379
etcd[26411]: listening for client requests on localhost:2379
etcd[26411]: recovered store from snapshot at index 72099127
[...snip...]
etcd[26411]: advertise client URLs = http://localhost:2379

I'm pretty sure that with 3.0.17 the advertised client URL was http://192.168.12.3:2379 but now it is reverting to localhost. I don't think there are any environment variables in play in this case.

I'm not sure what to make about the 'advertising detected default host', which is referencing an address I don't want used at all.

[xiang90]

Can you provide us an example yaml file that can reproduce your issue? Thank you!

[gyuho]

@markshuttle Ping?

[markshuttle]

Thanks Gyuho, here is the YAML I am using:

name: 'XXX'
data-dir: '/var/snap/etcd/current/XXX.etcd'
wal-dir: ''
snapshot-count:
heartbeat-interval:
election-timeout:
listen-peer-urls: 'http://192.168.33.2:7001,http://192.168.33.2:2380'
listen-client-urls: 'http://192.168.33.2:2379,http://localhost:2379'
max-snapshots:
max-wals:
cors: ''
advertise-client-urls: 'http://192.168.33.2:2379'
discovery: ''
discovery-srv: ''
discovery-fallback: ''
discovery-proxy: ''
strict-reconfig-check:
proxy: ''
proxy-failure-wait:
proxy-refresh-interval:
proxy-dial-timeout:
proxy-write-timeout:
proxy-read-timeout:
client-transport-security:
cert-file: ''
key-file: ''
client-cert-auth:
trusted-ca-file: ''
auto-tls: false
peer-transport-security:
cert-file: ''
key-file: ''
client-cert-auth:
trusted-ca-file: ''
auto-tls: false
debug:
log-package-levels: ''

[markshuttle]

@gyuho anything jump out as problematic there in the configuration? Or can you reproduce the issue with this YAML? Happy to do tests on alternative configs or to run test builds.

[ChrLau]

@markshuttle I wanted to comment on this issue before too with "I'm having the same problem." but in my case it turned out that everything worked perfectly when I deleted the data-dir (was a test setup) on all members and re-build the cluster from scratch.
As I was testing out different client/peer and port combinations I think that maybe something was still cached somewhere? (Not sure about HOW etcd should behave, as I'm relatively new to it.)

So, have you tried this?

[gyuho]

@markshuttle I can reproduce this. Sorry for delay.
I will try to fix this by this week.

[markshuttle]

@gyuho, thank you! Happy to test a branch for you if that will help. Christian, I haven't had a chance to try rebuilding the data dir. Is the config partially stored in the data dir? Mark

[gyuho]

@markshuttle Could you try with current master branch? We will backport the fix and ship it with upcoming v3.1.3 release. Thanks!

[markshuttle]

Thanks @gyuho that's confirmed fixed on master, via today's edge snap. Very much appreciate your work! Mark

[ozdanborne]

Is there a workaround to prevent "advertise client URLs" from being overwritten?

[heyitsanthony]

@djosborne it should always overwrite; what is your use case?

There is a workaround for what it does now. The one time it ever overwrites is if 0.0.0.0 is explicitly given and if the default advertise is set to localhost. In that case it'll use default host. To avoid overwriting with the default host and keep loopback over localhost, the advertise urls can be explicitly set to 127.0.0.1 or ::1

[ozdanborne]

Let me clarify my issue - I'm setting the following values:

ETCD_ADVERTISE_CLIENT_URLS=https://10.240.0.32:2379
ETCD_LISTEN_CLIENT_URLS=https://10.240.0.32:2379

And I'm seeing the behavior described above: listen-client-urls is being set to localhost:2379.

@heyitsanthony I'm having trouble understanding your response, but I would like to as seems like it is the answer to my question.

The one time it ever overwrites is if 0.0.0.0 is explicitly given

Neither mine nor @markshuttle 's config sets anything explicitly to 0.0.0.0 and we're both seeing this. Which parameter field are you describing?

and if the default advertise is set to localhost

Again, neither my config nor mark's sets default advertise to localhost. Which parameter field are you describing?

It's possible there's some confusion here because I'm not sure which advertise params (client-urls or peer-urls) you are talking about in your response.

And apologies if I'm misunderstanding something simple! Thank you for your time!

[heyitsanthony]

@djosborne is this happening with the fix? There's no workaround for the broken version as far as I know.

[ozdanborne]

Nope, this is with 3.x, I think 3.0.

Fine if there's no workaround for the broken version. I've got things working by downgrading to 2.x.

I thought by the sound of it I just needed to explicitly set some variable off it's default and it would work, but I guess that's not the case.

So never mind! And thanks again for your time.

UPDATE: I was mistaken, it was 3.1.0, so same issue.

[heyitsanthony]

@djosborne that sounds like a separate issue. Only 3.1.0-2 should be affected by this one.