BugFix: 1.0 Ingesters having availability zone value erased by pre-1.0 ingesters during rolling upgrade.

[khaines]

What this PR does: Corrects a bug reported by @pracucci when upgrading ingesters from pre-1.0, which causes the new ingesters to have their availability zone string overwritten by a blank value. This PR addresses the issue by having the ingesters update the ring data with their zone on every heartbeat, similar to what they do with address information.

Which issue(s) this PR fixes:
Fixes #2401

Checklist

• [ x ] Tests updated
• [ X ] CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[pstibrany]

This is a step in the right direction. Unfortunately, it doesn't fix the fact that old ingesters (not supporting the zones) will clear the zone information for all ingesters on each heartbeat.

I think we should also make "no zone" to be empty string (and not ingester ID as it is today), to avoid seeing artifacts like this when watching the /ring page. Also visual check of the /ring would be faster -- it would be easy to see which ingesters have zone set. What do you think?

[khaines]

@pstibrany this is only an issue during a rolling upgrade from pre1.0, once the old ingesters are rolled off the zones will be no longer overwritten each heartbeat. I would like to hear more if you are thinking this needs support outside the upgrade process.

Since it was added to ignore zoning for a blank string, I agree we should make that the default "no zone"

[pstibrany]

Thanks Ken, looks good!

[pracucci]

Thanks @khaines for working on it! LGTM to me too.