Same hostname, but different set-identifier is not creating multiple records

[marlenepereira]

What happened:
Adding external-dns.alpha.kubernetes.io/hostname annotation to different service resources with the same hostname doesn't create multiple weighted DNS records.

We Istio ingress gateways, AWS load balancers and AWS Route53 weighted DNS.
We want to annotate Istio ingress gateway services with the following annotations to create Route53 DNS entries:

# Istio ingress service at version 1
external-dns.alpha.kubernetes.io/alias: "true"
external-dns.alpha.kubernetes.io/hostname: "ingress.my-domain-1.com"
external-dns.alpha.kubernetes.io/set-identifier: "version-1"
external-dns.alpha.kubernetes.io/aws-weight: "100"

During Istio upgrades, we deploy the new version of Istio ingress gateways and add the following annotations:

# Istio Istio ingress service at version 2
external-dns.alpha.kubernetes.io/alias: "true"
external-dns.alpha.kubernetes.io/hostname: "ingress.my-domain-1.com"
external-dns.alpha.kubernetes.io/set-identifier: "version-2"
external-dns.alpha.kubernetes.io/aws-weight: "0"

When testing this setup, version 1 of the service is deployed first and ExternalDNS creates the DNS record. However, when we later deploy the service at version 2, ExternalDNS "upserts" the DNS record with nameingress.my-domain-1.com (see logs), but doesn't create a new DNS record nor update the existing one.

ExternalDNS logs

time="2021-08-25T05:10:57Z" level=info msg="Applying provider record filter for domains: [my-domain-1.com. .my-domain-1.com. my-domain-1.com. .my-domain-1.com.]"
time="2021-08-25T05:10:57Z" level=info msg="Desired change: CREATE ingress.my-domain-1.com A [Id: /hostedzone/ABCDE]"
time="2021-08-25T05:10:57Z" level=info msg="Desired change: CREATE ingress.my-domain-1.commy-domain-1.com TXT [Id: /hostedzone/ABCDE]"
time="2021-08-25T05:10:58Z" level=info msg="2 record(s) in zone my-domain-1.com. [Id: /hostedzone/ABCDE] were successfully updated"
time="2021-08-25T05:12:11Z" level=info msg="Applying provider record filter for domains: [my-domain-1.com. .my-domain-1.com. my-domain-1.com. .my-domain-1.com.]"
time="2021-08-25T05:12:11Z" level=info msg="Desired change: UPSERT ingress.my-domain-1.com A [Id: /hostedzone/ABCDE]"
time="2021-08-25T05:12:11Z" level=info msg="Desired change: UPSERT ingress.my-domain-1.com TXT [Id: /hostedzone/ABCDE]"
time="2021-08-25T05:12:12Z" level=info msg="2 record(s) in zone my-domain-1.com. [Id: /hostedzone/ABCDE] were successfully updated"

What you expected to happen:
I expected to have two DNS records for ingress.my-domain-1.com with different record id and pointing to the load balancer that corresponds to the ingress service for each version, i.e:

ingress.my-domain-1.com  |  A  | Weighted  | 100 |  load-balacer-istio-version-1 (record id: version-1)
ingress.my-domain-1.com  | TXT | Weighted  | 100 |  "heritage=external-dns,external-dns/owner=cluster-a,external-dns/resource=service/istio-system/ingressgateway-1"
ingress.my-domain-1.com  |  A  | Weighted  |  0  |  load-balacer-istio-version-2  (record id: version-2)
ingress.my-domain-1.com  | TXT | Weighted  |  0  |  "heritage=external-dns,external-dns/owner=cluster-a,external-dns/resource=service/istio-system/ingressgateway-2"

How to reproduce it (as minimally and precisely as possible):

Deploy two Istio ingress gateways and add the annotations described above.

Anything else we need to know?:
Both ingress services have the annotations.

Environment:

• DNS provider: aws
• chart version: 1.2.0

chart values

values:
    fullnameOverride: external-dns
    serviceMonitor:
      enabled: true
    provider: aws
    triggerLoopOnEvent: true
    interval: 5m
    policy: sync
    registry: txt
    txtOwnerId: cluster-a
    domainFilters:
    - "my-domain-1.com"
    - "my-domain-2.com"
    sources:
      - service
      - ingress

[jangjaelee]

did you solve this problem?

[marlenepereira]

did you solve this problem?

No, I have an open PR to solve this issue, but it hasn't been reviewed/merged yet.