You've probably created many deployments. Typically, you expose your deployment to the Internet by creating a Service with type=LoadBalancer
. Depending on your environment, this usually assigns a random publicly available endpoint to your service that you can access from anywhere in the world. On Google Container Engine, this is a public IP address:
$ kubectl get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx 10.3.249.226 35.187.104.85 80:32281/TCP 1m
But dealing with IPs for service discovery isn't nice, so you register this IP with your DNS provider under a better name—most likely, one that corresponds to your service name. If the IP changes, you update the DNS record accordingly.
Those times are over! ExternalDNS takes care of that last step for you by keeping your DNS records synchronized with your external entry points.
ExternalDNS' usefulness also becomes clear when you use Ingresses to allow external traffic into your cluster. Via Ingress, you can tell Kubernetes to route traffic to different services based on certain HTTP request attributes, e.g. the Host header:
$ kubectl get ing
NAME HOSTS ADDRESS PORTS AGE
entrypoint frontend.example.org,backend.example.org 35.186.250.78 80 1m
But there's nothing that actually makes clients resolve those hostnames to the Ingress' IP address. Again, you normally have to register each entry with your DNS provider. Only if you're lucky can you use a wildcard, like in the example above.
ExternalDNS can solve this for you as well.
Currently, the following providers are supported:
- Google Cloud DNS
- AWS Route 53
- AzureDNS
- CloudFlare
- DigitalOcean
- DNSimple
- Infoblox
- Dyn
- OpenStack Designate
- PowerDNS
- CoreDNS
- Exoscale
- Oracle Cloud Infrastructure DNS
- Linode DNS
- RFC2136
- TransIP
As stated in the README, we are currently looking for stable maintainers for those providers, to ensure that bugfixes and new features will be available for all of those.
Services exposed via type=LoadBalancer
and for the hostnames defined in Ingress objects as well as headless hostPort services. An initial effort to support type NodePort
was started as of May 2018 and it is in progress at the time of writing.
There are three sources of information for ExternalDNS to decide on DNS name. ExternalDNS will pick one in order as listed below:
-
For ingress objects ExternalDNS will create a DNS record based on the host specified for the ingress object. For services ExternalDNS will look for the annotation
external-dns.alpha.kubernetes.io/hostname
on the service and use the corresponding value. -
If compatibility mode is enabled (e.g.
--compatibility={mate,molecule}
flag), External DNS will parse annotations used by Zalando/Mate, wearemolecule/route53-kubernetes. Compatibility mode with Kops DNS Controller is planned to be added in the future. -
If
--fqdn-template
flag is specified, e.g.--fqdn-template={{.Name}}.my-org.com
, ExternalDNS will use service/ingress specifications for the provided template to generate DNS name.
Yes, you can. Pass in a comma separated list to --fqdn-template
. Beaware this will double (triple, etc) the amount of DNS entries based on how many services, ingresses and so on you have and will get you faster towards the API request limit of your DNS provider.
Regarding Services, we'll support the OSI Layer 4 load balancers that Kubernetes creates on AWS and Google Container Engine, and possibly other clusters running on Google Compute Engine.
Regarding Ingress, we'll support:
- Google's Ingress Controller on GKE that integrates with their Layer 7 load balancers (GLBC)
- nginx-ingress-controller v0.9.x with a fronting Service
- Zalando's AWS Ingress controller, based on AWS ALBs and Skipper
- Traefik 1.7 and above, when
kubernetes.ingressEndpoint
is configured (kubernetes.ingressEndpoint.useDefaultPublishedService
in the Helm chart)
For Ingress objects, ExternalDNS will attempt to discover the target hostname of the relevant Ingress Controller automatically. If you are using an Ingress Controller that is not listed above you may have issues with ExternalDNS not discovering Endpoints and consequently not creating any DNS records. As a workaround, it is possible to force create an Endpoint by manually specifying a target host/IP for the records to be created by setting the annotation external-dns.alpha.kubernetes.io/target
in the Ingress object.
Another reason you may want to override the ingress hostname or IP address is if you have an external mechanism for handling failover across ingress endpoints. Possible scenarios for this would include using keepalived-vip to manage failover faster than DNS TTLs might expire.
Note that if you set the target to a hostname, then a CNAME record will be created. In this case, the hostname specified in the Ingress object's annotation must already exist. (i.e. you have a Service resource for your Ingress Controller with the external-dns.alpha.kubernetes.io/hostname
annotation set to the same value.)
ExternalDNS is a joint effort to unify different projects accomplishing the same goals, namely:
- Kops' DNS Controller
- Zalando's Mate
- Molecule Software's route53-kubernetes
We strive to make the migration from these implementations a smooth experience. This means that, for some time, we'll support their annotation semantics in ExternalDNS and allow both implementations to run side-by-side. This enables you to migrate incrementally and slowly phase out the other implementation.
ExternalDNS will allow you to opt into any Services and Ingresses that you want it to consider, by an annotation. This way, it can co-exist with other implementations running in the same cluster if they also support this pattern. However, we'll most likely declare ExternalDNS to be the default implementation. This means that ExternalDNS will consider Services and Ingresses that don't specifically declare which controller they want to be processed by; this is similar to the ingress.class
annotation on GKE.
ExternalDNS since v0.3 implements the concept of owning DNS records. This means that ExternalDNS will keep track of which records it has control over, and will never modify any records over which it doesn't have control. This is a fundamental requirement to operate ExternalDNS safely when there might be other actors creating DNS records in the same target space.
For now ExternalDNS uses TXT records to label owned records, and there might be other alternatives coming in the future releases.
Yes, multiple companies are using ExternalDNS in production. Zalando, as an example, has been using it in production since its v0.3 release, mostly using the AWS provider.
Check out the following descriptive tutorials on how to run ExternalDNS in GKE and AWS or any other supported provider.
Why is ExternalDNS only adding a single IP address in Route 53 on AWS when using the nginx-ingress-controller
? How do I get it to use the FQDN of the ELB assigned to my nginx-ingress-controller
Service instead?
By default the nginx-ingress-controller
assigns a single IP address to an Ingress resource when it's created. ExternalDNS uses what's assigned to the Ingress resource, so it too will use this single IP address when adding the record in Route 53.
In most AWS deployments, you'll instead want the Route 53 entry to be the FQDN of the ELB that is assigned to the nginx-ingress-controller
Service. To accomplish this, when you create the nginx-ingress-controller
Deployment, you need to provide the --publish-service
option to the /nginx-ingress-controller
executable under args
. Once this is deployed new Ingress resources will get the ELB's FQDN and ExternalDNS will use the same when creating records in Route 53.
According to the nginx-ingress-controller
docs the value you need to provide --publish-service
is:
Service fronting the ingress controllers. Takes the form namespace/name. The controller will set the endpoint records on the ingress objects to reflect those on the service.
For example if your nginx-ingress-controller
Service's name is nginx-ingress-controller-svc
and it's in the default
namespace the start of your resource YAML might look like the following. Note the second to last line.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-ingress-controller
spec:
replicas: 1
template:
metadata:
labels:
app: nginx-ingress
spec:
hostNetwork: false
containers:
- name: nginx-ingress-controller
image: "gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.11"
imagePullPolicy: "IfNotPresent"
args:
- /nginx-ingress-controller
- --default-backend-service={your-backend-service}
- --publish-service=default/nginx-ingress-controller-svc
- --configmap={your-configmap}
ExternalDNS can be configured to only use Services or Ingresses as source. In case Services or Ingresses seem to be ignored in your setup, consider checking how the flag --source
was configured when deployed. For reference, see the issue kubernetes-sigs#267.
I'm using an ELB with TXT registry but the CNAME record clashes with the TXT record. How to avoid this?
CNAMEs cannot co-exist with other records, therefore you can use the --txt-prefix
flag which makes sure to create a TXT record with a name following the pattern prefix.<CNAME record>
. For reference, see the issue kubernetes-sigs#262.
You need to add either https://www.googleapis.com/auth/ndev.clouddns.readwrite or https://www.googleapis.com/auth/cloud-platform on your instance group's scope.
ExternalDNS exposes 2 types of metrics: Sources and Registry errors.
Source
s are mostly Kubernetes API objects. Examples of source
errors may be connection errors to the Kubernetes API server itself or missing RBAC permissions. It can also stem from incompatible configuration in the objects itself like invalid characters, processing a broken fqdnTemplate, etc.
Registry
errors are mostly Provider errors, unless there's some coding flaw in the registry package. Provider errors often arise due to accessing their APIs due to network or missing cloud-provider permissions when reading records. When applying a changeset, errors will arise if the changeset applied is incompatible with the current state.
In case of an increased error count, you could correlate them with the http_request_duration_seconds{handler="instrumented_http"}
metric which should show increased numbers for status codes 4xx (permissions, configuration, invalid changeset) or 5xx (apiserver down).
You can use the host label in the metric to figure out if the request was against the Kubernetes API server (Source errors) or the DNS provider API (Registry/Provider errors).
How can I run ExternalDNS under a specific GCP Service Account, e.g. to access DNS records in other projects?
Have a look at https://github.com/linki/mate/blob/v0.6.2/examples/google/README.md#permissions
Separate the individual values via a line break. The equivalent of --source=service --source=ingress
would be service\ningress
. However, it can be tricky do define that depending on your environment. The following examples work (zsh):
Via docker:
$ docker run \
-e EXTERNAL_DNS_SOURCE=$'service\ningress' \
-e EXTERNAL_DNS_PROVIDER=google \
-e EXTERNAL_DNS_DOMAIN_FILTER=$'foo.com\nbar.com' \
registry.opensource.zalan.do/teapot/external-dns:latest
time="2017-08-08T14:10:26Z" level=info msg="config: &{Master: KubeConfig: Sources:[service ingress] Namespace: ...
Locally:
$ export EXTERNAL_DNS_SOURCE=$'service\ningress'
$ external-dns --provider=google
INFO[0000] config: &{Master: KubeConfig: Sources:[service ingress] Namespace: ...
$ EXTERNAL_DNS_SOURCE=$'service\ningress' external-dns --provider=google
INFO[0000] config: &{Master: KubeConfig: Sources:[service ingress] Namespace: ...
In a Kubernetes manifest:
spec:
containers:
- name: external-dns
args:
- --provider=google
env:
- name: EXTERNAL_DNS_SOURCE
value: "service\ningress"
Or preferably:
spec:
containers:
- name: external-dns
args:
- --provider=google
env:
- name: EXTERNAL_DNS_SOURCE
value: |-
service
ingress
Sometimes you need to run an internal and an external dns service. The internal one should provision hostnames used on the internal network (perhaps inside a VPC), and the external one to expose DNS to the internet.
To do this with ExternalDNS you can use the --annotation-filter
to specifically tie an instance of ExternalDNS to
an instance of a ingress controller. Let's assume you have two ingress controllers nginx-internal
and nginx-external
then you can start two ExternalDNS providers one with --annotation-filter=kubernetes.io/ingress.class=nginx-internal
and one with --annotation-filter=kubernetes.io/ingress.class=nginx-external
.
Can external-dns manage(add/remove) records in a hosted zone which is setup in different AWS account?
Yes, give it the correct cross-account/assume-role permissions and use the --aws-assume-role
flag kubernetes-sigs#524 (comment)
Separate them by ,
.
When we tag a new release, we push a Docker image on Zalando's public Docker registry with the following name:
registry.opensource.zalan.do/teapot/external-dns
As tags, you can use your version of choice or use latest
that always resolves to the latest tag.
If you wish to build your own image, you can use the provided Dockerfile as a starting point.
We are currently working with the Kubernetes community to provide official images for the project similarly to what is done with the other official Kubernetes projects, but we don't have an ETA on when those images will be available.