Support for multiple domains within --domain-filter #252

totallyunknown · 2017-06-27T17:35:30Z

The parameter accepts a comma separated list of domains with or without trailing dot. Example: --domain-filter="example.org, company.test.,staging.com". Closes #247 and #229

The parameter accepts a comma separated list of domains with or without trailing dot. Example: --domain-filter="example.org, company.test.,staging.com". Closes #247 and #229

hjacobs · 2017-06-27T21:57:48Z

Thanks for the PR!

linki · 2017-06-28T16:07:31Z

main.go

@@ -108,20 +109,22 @@ func main() {

 	endpointsSource := source.NewDedupSource(source.NewMultiSource(sources))

+	domainFilter := domains.NewDomainFilter(cfg.DomainFilter)


👍 very nice!

linki · 2017-06-28T16:12:50Z

pkg/apis/externaldns/types.go

@@ -99,7 +99,7 @@ func (cfg *Config) ParseFlags(args []string) error {
 	// Flags related to providers
 	app.Flag("provider", "The DNS provider where the DNS records will be created (required, options: aws, google, azure, cloudflare, digitalocean, inmemory)").Required().PlaceHolder("provider").EnumVar(&cfg.Provider, "aws", "google", "azure", "cloudflare", "digitalocean", "inmemory")
 	app.Flag("google-project", "When using the Google provider, specify the Google project (required when --provider=google)").Default(defaultConfig.GoogleProject).StringVar(&cfg.GoogleProject)
-	app.Flag("domain-filter", "Limit possible target zones by a domain suffix (optional)").Default(defaultConfig.DomainFilter).StringVar(&cfg.DomainFilter)
+	app.Flag("domain-filter", "Limit possible target zones by a comma separeted list of domain suffixes (optional)").Default(defaultConfig.DomainFilter).StringVar(&cfg.DomainFilter)


I would like to change this to use StringVars, note the traling s. This already supports processing multiple strings into a single string slice. This avoids strings.Split(",") in your code.

Furthermore, it changes the usage to provide the flag --domain-filter multiple times. Your example would become:

external-dns \ --domain-filter=example.com \ --domain-filter=company.org \ --domain-filter=staging-company.org

This approach is also used by helm and kubernetes.

The reason doing this way is, it's much easier to inject multiple domains as a single environmental variable into the pod. But might we work with a variable like this DOMAINS="--domain-filter foobar.org --domain-filter bla.org"..

True, but I would like to push this problem to kingpin. Ideally kingpin would allow to support both modes when using StringVars. FWIW kingpin's env var for this flag is DOMAIN_FILTER and you can specify multiple values by separating them with a line break.

lol, you already know that 😄

Yeah, but it's ugly.

linki · 2017-06-28T16:14:56Z

pkg/util/domains/domain_test.go

+	for i, tt := range domainFilterTests {
+		domainFilter := NewDomainFilter(tt.domainFilter)
+		for _, domain := range tt.domains {
+			require.Equal(t, tt.expected, domainFilter.Match(domain), "should not fail: %v in test-case #%v", domain, i)


I would prefer assert.Equal here.

linki · 2017-06-28T16:15:26Z

pkg/util/domains/domain_test.go

+	for _, tt := range domainFilterTests {
+		domainFilter := DomainFilter{}
+		for i, domain := range tt.domains {
+			require.True(t, domainFilter.Match(domain), "should not fail: %v in test-case #%v", domain, i)


I would prefer assert.True here.

linki · 2017-06-28T16:18:20Z

pkg/util/domains/domains.go

+			filters = append(filters[:i], filters[i+1:]...)
+			i--
+		}
+	}


That looks quite complicated. Maybe we can add some comments on what it's doing.

linki · 2017-06-28T16:21:03Z

pkg/util/domains/domains.go

+
+	for _, filter := range df.filters {
+
+		// user can define domains either with trailing dot or without.


I think it's better to sanitize both domains and then have only one suffix check. E.g. you could append or strip it from the provided filters before storing them. And then do the same with domain here before going through the filters.

We settled with stripping it from the domain in several parts of the code. But yeah, it's a little annoying.

totallyunknown · 2017-06-28T19:10:21Z

Please take a look. Changed the implementation to StringsVar() and change the code regarding your comments.

ideahitme · 2017-06-29T08:50:32Z

main.go

@@ -35,6 +35,7 @@ import (
 	"github.com/kubernetes-incubator/external-dns/controller"
 	"github.com/kubernetes-incubator/external-dns/pkg/apis/externaldns"
 	"github.com/kubernetes-incubator/external-dns/pkg/apis/externaldns/validation"
+	"github.com/kubernetes-incubator/external-dns/pkg/util/domains"


why is this in the util ? it is generally not a good practice to put things into util, if it is part of the provider, it should be in the provider package

Should I put it directly into provider package?

If the domain filter only to be used by providers then yes, it should be either in provider package directory, or it's child subdirectory.
If we find that provider exposes too many types/functions we can think of refactoring it later on

ideahitme · 2017-06-29T09:05:12Z

pkg/util/domains/domains_test.go

+	}
+}
+
+func TestDomainFilter_Match_default_Filter_always_matches(t *testing.T) {


please remove underscores and use camelcase

ideahitme · 2017-06-29T09:15:02Z

docs/tutorials/aws.md

@@ -84,7 +84,7 @@ spec:
        args:
        - --source=service
        - --source=ingress
-        - --domain-filter=external-dns-test.my-org.com. # will make ExternalDNS see only the hosted zones matching provided domain, omit to process all available hosted zones
+        - --domain-filter=external-dns-test.my-org.com # will make ExternalDNS see only the hosted zones matching provided domain, omit to process all available hosted zones


I would not change tutorials before the new release, the tutorial is anyway tied to the previous release

Out of curiosity, is there a branch that is setup for those types of changes that PRs could be created against to allow PR authors the ability to update the docs with a stable 'release' process and limit your need to update them?

@jrnt30 no, but would be interesting to hear if you don't mind explaining the process in the docs (maybe create a PR with modified contribution guidelines) ?

ideahitme · 2017-06-29T09:33:10Z

pkg/util/domains/domains.go

+
+// NewDomainFilter returns a new DomainFilter given a comma separated list of domains
+func NewDomainFilter(filters []string) DomainFilter {
+


nitpick: it is better to make a new slice and copy the strings into it without retaining pointer from SliceHeader

ideahitme · 2017-06-29T10:18:47Z

please update the changelog, and it is good to be merged

totallyunknown · 2017-06-29T10:37:49Z

Updated the changelog.

ideahitme

thanks

linki · 2017-06-29T15:27:40Z

@totallyunknown This is also a fix for #229 right?

linki

great!

totallyunknown · 2017-06-29T17:07:22Z

Thanks for merging!

linki · 2017-06-30T09:09:33Z

@totallyunknown Thanks!

) * Support for multiple domains within --domain-filter The parameter accepts a comma separated list of domains with or without trailing dot. Example: --domain-filter="example.org, company.test.,staging.com". Closes kubernetes-sigs#247 and kubernetes-sigs#229 * Add boilerplate header * Add documentation for methods and structs * use StringsVar for the domain-filter flag * go fmt * Remove camel case from tests * Revert changes in README.md * Move DomainFilter to provider package * Make a new slice and copy elements to it * Update CHANGELOG.md * docs: change minor spelling mistake

Support for multiple domains within --domain-filter

8927463

The parameter accepts a comma separated list of domains with or without trailing dot. Example: --domain-filter="example.org, company.test.,staging.com". Closes #247 and #229

k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Jun 27, 2017

totallyunknown added 2 commits June 27, 2017 19:44

Add boilerplate header

e33e7be

Add documentation for methods and structs

7f31af3

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Jun 27, 2017

linki reviewed Jun 28, 2017

View reviewed changes

totallyunknown added 2 commits June 28, 2017 20:36

use StringsVar for the domain-filter flag

3312654

go fmt

80ec6eb

ideahitme reviewed Jun 29, 2017

View reviewed changes

totallyunknown added 2 commits June 29, 2017 11:19

Remove camel case from tests

950e3db

Revert changes in README.md

0d19ebf

ideahitme reviewed Jun 29, 2017

View reviewed changes

totallyunknown added 2 commits June 29, 2017 11:35

Move DomainFilter to provider package

cf61831

Make a new slice and copy elements to it

d496ec6

ideahitme added the kind/feature Categorizes issue or PR as related to a new feature. label Jun 29, 2017

ideahitme mentioned this pull request Jun 29, 2017

Simplify DNS provider requirements by extracting common parts #228

Closed

5 tasks

Update CHANGELOG.md

1bdc54a

ideahitme approved these changes Jun 29, 2017

View reviewed changes

docs: change minor spelling mistake

df81d1e

kubernetes-sigs deleted a comment from k8s-ci-robot Jun 29, 2017

linki approved these changes Jun 29, 2017

View reviewed changes

ideahitme merged commit 73d3979 into kubernetes-sigs:master Jun 29, 2017

hjacobs mentioned this pull request Jun 29, 2017

domain-filter for aws requires a ending period #229

Closed

chrisghill mentioned this pull request Jun 8, 2022

creating one external-dns per hosted zone massdriver-cloud/terraform-modules#2

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for multiple domains within --domain-filter #252

Support for multiple domains within --domain-filter #252

totallyunknown commented Jun 27, 2017 •

edited

Loading

hjacobs commented Jun 27, 2017

linki Jun 28, 2017

linki Jun 28, 2017

totallyunknown Jun 28, 2017

linki Jun 29, 2017

linki Jun 29, 2017

totallyunknown Jun 29, 2017

linki Jun 28, 2017

linki Jun 28, 2017

linki Jun 28, 2017

linki Jun 28, 2017 •

edited

Loading

totallyunknown commented Jun 28, 2017

ideahitme Jun 29, 2017

totallyunknown Jun 29, 2017

ideahitme Jun 29, 2017

ideahitme Jun 29, 2017

totallyunknown Jun 29, 2017

ideahitme Jun 29, 2017 •

edited

Loading

totallyunknown Jun 29, 2017

jrnt30 Jun 29, 2017

ideahitme Jun 29, 2017

ideahitme Jun 29, 2017 •

edited

Loading

ideahitme commented Jun 29, 2017

totallyunknown commented Jun 29, 2017

ideahitme left a comment

linki commented Jun 29, 2017

linki left a comment

totallyunknown commented Jun 29, 2017

linki commented Jun 30, 2017

		@@ -108,20 +109,22 @@ func main() {

		endpointsSource := source.NewDedupSource(source.NewMultiSource(sources))

		domainFilter := domains.NewDomainFilter(cfg.DomainFilter)


		for _, filter := range df.filters {

		// user can define domains either with trailing dot or without.


		// NewDomainFilter returns a new DomainFilter given a comma separated list of domains
		func NewDomainFilter(filters []string) DomainFilter {

Support for multiple domains within --domain-filter #252

Support for multiple domains within --domain-filter #252

Conversation

totallyunknown commented Jun 27, 2017 • edited Loading

hjacobs commented Jun 27, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linki Jun 28, 2017 • edited Loading

Choose a reason for hiding this comment

totallyunknown commented Jun 28, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ideahitme Jun 29, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ideahitme Jun 29, 2017 • edited Loading

Choose a reason for hiding this comment

ideahitme commented Jun 29, 2017

totallyunknown commented Jun 29, 2017

ideahitme left a comment

Choose a reason for hiding this comment

linki commented Jun 29, 2017

linki left a comment

Choose a reason for hiding this comment

totallyunknown commented Jun 29, 2017

linki commented Jun 30, 2017

totallyunknown commented Jun 27, 2017 •

edited

Loading

linki Jun 28, 2017 •

edited

Loading

ideahitme Jun 29, 2017 •

edited

Loading

ideahitme Jun 29, 2017 •

edited

Loading