Add e2e tests for multi-namespaces deployments

[sebgl]

Use case:

• Kibana in one namespace
• Elasticsearch in another namespace

Association should correctly be established between both.

So far our E2E tests do rely on a default e2e namespace for most tests.
Here we need more plumbing to support multiple namespaces in the tests.

[charith-elastic]

Some of the current assumptions in our test bootstrap code makes implementing this a bit tricky and more involved than I anticipated. I raised #1251 to discuss this further.

[anyasabo]

Can you help me understand this use case? Why would we want them in separate namespaces?

[sebgl]

Can you help me understand this use case? Why would we want them in separate namespaces?

The use case would be to allow certain users (eg. a team in your org) to manipulate ES clusters, and let other users (eg. another team) tweak their Kibana settings with no access to the Elasticsearch namespace.
This is probably a corner case that may not be very useful. We still implemented it to make sure we get this flexibility if we need to.

[charith-elastic]

I encountered a catch-22 situation while trying to write this test.

• When we setup the E2E test environment each namespace gets its own operator because creating a single operator for all the test namespaces will require too many privileges and makes test isolation difficult.
• When KB and ES are managed by two different operators, the association does not work because the association controller can't watch the Elasticsearch resource running in the other namespace.

Due to these reasons, I have a test that works locally (operator is managing all namespaces) but it cannot be run in the CI environment (operator restricted to a single namespace). Rather than adding too many privileges to the CI environment, I am inclined to wait for a fix to #1438.

[barkbay]

Reopening this issue as multi-namespaces scenarios are still not run in our CI process.

[pebrc]

We have now this:

cloud-on-k8s/test/e2e/kb/association_test.go

Lines 24 to 40 in 459d005

	func TestCrossNSAssociation(t *testing.T) {
	esNamespace := test.Ctx().ManagedNamespace(0)
	kbNamespace := test.Ctx().ManagedNamespace(1)
	name := "test-cross-ns-assoc"
	esBuilder := elasticsearch.NewBuilder(name).
	WithNamespace(esNamespace).
	WithESMasterDataNodes(1, elasticsearch.DefaultResources).
	WithRestrictedSecurityContext()
	kbBuilder := kibana.NewBuilder(name).
	WithNamespace(kbNamespace).
	WithElasticsearchRef(esBuilder.Ref()).
	WithNodeCount(1).
	WithRestrictedSecurityContext()
	test.Sequence(nil, test.EmptySteps, esBuilder, kbBuilder).RunSequential(t)
	}

Which makes me think we can close this issue. Please reopen if you had additional scenarios in mind.