Add flags which allow to pass namespace to e2e tests #1623

akremsa · 2020-06-11T14:39:49Z

What type of PR is this?

/kind feature

What this PR does / Why we need it:
We need this flag in order to be able to pass a randomly generated namespace to e2e tests.

Which issue(s) this PR fixes:
Closes #1074

agones-bot · 2020-06-11T14:42:50Z

Build Failed 😱

Build Id: d96a0a39-6643-48bf-86ea-cfc7d47b15b7

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2020-06-11T15:11:23Z

Build Succeeded 👏

Build Id: 65035d33-96de-45c8-9319-6e06612cfa38

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:1.7.0-23ddf83
image: gcr.io/agones-images/agones-sdk:1.7.0-23ddf83
image: gcr.io/agones-images/agones-ping:1.7.0-23ddf83
Linux C++ SDK (build): agonessdk-1.7.0-23ddf83-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.7.0-23ddf83.zip

A preview of the website (the last 30 builds are retained):

https://23ddf83-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/1623/head:pr_1623 && git checkout pr_1623
helm install ./install/helm/agones --namespace agones-system --name agones --set agones.image.tag=1.7.0-23ddf83

pooneh-m · 2020-06-11T17:38:08Z

How about creating a new namespace at the test runtime?

aLekSer · 2020-06-11T19:51:58Z

May be we can use both options, a flag to generate the namespace and another option as in this PR. Do you mean we should add ServiceAccount and ClusterRole per gameservers.namespaces not in make install step but in the test?

agones/install/helm/agones/templates/serviceaccounts/sdk.yaml

Line 30 in 54f6a38

kind: ClusterRole

pooneh-m · 2020-06-11T21:45:09Z

May be we can use both options, a flag to generate the namespace and another option as in this PR. Do you mean we should add ServiceAccount and ClusterRole per gameservers.namespaces not in make install step but in the test?

agones/install/helm/agones/templates/serviceaccounts/sdk.yaml

Line 30 in 54f6a38

kind: ClusterRole

CreateNamespace() also sets up a new namespaces with ServiceAccount but for test only. If your motivation to add the flag is for testing on a non-default namespace, using CreateNamespace() does that for you, with all the required setup.

pooneh-m · 2020-06-12T05:28:02Z

The original idea for #1074 was to create a namespace per test suite at runtime and use the new namespace instead of default to help with isolation and debugging e.g. here. Is there a usecase for passing in a namespace as an argument?

aLekSer · 2020-06-12T11:49:48Z

@pooneh-m, now I got your idea.
I think for backward compatibility, let's leave an option to use default or any other namespace. If namespace is an empty string then use random approach you are suggesting.
So running:
make test-e2e GAMESERVERS_NAMESPACE=default
is equivalent to current state.
make test-e2e new randomised approach.
createNamespace(time.Now()) could be used for example.
This way we can switch back easily if something goes wrong.
And no changes to e2e.sh is needed in Cloud Build.
By the way CreateNamespace() should be renamed to CreateNamespaceWithServiceAccount() or just include this into comment in:
https://github.com/googleforgames/agones/blob/master/test/e2e/framework/framework.go#L490

akremsa · 2020-06-12T12:14:24Z

I will update this PR

akremsa · 2020-06-16T13:27:38Z

@pooneh-m @aLekSer please, review this PR again.

At this moment there is the following behavior:
GAMESERVERS_NAMESPACE is empty by default, so random namespace will be created after you run make test-e2e.
If you specify a namespace like make test-e2e GAMESERVERS_NAMESPACE="xbox", agones will run tests against specified namespace. If there is no such a namespace - error will be returned. In this case, you need to create a namespace beforehand.

One remark: I've noticed that in case if an error is returned after namespace is created and tests fail with error - that recently created namespace will remain. Should I add better error validations to remove dangling namespaces or it's better to do that in a separate PR?

aLekSer · 2020-06-16T13:32:22Z

Overall looks good, but I think we should give some hints how to run test-e2e with and without GAMESERVERS_NAMESPACE flag in build/README.md and probably additional comment in Makefile itself is needed.

aLekSer

Minor comments left about additional details on flags usage.

aLekSer · 2020-06-16T16:59:58Z

E2E test failed:

time="2020-06-16 13:58:19.721" level=info msg="failing Allocate request" error="rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = \"transport: authentication handshake failed: x509: cannot validate certificate for 35.233.192.59 because it doesn't contain any IP SANs\""
--- FAIL: TestTlsWebhook (391.55s)
    fleetautoscaler_test.go:636: error waiting for fleet condition on fleet simple-fleet-778hp
FAIL
time="2020-06-16 14:05:07.569" level=info msg="Namespace 1592315887 is deleted"

pooneh-m · 2020-06-16T17:17:02Z

@pooneh-m @aLekSer please, review this PR again.

Looks great!!! Thanks for the change.

At this moment there is the following behavior:
GAMESERVERS_NAMESPACE is empty by default, so random namespace will be created after you run make test-e2e.
If you specify a namespace like make test-e2e GAMESERVERS_NAMESPACE="xbox", agones will run tests against specified namespace. If there is no such a namespace - error will be returned. In this case, you need to create a namespace beforehand.

One remark: I've noticed that in case if an error is returned after namespace is created and tests fail with error - that recently created namespace will remain. Should I add better error validations to remove dangling namespaces or it's better to do that in a separate PR?

The defer should clean up the namespace, unless there is a failure with deleting the namespace. If the namespace is pre-created, then the creator of the namespace should clean up.

test/e2e/main_test.go

akremsa · 2020-06-17T14:14:01Z

@aLekSer thanks for comments. Updated.

google-oss-robot · 2020-06-17T14:14:34Z

New changes are detected. LGTM label has been removed.

agones-bot · 2020-06-17T14:25:48Z

Build Failed 😱

Build Id: 58f771ac-fef4-4e49-803b-4d677a62a5bf

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2020-06-17T15:28:33Z

Build Succeeded 👏

Build Id: 9f50c457-e8a8-4459-a463-4b78dd32648e

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:1.7.0-a1e7b21
image: gcr.io/agones-images/agones-sdk:1.7.0-a1e7b21
image: gcr.io/agones-images/agones-ping:1.7.0-a1e7b21
Linux C++ SDK (build): agonessdk-1.7.0-a1e7b21-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.7.0-a1e7b21.zip

A preview of the website (the last 30 builds are retained):

https://a1e7b21-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/1623/head:pr_1623 && git checkout pr_1623
helm install ./install/helm/agones --namespace agones-system --name agones --set agones.image.tag=1.7.0-a1e7b21

test/e2e/allocator_test.go

agones-bot · 2020-06-18T11:26:33Z

Build Failed 😱

Build Id: 63d01592-d64e-45f2-9d6d-3673cf58b56e

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2020-06-18T14:20:29Z

Build Failed 😱

Build Id: 03887d1b-9a3e-4707-bfc1-cc6ba2296808

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2020-06-18T14:37:28Z

Build Failed 😱

Build Id: ea19c2cc-f01d-4d71-8a86-c8c48b5a5686

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2020-06-18T14:56:25Z

Build Failed 😱

Build Id: 5ccd4fc5-16f5-402f-86cd-1e93393506d4

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

namespace flag is empty by default

update

agones-bot · 2020-06-18T15:32:00Z

Build Succeeded 👏

Build Id: 322c2409-8231-465b-9087-8f15c45e947d

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:1.7.0-9dd28ad
image: gcr.io/agones-images/agones-sdk:1.7.0-9dd28ad
image: gcr.io/agones-images/agones-ping:1.7.0-9dd28ad
Linux C++ SDK (build): agonessdk-1.7.0-9dd28ad-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.7.0-9dd28ad.zip

A preview of the website (the last 30 builds are retained):

https://9dd28ad-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/1623/head:pr_1623 && git checkout pr_1623
helm install ./install/helm/agones --namespace agones-system --name agones --set agones.image.tag=1.7.0-9dd28ad

…1623) * Added flags which allow to pass namespace to e2e tests * use random namespace when flag is empty namespace flag is empty by default * Added a couple of comments related to GAMESERVERS_NAMESPACE flag * Fixed TestTlsWebhook test * reuse existing namespace in several tests update

googlebot added the cla: yes label Jun 11, 2020

google-oss-robot requested review from dzlier-gcp and pooneh-m June 11, 2020 14:39

google-oss-robot added the size/L label Jun 11, 2020

akremsa force-pushed the add-namespace-flag branch from 4c1e35a to efcf155 Compare June 11, 2020 14:41

akremsa force-pushed the add-namespace-flag branch from efcf155 to 23ddf83 Compare June 11, 2020 14:44

akremsa changed the title ~~Added flags which allow to pass namespace to e2e tests~~ Add flags which allow to pass namespace to e2e tests Jun 11, 2020

akremsa force-pushed the add-namespace-flag branch from 23ddf83 to 1239d7b Compare June 16, 2020 12:58

google-oss-robot added size/XL and removed size/L labels Jun 16, 2020

akremsa force-pushed the add-namespace-flag branch from 1239d7b to f7d8683 Compare June 16, 2020 13:20

aLekSer approved these changes Jun 16, 2020

View reviewed changes

google-oss-robot assigned aLekSer Jun 16, 2020

google-oss-robot added lgtm and removed lgtm labels Jun 16, 2020

akremsa force-pushed the add-namespace-flag branch from 4d88e50 to f666568 Compare June 16, 2020 13:40

pooneh-m approved these changes Jun 16, 2020

View reviewed changes

google-oss-robot assigned pooneh-m Jun 16, 2020

aLekSer reviewed Jun 17, 2020

View reviewed changes

test/e2e/main_test.go Outdated Show resolved Hide resolved

aLekSer reviewed Jun 17, 2020

View reviewed changes

test/e2e/main_test.go Outdated Show resolved Hide resolved

aLekSer reviewed Jun 17, 2020

View reviewed changes

test/e2e/main_test.go Outdated Show resolved Hide resolved

akremsa force-pushed the add-namespace-flag branch from 989d032 to 9a8a0a6 Compare June 17, 2020 14:14

google-oss-robot removed the lgtm label Jun 17, 2020

akremsa force-pushed the add-namespace-flag branch from 9a8a0a6 to a1e7b21 Compare June 17, 2020 15:04

pooneh-m reviewed Jun 17, 2020

View reviewed changes

test/e2e/allocator_test.go Outdated Show resolved Hide resolved

test/e2e/allocator_test.go Outdated Show resolved Hide resolved

akremsa force-pushed the add-namespace-flag branch from 1f27c31 to e07cfca Compare June 18, 2020 14:04

akremsa force-pushed the add-namespace-flag branch from e07cfca to a05c63b Compare June 18, 2020 14:26

akremsa force-pushed the add-namespace-flag branch from a05c63b to 243e65c Compare June 18, 2020 14:39

alexey-kremsa-globant added 5 commits June 18, 2020 18:10

Added flags which allow to pass namespace to e2e tests

d7f7e99

use random namespace when flag is empty

920244c

namespace flag is empty by default

Added a couple of comments related to GAMESERVERS_NAMESPACE flag

48684b9

Fixed TestTlsWebhook test

6b65d81

reuse existing namespace in several tests

9dd28ad

update

akremsa force-pushed the add-namespace-flag branch from 243e65c to 9dd28ad Compare June 18, 2020 15:10

pooneh-m merged commit ee2f9b1 into googleforgames:master Jun 18, 2020

markmandel added this to the 1.7.0 milestone Jun 30, 2020

markmandel added area/tests Unit tests, e2e tests, anything to make sure things don't break kind/feature New features for Agones labels Jun 30, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add flags which allow to pass namespace to e2e tests #1623

Add flags which allow to pass namespace to e2e tests #1623

akremsa commented Jun 11, 2020 •

edited

Loading

agones-bot commented Jun 11, 2020

agones-bot commented Jun 11, 2020

pooneh-m commented Jun 11, 2020

aLekSer commented Jun 11, 2020

pooneh-m commented Jun 11, 2020 •

edited

Loading

pooneh-m commented Jun 12, 2020

aLekSer commented Jun 12, 2020 •

edited

Loading

akremsa commented Jun 12, 2020

akremsa commented Jun 16, 2020 •

edited

Loading

aLekSer commented Jun 16, 2020

aLekSer left a comment

aLekSer commented Jun 16, 2020

pooneh-m commented Jun 16, 2020 •

edited

Loading

akremsa commented Jun 17, 2020

google-oss-robot commented Jun 17, 2020

agones-bot commented Jun 17, 2020

agones-bot commented Jun 17, 2020

agones-bot commented Jun 18, 2020

agones-bot commented Jun 18, 2020

agones-bot commented Jun 18, 2020

agones-bot commented Jun 18, 2020

agones-bot commented Jun 18, 2020

Add flags which allow to pass namespace to e2e tests #1623

Add flags which allow to pass namespace to e2e tests #1623

Conversation

akremsa commented Jun 11, 2020 • edited Loading

agones-bot commented Jun 11, 2020

agones-bot commented Jun 11, 2020

pooneh-m commented Jun 11, 2020

aLekSer commented Jun 11, 2020

pooneh-m commented Jun 11, 2020 • edited Loading

pooneh-m commented Jun 12, 2020

aLekSer commented Jun 12, 2020 • edited Loading

akremsa commented Jun 12, 2020

akremsa commented Jun 16, 2020 • edited Loading

aLekSer commented Jun 16, 2020

aLekSer left a comment

Choose a reason for hiding this comment

aLekSer commented Jun 16, 2020

pooneh-m commented Jun 16, 2020 • edited Loading

akremsa commented Jun 17, 2020

google-oss-robot commented Jun 17, 2020

agones-bot commented Jun 17, 2020

agones-bot commented Jun 17, 2020

agones-bot commented Jun 18, 2020

agones-bot commented Jun 18, 2020

agones-bot commented Jun 18, 2020

agones-bot commented Jun 18, 2020

agones-bot commented Jun 18, 2020

akremsa commented Jun 11, 2020 •

edited

Loading

pooneh-m commented Jun 11, 2020 •

edited

Loading

aLekSer commented Jun 12, 2020 •

edited

Loading

akremsa commented Jun 16, 2020 •

edited

Loading

pooneh-m commented Jun 16, 2020 •

edited

Loading