Document running e2e test with non gcloud cluster #1833

aslom · 2019-09-06T18:40:13Z

Problem

Following https://github.com/knative/eventing/tree/master/test
used ./e2e-tests.sh --run-tests but fails as I do not have gcloud - I do have k8s cluster $KUBECONFIG

(base) aslom@m test (conformance-test-channel-headers-2) $ ./e2e-tests.sh --run-tests
=================================
==== SETTING UP TEST CLUSTER ====
=================================
==== Fri Sep  6 11:13:08 PDT 2019
=================================
(unset)
(unset)
Acquiring cluster-admin role for user 'keventing-e2e-cls'
ERROR: (gcloud.container.clusters.describe) You do not currently have an active account selected.
Please run:

  $ gcloud auth login

to obtain new credentials, or if you have already logged in with a
different account:

  $ gcloud config set account ACCOUNT

to select an already authenticated account to use.
Certificate in /var/folders/pc/8hqz8zns0_q4xznqt19pxmkw0000gn/T/tmp.zTSDQYWx, key in /var/folders/pc/8hqz8zns0_q4xznqt19pxmkw0000gn/T/tmp.wS95ikst
base64: invalid option -- d
Usage:	base64 [-hvD] [-b num] [-i in_file] [-o out_file]
  -h, --help     display this message
  -D, --decode   decodes input
  -b, --break    break encoded string into num character lines
  -i, --input    input file (default: "-" for stdin)
  -o, --output   output file (default: "-" for stdout)
ERROR: (gcloud.container.clusters.describe) You do not currently have an active account selected.
Please run:

  $ gcloud auth login

to obtain new credentials, or if you have already logged in with a
different account:

  $ gcloud config set account ACCOUNT

to select an already authenticated account to use.
(base) aslom@m test (conformance-test-channel-headers-2) $

and with --skip-knative-setup

(base) aslom@m eventing (conformance-test-channel-headers-2) $ test/e2e-tests.sh --run-tests --skip-knative-setup
=================================
==== SETTING UP TEST CLUSTER ====
=================================
==== Fri Sep  6 11:38:03 PDT 2019
=================================
(unset)
(unset)
Acquiring cluster-admin role for user 'keventing-e2e-cls'
ERROR: (gcloud.container.clusters.describe) You do not currently have an active account selected.
Please run:

  $ gcloud auth login

to obtain new credentials, or if you have already logged in with a
different account:

  $ gcloud config set account ACCOUNT

to select an already authenticated account to use.
Certificate in /var/folders/pc/8hqz8zns0_q4xznqt19pxmkw0000gn/T/tmp.146nMh2c, key in /var/folders/pc/8hqz8zns0_q4xznqt19pxmkw0000gn/T/tmp.VLjheEt1
base64: invalid option -- d
Usage:	base64 [-hvD] [-b num] [-i in_file] [-o out_file]
  -h, --help     display this message
  -D, --decode   decodes input
  -b, --break    break encoded string into num character lines
  -i, --input    input file (default: "-" for stdin)
  -o, --output   output file (default: "-" for stdout)
ERROR: (gcloud.container.clusters.describe) You do not currently have an active account selected.
Please run:

  $ gcloud auth login

to obtain new credentials, or if you have already logged in with a
different account:

  $ gcloud config set account ACCOUNT

to select an already authenticated account to use.
(base) aslom@m eventing (conformance-test-channel-headers-2) $

Persona:
Which persona is this feature for?

Exit Criteria
A measurable (binary) test that would indicate that the problem has been resolved.

Can run any k8s cluster

Time Estimate (optional):
How many developer-days do you think this may take to resolve?
1

Additional context (optional)
Add any other context about the feature request here.

Helps contributors that do not have gcloud account to contribute

The text was updated successfully, but these errors were encountered:

aslom · 2019-09-06T19:10:38Z

/assign @Fredy-Z

Let me know if I am missing something. I would happy to test it

aslom · 2019-09-09T19:12:29Z

/assign @grantr

grantr · 2019-09-09T20:03:49Z

Acquiring cluster-admin role for user 'keventing-e2e-cls'

@adrcunha @chaodaiG were there changes to the e2e test process recently that would have made this stop working without gcloud?

chizhg · 2019-09-09T20:53:29Z

Acquiring cluster-admin role for user 'keventing-e2e-cls'

@adrcunha @chaodaiG were there changes to the e2e test process recently that would have made this stop working without gcloud?

I think it's line 327-332 that caused the script not being able to run without gcloud.
https://github.com/knative/test-infra/blob/fabb00f0c0a9cb3707c594488b4114ea968859fd/scripts/e2e-tests.sh#L327

@adrcunha do you remember why we need the cluster admin role in the e2e tests?

adrcunha · 2019-09-09T21:06:29Z

Acquiring cluster-admin role for user 'keventing-e2e-cls'

@adrcunha @chaodaiG were there changes to the e2e test process recently that would have made this stop working without gcloud?

No. This is required for all E2E tests in order to install Knative.

adrcunha · 2019-09-09T21:08:30Z

Acquiring cluster-admin role for user 'keventing-e2e-cls'

@adrcunha @chaodaiG were there changes to the e2e test process recently that would have made this stop working without gcloud?

I think it's line 327-332 that caused the script not being able to run without gcloud.
https://github.com/knative/test-infra/blob/fabb00f0c0a9cb3707c594488b4114ea968859fd/scripts/e2e-tests.sh#L327

It doesn't make sense, this is old code. Are you sure you're not creating a cluster or. nstalling Knative in a non-standard way?

@adrcunha do you remember why we need the cluster admin role in the e2e tests?

Because it's required to install Knative. That's been this way since at least for a year. If this is broken, almost all E2E tests should be broken then.

grantr · 2019-09-09T21:12:48Z

Oops, my mistake. I interpreted cluster admin as a GCP thing, but you're right it's a K8s thing. It's been a long time since I thought about it.

adrcunha · 2019-09-09T22:45:13Z

The README file doesn't mention it, but --run-tests won't work outside a GCP environment.

chizhg · 2019-09-10T02:14:53Z

I double-checked the Knative installation guidance for different Kubernetes vendors in https://knative.dev/docs/install/, it seems the cluster admin role is only required by GKE (or the document is wrong?)

Further more, I tried to comment out line 327-332, install a minikube, and then run ./test/e2e-tests.sh --run-tests. The script was able to finish, though with a couple of test cases failing (which might be bugs of eventing itself)

This experiment does prove that knative/test-infra#1339 can make --run-tests with other Kubernetes cluster vendors, though the tests are not guaranteed to pass.

aslom · 2019-09-10T13:45:22Z

@Fredy-Z @grantr @adrcunha thank you fir helping wiht it and testing it. I will try commenting and test it with minikube, external cluster, and other test options like https://kind.sigs.k8s.io

If it works I am going to submit PR

aslom · 2019-09-10T14:44:21Z

I have tested with kind cluster and could nto run until commented out the lines 327-332 in vendor/knative.dev/test-infra/scripts/e2e-tests.sh

# If cluster admin role isn't set, this is a brand new cluster
  # Setup the admin role and also KO_DOCKER_REPO
  if [[ -z "$(kubectl get clusterrolebinding cluster-admin-binding 2> /dev/null)" ]]; then
    acquire_cluster_admin_role ${k8s_user} ${E2E_CLUSTER_NAME} ${E2E_CLUSTER_REGION} ${E2E_CLUSTER_ZONE}
    kubectl config set-context ${k8s_cluster} --namespace=default
    # Incorporate an element of randomness to ensure that each run properly publishes images.
    export KO_DOCKER_REPO=gcr.io/${E2E_PROJECT_ID}/${E2E_BASE_NAME}-e2e-img/${RANDOM}
  fi

Then it started tests until this error below - is it the error you see @Fredy-Z ?

{"level":"info","ts":"2019-09-10T14:06:30.475Z","logger":"source_controller.apiserver-source-controller","caller":"controller/controller.go:290","msg":"Started workers","commit":"7f43344","knative.dev/controller":"apiserver-source-controller"}
{"level":"info","ts":"2019-09-10T14:06:30.476Z","logger":"source_controller.container-source-controller","caller":"controller/controller.go:280","msg":"Starting controller and workers","commit":"7f43344","knative.dev/controller":"container-source-controller"}
{"level":"info","ts":"2019-09-10T14:06:30.476Z","logger":"source_controller.container-source-controller","caller":"controller/controller.go:290","msg":"Started workers","commit":"7f43344","knative.dev/controller":"container-source-controller"}
----------------------------------------------------------
Namespace, Pod, Container (Previous instance): knative-eventing, sources-controller-5b5d66646-qxtc4, controller
Error from server (BadRequest): previous terminated container "controller" in pod "sources-controller-5b5d66646-qxtc4" not found
============================================================
***************************************
***         E2E TEST FAILED         ***
***     End of information dump     ***
***************************************
=======================================
==== TEARING DOWN TEST ENVIRONMENT ====
=======================================
==== Tue Sep 10 07:25:23 PDT 2019
=======================================
Uninstalling In-Memory Channel CRD
clusterrole.rbac.authorization.k8s.io "imc-addressable-resolver" deleted
clusterrole.rbac.authorization.k8s.io "imc-channelable-manipulator" deleted
clusterrole.rbac.authorization.k8s.io "imc-controller" deleted
clusterrole.rbac.authorization.k8s.io "imc-dispatcher" deleted
service "imc-dispatcher" deleted
serviceaccount "imc-controller" deleted
serviceaccount "imc-dispatcher" deleted
clusterrolebinding.rbac.authorization.k8s.io "imc-controller" deleted
clusterrolebinding.rbac.authorization.k8s.io "imc-dispatcher" deleted
customresourcedefinition.apiextensions.k8s.io "inmemorychannels.messaging.knative.dev" deleted
deployment.apps "imc-controller" deleted
deployment.apps "imc-dispatcher" deleted
(base) aslom@m eventing (master) $

aslom · 2019-09-10T15:31:30Z

I was looking for the first error - it seems to be

=== RUN   TestDefaultBrokerWithManyTriggers/test_default_broker_with_many_attribute_and_extension_triggers
=== PAUSE TestDefaultBrokerWithManyTriggers/test_default_broker_with_many_attribute_and_extension_triggers
=== CONT  TestDefaultBrokerWithManyTriggers/test_default_broker_with_many_attribute_triggers
=== CONT  TestDefaultBrokerWithManyTriggers/test_default_broker_with_many_attribute_and_extension_triggers
=== CONT  TestDefaultBrokerWithManyTriggers/test_default_broker_with_many_deprecated_triggers
--- FAIL: TestDefaultBrokerWithManyTriggers (0.09s)
    --- FAIL: TestDefaultBrokerWithManyTriggers/test_default_broker_with_many_attribute_and_extension_triggers (0.04s)
        test_runner.go:80: namespace is : "test-default-broker-with-many-attribute-and-extension-triggers"
        broker_default_test.go:141: Error annotating namespace: Get https://kubernetes.docker.internal:6443/api/v1/namespaces/test-default-broker-with-many-attribute-and-extension-triggers: EOF
        test_runner.go:114: Could not delete the namespace "test-default-broker-with-many-attribute-and-extension-triggers": Delete https://kubernetes.docker.internal:6443/api/v1/namespaces/test-default-broker-with-many-attribute-and-extension-triggers: EOF
    --- FAIL: TestDefaultBrokerWithManyTriggers/test_default_broker_with_many_deprecated_triggers (0.05s)
        test_runner.go:80: namespace is : "test-default-broker-with-many-deprecated-triggers"
        broker_default_test.go:141: Error annotating namespace: Get https://kubernetes.docker.internal:6443/api/v1/namespaces/test-default-broker-with-many-deprecated-triggers: EOF
        test_runner.go:114: Could not delete the namespace "test-default-broker-with-many-deprecated-triggers": Delete https://kubernetes.docker.internal:6443/api/v1/namespaces/test-default-broker-with-many-deprecated-triggers: EOF
    --- FAIL: TestDefaultBrokerWithManyTriggers/test_default_broker_with_many_attribute_triggers (0.03s)
        test_runner.go:80: namespace is : "test-default-broker-with-many-attribute-triggers"
        broker_default_test.go:141: Error annotating namespace: Get https://kubernetes.docker.internal:6443/api/v1/namespaces/test-default-broker-with-many-attribute-triggers: EOF
        test_runner.go:114: Could not delete the namespace "test-default-broker-with-many-attribute-triggers": Delete https://kubernetes.docker.internal:6443/api/v1/namespaces/test-default-broker-with-many-attribute-triggers: EOF
=== RUN   TestEventTransformationForTrigger

any idea what could be causing broker_default_test.go:141: Error annotating namespace: Get https://kubernetes.docker.internal:6443/api/v1/namespaces/test-default-broker-with-many-attribute-triggers: EOF

go			// Label namespace so that it creates the default broker.
			if err := client.LabelNamespace(map[string]string{"knative-eventing-injection": "enabled"}); err != nil {
				t.Fatalf("Error annotating namespace: %v", err)
			}

https://github.com/knative/eventing/blob/master/test/e2e/broker_default_test.go#L141

lionelvillard · 2019-09-10T15:52:53Z

For me the first error I get is this one:

>> Publishing test images
./e2e-common.sh: line 59: ./upload-test-images.sh: No such file or directory

lionelvillard · 2019-09-10T15:57:25Z

which is fixed by running the tests from the eventing root dir:

eventing$ test/e2e-tests.sh --run-tests

duglin · 2019-09-11T00:17:46Z

I managed to get it to work (with a few hacks) by using:

go test -timeout=20m -tags=e2e -count=1 ./test/e2e

is there any reason to use the e2e-tests.sh script if I already have an env set up?

duglin · 2019-09-11T00:22:40Z

This issue is related: #1862

adrcunha · 2019-09-11T00:32:28Z

is there any reason to use the e2e-tests.sh script if I already have an env set up?

No. You actually shouldn't use the script if.you have an env setup. That's why I said the readme is outdated; see the one in serving, which mentions exactly what I just said.

paulrossman · 2019-09-16T17:50:36Z

@Fredy-Z could you update the readme?

n3wscott · 2020-08-18T16:43:07Z

This is one of the goals of the knative-sandbox/integration repo.

github-actions · 2020-12-26T01:52:57Z

This issue is stale because it has been open for 90 days with no
activity. It will automatically close after 30 more days of
inactivity. Reopen the issue with /reopen. Mark the issue as
fresh by adding the comment /remove-lifecycle stale.

Signed-off-by: Matthias Wessendorf <mwessend@redhat.com>

aslom added the kind/feature-request label Sep 6, 2019

aslom changed the title ~~More details on how to run e2e test with non gcloud cluster~~ Document running e2e test with non gcloud cluster Sep 6, 2019

aslom mentioned this issue Sep 6, 2019

Conformance test channel headers 2 #1813

Merged

knative-prow-robot assigned chizhg Sep 6, 2019

knative-prow-robot assigned grantr Sep 9, 2019

grantr added this to the Backlog milestone Aug 24, 2020

github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Dec 26, 2020

github-actions bot closed this as completed Jan 25, 2021

maschmid pushed a commit to maschmid/eventing that referenced this issue Aug 9, 2022

Ko publish changed, hence adjusting the patch 💄 (knative#1833)

d28f572

Signed-off-by: Matthias Wessendorf <mwessend@redhat.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Document running e2e test with non gcloud cluster #1833

Document running e2e test with non gcloud cluster #1833

aslom commented Sep 6, 2019

aslom commented Sep 6, 2019

aslom commented Sep 9, 2019

grantr commented Sep 9, 2019

chizhg commented Sep 9, 2019 •

edited

Loading

adrcunha commented Sep 9, 2019

adrcunha commented Sep 9, 2019

grantr commented Sep 9, 2019

adrcunha commented Sep 9, 2019

chizhg commented Sep 10, 2019 •

edited

Loading

aslom commented Sep 10, 2019

aslom commented Sep 10, 2019

aslom commented Sep 10, 2019

lionelvillard commented Sep 10, 2019

lionelvillard commented Sep 10, 2019

duglin commented Sep 11, 2019

duglin commented Sep 11, 2019

adrcunha commented Sep 11, 2019

paulrossman commented Sep 16, 2019

n3wscott commented Aug 18, 2020

github-actions bot commented Dec 26, 2020

Document running e2e test with non gcloud cluster #1833

Document running e2e test with non gcloud cluster #1833

Comments

aslom commented Sep 6, 2019

aslom commented Sep 6, 2019

aslom commented Sep 9, 2019

grantr commented Sep 9, 2019

chizhg commented Sep 9, 2019 • edited Loading

adrcunha commented Sep 9, 2019

adrcunha commented Sep 9, 2019

grantr commented Sep 9, 2019

adrcunha commented Sep 9, 2019

chizhg commented Sep 10, 2019 • edited Loading

aslom commented Sep 10, 2019

aslom commented Sep 10, 2019

aslom commented Sep 10, 2019

lionelvillard commented Sep 10, 2019

lionelvillard commented Sep 10, 2019

duglin commented Sep 11, 2019

duglin commented Sep 11, 2019

adrcunha commented Sep 11, 2019

paulrossman commented Sep 16, 2019

n3wscott commented Aug 18, 2020

github-actions bot commented Dec 26, 2020

chizhg commented Sep 9, 2019 •

edited

Loading

chizhg commented Sep 10, 2019 •

edited

Loading