Skip to content
This repository has been archived by the owner on Aug 19, 2024. It is now read-only.

ci: Run Integration Tests as part of PR checks #365

Closed

Conversation

rm3l
Copy link
Member

@rm3l rm3l commented May 21, 2024

Description

As the PR title suggests, this allows us to enforce that integration tests are run as part of the PR validation checks.
Currently, we are only running unit tests with make test. In the past, make test used to cover a lot more via EnvTest, but this part has been moved to a make integration-test target since #272).

As I almost always happen to run this target (along with the E2E tests) when reviewing PRs, I think we should enforce those integration tests as part of the automatic PR checks. Unlike the E2E tests, they don't take that long to run and do not require a cluster to run.

Which issue(s) does this PR fix or relate to

PR acceptance criteria

  • Tests
  • Documentation
  • If the bundle manifests have been updated, make sure to review the rhdh-operator.csv.yaml file accordingly

How to test changes / Special notes to the reviewer

Job passing: https://github.com/janus-idp/operator/actions/runs/9177666830/job/25235797784?pr=365

Copy link

openshift-ci bot commented May 21, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from rm3l. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

sonarcloud bot commented May 21, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

@rm3l
Copy link
Member Author

rm3l commented May 21, 2024

/cc @gazarenkov @coreydaley

Copy link
Member

@gazarenkov gazarenkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "make test" already included integration_tests, no?

Like:

ok      redhat-developer/red-hat-developer-hub-operator/controllers     0.529s  coverage: 22.8% of statements
ok      redhat-developer/red-hat-developer-hub-operator/integration_tests       7.824s  coverage: 42.2% of statements
ok      redhat-developer/red-hat-developer-hub-operator/pkg/model       1.240s  coverage: 84.6% of statements

@rm3l
Copy link
Member Author

rm3l commented May 27, 2024

I think "make test" already included integration_tests, no?

Like:

ok      redhat-developer/red-hat-developer-hub-operator/controllers     0.529s  coverage: 22.8% of statements
ok      redhat-developer/red-hat-developer-hub-operator/integration_tests       7.824s  coverage: 42.2% of statements
ok      redhat-developer/red-hat-developer-hub-operator/pkg/model       1.240s  coverage: 84.6% of statements

🤔 Interesting.. I thought so initially, but they didn't seem to be included on my machine:

$ make test                                                            
test -s /home/asoro/work/projects/backstage/janus-idp/operator/bin/controller-gen || GOBIN=/home/asoro/work/projects/backstage/janus-idp/operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.3
/home/asoro/work/projects/backstage/janus-idp/operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/home/asoro/work/projects/backstage/janus-idp/operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
test -s /home/asoro/work/projects/backstage/janus-idp/operator/bin/goimports || GOBIN=/home/asoro/work/projects/backstage/janus-idp/operator/bin go install golang.org/x/tools/cmd/goimports@v0.15.0
find . -not -path '*/\.*' -name '*.go' -exec /home/asoro/work/projects/backstage/janus-idp/operator/bin/goimports -w {} \;
go vet ./...
test -s /home/asoro/work/projects/backstage/janus-idp/operator/bin/setup-envtest || GOBIN=/home/asoro/work/projects/backstage/janus-idp/operator/bin go install sigs.k8s.io/controller-runtime/tools/setup-envtest@c7e1dc9b5302d649d5531e19168dd7ea0013736d
mkdir -p /home/asoro/work/projects/backstage/janus-idp/operator/bin/default-config && cp config/manager/default-config/* /home/asoro/work/projects/backstage/janus-idp/operator/bin/default-config
LOCALBIN=/home/asoro/work/projects/backstage/janus-idp/operator/bin KUBEBUILDER_ASSETS="/home/asoro/work/projects/backstage/janus-idp/operator/bin/k8s/1.25.0-linux-amd64" go test redhat-developer/red-hat-developer-hub-operator redhat-developer/red-hat-developer-hub-operator/api/v1alpha1 redhat-developer/red-hat-developer-hub-operator/controllers redhat-developer/red-hat-developer-hub-operator/integration_tests redhat-developer/red-hat-developer-hub-operator/pkg/model redhat-developer/red-hat-developer-hub-operator/pkg/utils -coverprofile cover.out
?       redhat-developer/red-hat-developer-hub-operator [no test files]
?       redhat-developer/red-hat-developer-hub-operator/api/v1alpha1    [no test files]
ok      redhat-developer/red-hat-developer-hub-operator/controllers     0.013s  coverage: 22.8% of statements
?       redhat-developer/red-hat-developer-hub-operator/pkg/utils       [no test files]
ok      redhat-developer/red-hat-developer-hub-operator/pkg/model       0.057s  coverage: 85.4% of statements

@rm3l
Copy link
Member Author

rm3l commented May 27, 2024

I think "make test" already included integration_tests, no?
Like:

ok      redhat-developer/red-hat-developer-hub-operator/controllers     0.529s  coverage: 22.8% of statements
ok      redhat-developer/red-hat-developer-hub-operator/integration_tests       7.824s  coverage: 42.2% of statements
ok      redhat-developer/red-hat-developer-hub-operator/pkg/model       1.240s  coverage: 84.6% of statements

🤔 Interesting.. I thought so initially, but they didn't seem to be included on my machine:

$ make test                                                            
test -s /home/asoro/work/projects/backstage/janus-idp/operator/bin/controller-gen || GOBIN=/home/asoro/work/projects/backstage/janus-idp/operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.3
/home/asoro/work/projects/backstage/janus-idp/operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/home/asoro/work/projects/backstage/janus-idp/operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
test -s /home/asoro/work/projects/backstage/janus-idp/operator/bin/goimports || GOBIN=/home/asoro/work/projects/backstage/janus-idp/operator/bin go install golang.org/x/tools/cmd/goimports@v0.15.0
find . -not -path '*/\.*' -name '*.go' -exec /home/asoro/work/projects/backstage/janus-idp/operator/bin/goimports -w {} \;
go vet ./...
test -s /home/asoro/work/projects/backstage/janus-idp/operator/bin/setup-envtest || GOBIN=/home/asoro/work/projects/backstage/janus-idp/operator/bin go install sigs.k8s.io/controller-runtime/tools/setup-envtest@c7e1dc9b5302d649d5531e19168dd7ea0013736d
mkdir -p /home/asoro/work/projects/backstage/janus-idp/operator/bin/default-config && cp config/manager/default-config/* /home/asoro/work/projects/backstage/janus-idp/operator/bin/default-config
LOCALBIN=/home/asoro/work/projects/backstage/janus-idp/operator/bin KUBEBUILDER_ASSETS="/home/asoro/work/projects/backstage/janus-idp/operator/bin/k8s/1.25.0-linux-amd64" go test redhat-developer/red-hat-developer-hub-operator redhat-developer/red-hat-developer-hub-operator/api/v1alpha1 redhat-developer/red-hat-developer-hub-operator/controllers redhat-developer/red-hat-developer-hub-operator/integration_tests redhat-developer/red-hat-developer-hub-operator/pkg/model redhat-developer/red-hat-developer-hub-operator/pkg/utils -coverprofile cover.out
?       redhat-developer/red-hat-developer-hub-operator [no test files]
?       redhat-developer/red-hat-developer-hub-operator/api/v1alpha1    [no test files]
ok      redhat-developer/red-hat-developer-hub-operator/controllers     0.013s  coverage: 22.8% of statements
?       redhat-developer/red-hat-developer-hub-operator/pkg/utils       [no test files]
ok      redhat-developer/red-hat-developer-hub-operator/pkg/model       0.057s  coverage: 85.4% of statements

They appear to be included in the CI, but I still am not sure why they are not on my machine. Closing this PR since they run in CI - will look into the local issue later.

/close

@openshift-ci openshift-ci bot closed this May 27, 2024
Copy link

openshift-ci bot commented May 27, 2024

@rm3l: Closed this PR.

In response to this:

I think "make test" already included integration_tests, no?
Like:

ok      redhat-developer/red-hat-developer-hub-operator/controllers     0.529s  coverage: 22.8% of statements
ok      redhat-developer/red-hat-developer-hub-operator/integration_tests       7.824s  coverage: 42.2% of statements
ok      redhat-developer/red-hat-developer-hub-operator/pkg/model       1.240s  coverage: 84.6% of statements

🤔 Interesting.. I thought so initially, but they didn't seem to be included on my machine:

$ make test                                                            
test -s /home/asoro/work/projects/backstage/janus-idp/operator/bin/controller-gen || GOBIN=/home/asoro/work/projects/backstage/janus-idp/operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.11.3
/home/asoro/work/projects/backstage/janus-idp/operator/bin/controller-gen rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/home/asoro/work/projects/backstage/janus-idp/operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
test -s /home/asoro/work/projects/backstage/janus-idp/operator/bin/goimports || GOBIN=/home/asoro/work/projects/backstage/janus-idp/operator/bin go install golang.org/x/tools/cmd/goimports@v0.15.0
find . -not -path '*/\.*' -name '*.go' -exec /home/asoro/work/projects/backstage/janus-idp/operator/bin/goimports -w {} \;
go vet ./...
test -s /home/asoro/work/projects/backstage/janus-idp/operator/bin/setup-envtest || GOBIN=/home/asoro/work/projects/backstage/janus-idp/operator/bin go install sigs.k8s.io/controller-runtime/tools/setup-envtest@c7e1dc9b5302d649d5531e19168dd7ea0013736d
mkdir -p /home/asoro/work/projects/backstage/janus-idp/operator/bin/default-config && cp config/manager/default-config/* /home/asoro/work/projects/backstage/janus-idp/operator/bin/default-config
LOCALBIN=/home/asoro/work/projects/backstage/janus-idp/operator/bin KUBEBUILDER_ASSETS="/home/asoro/work/projects/backstage/janus-idp/operator/bin/k8s/1.25.0-linux-amd64" go test redhat-developer/red-hat-developer-hub-operator redhat-developer/red-hat-developer-hub-operator/api/v1alpha1 redhat-developer/red-hat-developer-hub-operator/controllers redhat-developer/red-hat-developer-hub-operator/integration_tests redhat-developer/red-hat-developer-hub-operator/pkg/model redhat-developer/red-hat-developer-hub-operator/pkg/utils -coverprofile cover.out
?       redhat-developer/red-hat-developer-hub-operator [no test files]
?       redhat-developer/red-hat-developer-hub-operator/api/v1alpha1    [no test files]
ok      redhat-developer/red-hat-developer-hub-operator/controllers     0.013s  coverage: 22.8% of statements
?       redhat-developer/red-hat-developer-hub-operator/pkg/utils       [no test files]
ok      redhat-developer/red-hat-developer-hub-operator/pkg/model       0.057s  coverage: 85.4% of statements

They appear to be included in the CI, but I still am not sure why they are not on my machine. Closing this PR since they run in CI - will look into the local issue later.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@rm3l rm3l deleted the run_integration_tests_as_part_of_pr_checks branch May 27, 2024 11:32
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants