feat_: functional tests coverage #5805
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Jenkins BuildsClick to see older builds (172)
|
igor-sirotin
force-pushed
the
feat/integration-tests-coverage
branch
2 times, most recently
from
b2cd29f
to
383e145
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
✅ All tests successful. No failed tests found. Additional details and impacted files@@ Coverage Diff @@
## develop #5805 +/- ##
==========================================
Coverage ? 11.56%
==========================================
Files ? 834
Lines ? 151028
Branches ? 0
==========================================
Hits ? 17473
Misses ? 131476
Partials ? 2079
Flags with carried forward coverage won't be shown. Click here to find out more. |
igor-sirotin
force-pushed
the
feat/integration-tests-coverage
branch
3 times, most recently
from
6e498f0
to
4ae7b1f
Compare
igor-sirotin
force-pushed
the
feat/integration-tests-coverage
branch
from
6e498f0
to
c24eba8
Compare
|
You'll have to fix what user the tests in the container run as: Which means it should run as user with |
|
@jakubgs thanks you! Indeed, I completely forgot that all of this is executed in docker 🤦 I have 2 questions though.
|
|
jakubgs
reviewed
Sep 12, 2024
jakubgs
reviewed
Sep 12, 2024
jakubgs
reviewed
Sep 12, 2024
|
The coverage report is empty, because With a simple |
|
Alright, shutdown fixed by changing the stop signal: Seems that |
|
But now there's a new issue. UPD: never mind, the testswere just never launched. Because one of the anvil containers didn't start. |
|
Now the tests are simply failing 🤔 Error not in chain call
error="anvil:8545.error: Post \"http://anvil:8545\": dial tcp: lookup anvil on 127.0.0.11:53: server misbehaving"
chain=31337OK, this is fixed now. |
igor-sirotin
force-pushed
the
feat/integration-tests-coverage
branch
3 times, most recently
from
daad5db
to
c1eb094
Compare
igor-sirotin
changed the title
igor-sirotin
force-pushed
the
feat/integration-tests-coverage
branch
2 times, most recently
from
11e500b
to
e745d4d
Compare
antdanchenko
approved these changes
Sep 24, 2024
|
@status-im/devops can I please have your eyes on this as well? 🙏 |
igor-sirotin
force-pushed
the
feat/integration-tests-coverage
branch
from
e745d4d
to
c3ab708
Compare
jakubgs
approved these changes
Sep 24, 2024
| @@ -494,18 +494,15 @@ test-verif-proxy-wrapper: | |||
| CGO_CFLAGS="$(CGO_CFLAGS)" go test -v github.com/status-im/status-go/rpc -tags gowaku_skip_migrations,nimbus_light_client -run ^TestProxySuite$$ -testify.m TestRun -ldflags $(LDFLAGS) | |||
|
|
|||
|
|
|||
| run-integration-tests: SHELL := /bin/sh | |||
There was a problem hiding this comment.
Why are we removing the SHELL override here? The script just depends on Docker right? And that's always provided by the system, never by Nix shell, so this override is valid.
There was a problem hiding this comment.
@jakubgs this is because of codecov CLI tool, which we have as nix derivation:
It's the only thing, but I didn't see a simple way to avoid it here 🤷
| @@ -42,11 +58,11 @@ pipeline { | |||
| always { | |||
| script { | |||
| archiveArtifacts( | |||
| artifacts: '**/results.xml', | |||
| artifacts: 'integration-tests/reports/*.xml, integration-tests/*.log, integration-tests/coverage/coverage.html', | |||
There was a problem hiding this comment.
If this is a list we should indicate it as such:
Suggested change
| artifacts: 'integration-tests/reports/*.xml, integration-tests/*.log, integration-tests/coverage/coverage.html', | |
| artifacts: ['integration-tests/reports/*.xml, integration-tests/*.log, integration-tests/coverage/coverage.html'], |
igor-sirotin
force-pushed
the
feat/integration-tests-coverage
branch
3 times, most recently
from
06edb43
to
28e19e4
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Accompanying changes
Moved the whole
run-integration-teststarget to_assets/scripts/run_integration_tests.shBecause it became too long to be in the
Makefile.Fixed coverage uploading to codecov
There was a missing
upload-processcommand.Refactored things a bit to use relatives path and not to have looong lines
Fixed
.codecov.ymlfileFixed integration tests run to properly shutdown statusd with
stop_signal: SIGINTAdded logs and coverage CI artifacts
Run statusd containers with
INTEGRATION_TESTS_DOCKER_UIDuserFunctional tests coverage
Implemented according to https://go.dev/blog/integration-test-coverage
You can see the results in the Codecov comment (to be improved):
Or at the codecov report page: filtered for
functionalflag: