[SDK-3997] Upload test results to observability server #1551
Conversation
github-actions
bot
temporarily deployed
to
staging/pull/1551/features
github-actions
bot
temporarily deployed
to
staging/pull/1551/bundle-report
github-actions
bot
temporarily deployed
to
staging/pull/1551/typedoc
|
Sorry, just realised there's still a |
lawrence-forooghian
force-pushed
the
1548-upload-test-results-to-observability-server
branch
from
056f4c9 to
3c684b3
Compare
github-actions
bot
temporarily deployed
to
staging/pull/1551/features
github-actions
bot
temporarily deployed
to
staging/pull/1551/bundle-report
github-actions
bot
temporarily deployed
to
staging/pull/1551/typedoc
lawrence-forooghian
requested review from
VeskeR and
owenpearson
and removed request for
VeskeR
|
Sorry about that — ready for review now. |
github-actions
bot
temporarily deployed
to
staging/pull/1551/features
github-actions
bot
temporarily deployed
to
staging/pull/1551/bundle-report
github-actions
bot
temporarily deployed
to
staging/pull/1551/typedoc
|
I've pushed 0f0dbb3 to fix a couple of mistakes. Will squash it once the PR is approved; I don't want to rebase it at the moment since I've got a couple of other WIP branches that are based on an earlier commit in this PR and I don't want to sever that connection right now. |
There was a problem hiding this comment.
Integration of mocha-junit-reporter LGTM. Happy to merge after comments are resolved.
Also, any particular reason we don't do the same for react-hooks tests? It seems like vitest has a built-in JUnit reporter too.
The test reports emitted by mocha-junit-reporter have <testcase> tags
with slightly weird `name` and `classname` properties, e.g.
<testsuite name="rest/defaults">
<testcase name="rest/defaults Init with given environment" classname="Init with given environment">
<testcase/>
</testsuite>
This looks a bit odd in the observability server web UI, which displays
the testcase.name and testcase.classname properties. I don’t know
whether it’d be better to do some pre-upload manipulation of the ably-js
reports, or to change the fields displayed in the web UI. Not going to
do anything about it now; it’s usable enough and I don’t want to mess up
the display of the ably-cocoa test reports.
Co-authored-by: Owen Pearson <owen.pearson@ably.com>
Resolves #1548. Co-authored-by: Owen Pearson <owen.pearson@ably.com>
lawrence-forooghian
force-pushed
the
1548-upload-test-results-to-observability-server
branch
from
0f0dbb3 to
95e0e68
Compare
I didn't even consider it, to be honest. The normal test suite is where we've been seeing the flakiness, as far as I'm aware. Also the observability server doesn't currently support differentiating between different test suites in a given repository. |
I see now, seems reasonable. No need for |
lawrence-forooghian
deleted the
1548-upload-test-results-to-observability-server
branch
Generates JUnit format results for the Node and browser tests, and uploads them to the test observability server. Based on work started by @owenpearson in #1009. See commit messages for more details.
Here is an example upload with some failures. (Before merging this PR, I'll delete all of the existing ably-js uploads so as to not skew the results with these example failures.)
Note that the observability server UI doesn't currently display a link to the individual matrix job that caused the upload. This is addressed by the following open PRs:
Resolves #1548.