Update conformance testee for @connectrpc/connect-web #1183
Conversation
Signed-off-by: Timo Stamm <ts@timostamm.de>
timostamm
force-pushed
the
tstamm/update-connect-web-conformance
branch
from
f7a442d to
8c28884
Compare
| node: | ||
| runs-on: ubuntu-22.04 |
There was a problem hiding this comment.
We're running conformance tests for @connectrpc/connect-web on Node.js now. They run quick enough, seems worth it.
| async function extractBin(path: string) { | ||
| if (path.endsWith(".zip")) { | ||
| const unzipped = unzipSync(readFileSync(path), { | ||
| async function extractBin(archivePath: string, binPath: string) { |
There was a problem hiding this comment.
Running the connectconformance command in parallel used to fail without an error message, because this script tried to write to the file that was already running.
|
|
||
| .PHONY: testwebconformancelocal |
There was a problem hiding this comment.
This didn't work as documented. Since the npm script runs two commands with &&, the --browser flag was only appended to the second command. The replacement is explained in packages/connect-conformance/README.md
| value: new ClientErrorResult({ | ||
| message: `Failed to run test case: ${String(err)}`, | ||
| }), |
There was a problem hiding this comment.
Wrapping the error isn't helpful here, this keeps more info.
| `const p = document.createElement("p"); | ||
| p.innerText = "Tests done. You can inspect requests in the network explorer." | ||
| document.body.append(p);`, |
There was a problem hiding this comment.
document.write raises an error in Firefox.
| @@ -34,11 +33,13 @@ const { values: flags } = parseArgs({ | |||
| options: { | |||
| browser: { type: "string", default: "chrome" }, | |||
| headless: { type: "boolean" }, | |||
| openBrowser: { type: "boolean" }, | |||
There was a problem hiding this comment.
Instead of opting in to run headless with --headless, you now opt in to run with UI with --openBrowser. This allows us to re-use the npm scripts. It's explained in packages/connect-conformance/README.md
| "conformance:client:safari": "connectconformance --mode client --conf ./conformance/conformance-web.yaml -v -- ./conformance/client.ts --browser safari --headless && connectconformance --mode client --conf ./conformance/conformance-web.yaml -v --known-failing @./conformance/known-failing-callback-client.txt -- ./conformance/client.ts --browser safari --headless --useCallbackClient", | ||
| "conformance:client:node": "connectconformance --mode client --conf ./conformance/conformance-web-node.yaml -v -- ./conformance/client.ts --browser node && connectconformance --mode client --conf ./conformance/conformance-web-node.yaml -v --known-failing @./conformance/known-failing-callback-client.txt -- ./conformance/client.ts --browser node --useCallbackClient", | ||
| "conformance:client:browser": "connectconformance --mode client --conf ./conformance/conformance-web.yaml -v -- ./conformance/client.ts && connectconformance --mode client --conf ./conformance/conformance-web.yaml -v --known-failing @./conformance/known-failing-callback-client.txt -- ./conformance/client.ts --useCallbackClient", | ||
| "conformance:client:chrome:promise": "connectconformance --mode client --conf ./conformance/conformance-web.yaml -v -- ./conformance/client.ts --browser chrome", |
There was a problem hiding this comment.
That's an awkward amount of scripts, but it's useful to be able to run just one suite.
Several small changes, see below.
The Firefox known-failing cases are a caused by a bug in the Firefox webdriver:
AbortSignal.abort().reason instanceof Errorisfalsewhen run throughWebdriverIO.Browser.executeScript.We shouldn't need to ignore this. I plan to fix it in a follow-up.
The callback client known-failing cases happen because we swallow the AbortError for both client-side cancel and server-side cancel. It makes sense for the callback-API for client-side cancel, but it would be good to not do this for the server-side.