CI: add Playwright or wasm-bindgen-test testing for examples and run them using cargo-make

[gbj]

No description provided.

[agilarity]

Progress Report

GitHub Workflows

Example workflows only run when needed.

• Check Examples runs when a leptos source package is changed
• Verfiy All Examples runs daily and on version change
• Verify Changed Examples runs for examples that changed

Cargo Make

Custom cargo make tasks and files make it easy to configure and run ci consistently.

• cargo make tasks run at the workspace and project levels
• cargo make files provide a one-line strategy to add support for WASM Test, Playwright Test, Cargo Leptos Test, and Trunk

CI Testing

CI test runners by example...

counter -C-W
counters -W
counters_stable -W-P
counter_without_macros -R-W
js-framework-benchmark -W
router -P
tailwind -L

Test Runners: C = Cargo Test, L = Cargo Leptos Test, P = Playwright Test, R = RS Test, W = WASM Test

• 7 examples (26.9%) have some ci testing
• Examples demonstrate testing with WASM and Playwright
• Examples demonstrate two ways to start the client for e2e testing (See counters_stable and tailwind)

What's Left?

We have a stable example testing platform with a nice DX.

Apart from adding more tests, one potential improvements come to mind.

• Pull up nightly build and check tasks into cargo-make (build(examples): pull up compile tasks #1417)
• Only run Check Stable workflow on leptos source change

@gbj Is there anything else that should be considered?

[gbj]

@gbj Is there anything else that should be considered?

No, I think this is really comprehensive. Your work here has been really astounding, and it's completely outside my area of knowledge so I appreciate it very much.

CI process-wise, your pending "Only run Check Stable on leptos source change" sounds right. I can't think of any other changes to the process.

Test-wise, I think the most useful remaining tests to add are

• todo_app_sqlite, which would essentially test that <Suspense/> and hydration are working at any given moment (tests can be copied over to todo_app_sqlite_axum, with same functionality)
• leptos/tests/test_examples/suspense-tests: On each of the possible URLs for this page, a full reload, waiting 2.5 seconds, and then clicking the button should lead to correct button reloads and no errors/warnings in console. (I tend to use this manually quite a bit when working on <Suspense/> issues. It's not really essential to have automated testing for it but would be cool.)

I don't think we need 100% example test coverage. What's already tested + the above sufficiently tests the features of the framework, in my opinion, as well as providing examples of how to test effectively, which are the two goals.

[gbj]

I'm going to close this as completed. Thank you @agilarity for a huge amount of work on this. We're now testing a wide range of features of the framework pretty thoroughly via the examples, and these provide a really strong reference for how users can implement testing in their own apps. Feel free to open separate issues in the future for additional testing needs, but this one has been well done.