Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run long-running tests on a periodic basis #2074

Open
echeran opened this issue Jun 16, 2022 · 2 comments
Open

Run long-running tests on a periodic basis #2074

echeran opened this issue Jun 16, 2022 · 2 comments
Assignees
Labels
C-test-infra Component: Integration test infrastructure S-small Size: One afternoon (small bug fix or enhancement) T-docs-tests Type: Code change outside core library

Comments

@echeran
Copy link
Contributor

echeran commented Jun 16, 2022

For collator and/or normalizer components, there are fuzz tests that are long-running but not included in the cargo test set of tests, or in #2039 the conformance tests are not actually included in the repo to save space in the repo, but those can be downloaded and then the tests can be run with the actual input.

We should probably create a workflow for the long-running tests, and then configure that workflow to run periodically once in a while (frequency TBD) on main.

@echeran echeran added T-docs-tests Type: Code change outside core library C-test-infra Component: Integration test infrastructure S-small Size: One afternoon (small bug fix or enhancement) labels Jun 16, 2022
@sffc sffc added this to the ICU4X 1.x Untriaged milestone Jun 23, 2022
@echeran
Copy link
Contributor Author

echeran commented Jun 23, 2022

If we have a separate workflow, then we can configure it to run periodically in a cron job manner. Even if fuzz tests run for 24+ hours, we can run this workflow with a period size >> 24+ hrs (ex: 1 week). Perhaps make it start running early Saturday morning since weekends would have less demand for available slots for running CI jobs in Github Actions from people working on PRs.

@robertbastian
Copy link
Member

The full datagen CI is a candidate for this as well, it already takes half an hour and will become even slower with #2058.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-test-infra Component: Integration test infrastructure S-small Size: One afternoon (small bug fix or enhancement) T-docs-tests Type: Code change outside core library
Projects
None yet
Development

No branches or pull requests

4 participants