ci: Optimize build + test pipeline #3300
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportPatch coverage has no change and project coverage change:
Additional details and impacted files@@ Coverage Diff @@
## develop #3300 +/- ##
============================================
- Coverage 86.24% 52.40% -33.84%
============================================
Files 233 234 +1
Lines 12493 12642 +149
============================================
- Hits 10774 6625 -4149
- Misses 1719 6017 +4298
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
frascuchon
force-pushed
the
tests/review-test-workflow-in-CI
branch
from
93157ab
to
42013d9
Compare
frascuchon
changed the title
frascuchon
force-pushed
the
tests/review-test-workflow-in-CI
branch
2 times, most recently
from
8251f01
to
cbbd8a6
Compare
frascuchon
force-pushed
the
tests/review-test-workflow-in-CI
branch
from
cbbd8a6
to
de5d908
Compare
frascuchon
force-pushed
the
tests/review-test-workflow-in-CI
branch
from
1440e16
to
13ed3a6
Compare
frascuchon
force-pushed
the
tests/review-test-workflow-in-CI
branch
from
79a0149
to
efb6cd1
Compare
frascuchon
force-pushed
the
tests/review-test-workflow-in-CI
branch
2 times, most recently
from
504335b
to
14f4501
Compare
frascuchon
force-pushed
the
tests/review-test-workflow-in-CI
branch
from
14f4501
to
339204f
Compare
gabrielmbmb
reviewed
Jul 3, 2023
tomaarsen
reviewed
Jul 3, 2023
tomaarsen
approved these changes
Jul 3, 2023
frascuchon
force-pushed
the
tests/review-test-workflow-in-CI
branch
9 times, most recently
from
b4b2814
to
3971bb8
Compare
frascuchon
force-pushed
the
tests/review-test-workflow-in-CI
branch
from
834014d
to
5bb2dff
Compare
alvarobartt
pushed a commit
that referenced
this pull request
Jul 4, 2023
<!-- Thanks for your contribution! As part of our Community Growers initiative 🌱, we're donating Justdiggit bunds in your name to reforest sub-Saharan Africa. To claim your Community Growers certificate, please contact David Berenstein in our Slack community or fill in this form https://tally.so/r/n9XrxK once your PR has been merged. --> # Description This PR reviews and organizes the CI pipeline. With the changes introduced here, all tests are running as expected. Also, the cache action has been updated in order to fix a cache resolution problem which allows to reuse the cache for the rest of the jobs. ## Two blocks of tests are launched separately **base tests** (most of the server tests and some basic client tests) and **extra tests** (training, monitor, listeners...). The pipeline falls into errors if resource all tests are launched in the same job. In the next step, those groups should be reviewed and improved. Refs #3291 ## Reusable jobs extraction The job running the tests has been extracted as a separate workflow, which helps with reusability and workflow code maintenance. The same has been applied to the build Python package. ## Improve conditional executions The `IS_DEPLOYABLE` step conditions have been removed. Now, this condition is executed just once at the related job level.
leiyre
pushed a commit
that referenced
this pull request
Jul 4, 2023
* develop: refactor: add `HuggingFaceDatasetMixIn` under `integrations` (#3326) feat: add list user workspaces endpoint (#3308) ci: Stop linking issues to team work project chore: add missing `greenlet` dependency in `server` extra (#3330) fix: unit test failing if not local db (#3307) ci: Optimize build + test pipeline (#3300) refactor: simplify old bulk endpoints to avoid create datasets if does not exists (#3306) 📝 Update doc site link (#3299) 🚑 Fix dependencies (#3302) feat: migrate to async SQLAlchemy engine (#3162)
leiyre
pushed a commit
that referenced
this pull request
Jul 5, 2023
* develop: fix: return all workspaces in system for owner users (#3343) fix: `rg.init` with argilla user using quickstart images raise an unexpected error (#3341) feat: add `Suggestion` endpoints (#3304) feat: add `list_user_workspaces` and `User.workspaces` property (#3334) refactor: add `HuggingFaceDatasetMixIn` under `integrations` (#3326) feat: add list user workspaces endpoint (#3308) ci: Stop linking issues to team work project chore: add missing `greenlet` dependency in `server` extra (#3330) docs: update developer docs (#3314) Docs/3312 docs 112 is not building correctly (#3313) fix: unit test failing if not local db (#3307) ci: Optimize build + test pipeline (#3300) refactor: simplify old bulk endpoints to avoid create datasets if does not exists (#3306)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR reviews and organizes the CI pipeline. With the changes introduced here, all tests are running as expected.
Also, the cache action has been updated in order to fix a cache resolution problem which allows to reuse the cache for the rest of the jobs.
Two blocks of tests are launched separately
base tests (most of the server tests and some basic client tests) and extra tests (training, monitor, listeners...). The pipeline falls into errors if resource all tests are launched in the same job.
In the next step, those groups should be reviewed and improved.
Refs #3291
Reusable jobs extraction
The job running the tests has been extracted as a separate workflow, which helps with reusability and workflow code maintenance. The same has been applied to the build Python package.
Improve conditional executions
The
IS_DEPLOYABLEstep conditions have been removed. Now, this condition is executed just once at the related job level.