Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: ensure consistent Julia environments in live job tests #47

Merged
merged 8 commits into from
Feb 5, 2024

Conversation

mortenpi
Copy link
Member

@mortenpi mortenpi commented Feb 5, 2024

JuliaHub jobs have a fixed Julia version (1.9 presently), so you need to make sure you submit a Manifest.toml that is compatible with it. However, the current live tests sometimes resolve things with the running Julia version, and we run the tests with a whole range of Julia versions. With nightly Julia builds, this now causes issues (specifically, the StyledStrings standard library is not available on older Julia versions).

  • Use script""noenv for all the tests that submit simple scripts. Standard script"" attaches the currently running environment, which may declare dependencies that are not valid when running with a different Julia version. Those tests don't require any packages.
  • Turn the distributed tests into an appbundle with a pre-resolved environment.
  • Update the job1 manifest to 1.9.

@codecov-commenter
Copy link

codecov-commenter commented Feb 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (28e105c) 78.49% compared to head (dd61b3b) 40.74%.

Additional details and impacted files
@@             Coverage Diff             @@
##             main      #47       +/-   ##
===========================================
- Coverage   78.49%   40.74%   -37.75%     
===========================================
  Files          16       16               
  Lines        2241     2216       -25     
===========================================
- Hits         1759      903      -856     
- Misses        482     1313      +831     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mortenpi mortenpi marked this pull request as ready for review February 5, 2024 10:44
@mortenpi mortenpi enabled auto-merge (squash) February 5, 2024 10:44
@mortenpi mortenpi requested a review from krynju February 5, 2024 10:45
Copy link
Member

@krynju krynju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also attach only Project.toml for testing as the job code will resolve the Project.toml into a valid Manifest.toml
This should cover most cases (I think only these stdlib changes could affect it somehow)

This works too, so feel free to use the above if you think it could help

@mortenpi mortenpi merged commit 129d1c5 into main Feb 5, 2024
11 checks passed
@mortenpi mortenpi deleted the mp/job-envs branch February 5, 2024 13:54
@mortenpi
Copy link
Member Author

mortenpi commented Feb 5, 2024

We can also attach only Project.toml for testing as the job code will resolve the Project.toml into a valid Manifest.toml
This should cover most cases (I think only these stdlib changes could affect it somehow)

One of the tests specifically submits the same job twice, once with a Manifest and then without, and detects the environment difference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants