Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update default profile to be a workflow-profile #361

Merged

Conversation

timtroendle
Copy link
Member

@timtroendle timtroendle commented Apr 15, 2024

Default profile (in profiles/default) is now the default workflow profile. Snakemake activates it automatically using the name. This allows the user to inject other profile parameters using the --profile cli option. This also allows us to provide the conda configuration as an addition to, rather than a replacement of, the default profile.

Fixes #268

Checklist

Any checks which are not relevant to the PR can be pre-checked by the PR creator. All others should be checked by the reviewer. You can add extra checklist items here if required by the PR.

  • CHANGELOG updated
  • Minimal workflow tests pass
  • Tests added to cover contribution
  • Documentation updated
  • Configuration schema updated

Default profile (in `profiles/default`) is now the default workflow profile. Snakemake activates it automatically using the name. This allows the user to inject other profile parameters using the `--profile` cli option. This also allows us to provide the conda configuration as an addition to, rather than a replacement of, the default profile.

Fixes calliope-project#268
Copy link
Member

@sjpfenninger sjpfenninger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs actually don't mention that there is a default workflow profile. This should probably be added, e.g. under the "Build" heading in build.md. Particularly mentioning that it sets cores to 2.

docs/workflow/build-remote.md Outdated Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As of v23.10.0, conda uses libmamba as its solver by default. So the two should be equally fast. Do we still need two separate profiles? Furthermore, I think as long as conda is installed as well as mamba, conda is used anyway, irrespective of whether you used mamba to install the euro-calliope environment -- unless you run snakemake with --conda-frontend mamba.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make the question whether we want to drop mamba a separate question? Right now, mamba is the default and whenever you want to run the workflow without, you have to use --profile="profiles/conda" or --conda-frontend conda which is essentially the same.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, agreed, that question is beyond the scope of this PR

@brynpickering
Copy link
Member

Am I right in saying that it would now not be necessary to include --profile profiles/conda when calling snakemake?

@timtroendle
Copy link
Member Author

timtroendle commented Apr 16, 2024

No, it's still necessary. This pull request isn't changing any behaviour, but it adds the possibility to inject your own profile through --profile="your-profile".

We could get rid of this by dropping mamba support and running conda by default.

Snakefile Show resolved Hide resolved
Copy link
Member

@brynpickering brynpickering left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should euler/config.yaml also be updated? It has conda-prefix and software-deployment-method which are now duplicated in the default profile.

@timtroendle
Copy link
Member Author

Yes, that makes sense. Done.

@timtroendle timtroendle merged commit 82dccb5 into calliope-project:develop May 2, 2024
4 checks passed
@timtroendle timtroendle deleted the feature-workflow-profile branch May 2, 2024 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Snakemake workflow profile as override of local profile
3 participants