From e2d047210e8bf51e14114c91ba864919fc58e142 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tim=20Tr=C3=B6ndle?= Date: Mon, 15 Apr 2024 15:34:32 +0200 Subject: [PATCH 1/5] Update default profile to be a workflow-profile Default profile (in `profiles/default`) is now the default workflow profile. Snakemake activates it automatically using the name. This allows the user to inject other profile parameters using the `--profile` cli option. This also allows us to provide the conda configuration as an addition to, rather than a replacement of, the default profile. Fixes #268 --- CHANGELOG.md | 2 +- docs/workflow/build-remote.md | 4 ++-- docs/workflow/customisation.md | 10 ++-------- docs/workflow/overview.md | 2 +- docs/workflow/troubleshooting.md | 2 +- environment.yaml | 2 -- profiles/conda/config.yaml | 5 ++--- 7 files changed, 9 insertions(+), 18 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 597a5936..8081102c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -42,7 +42,7 @@ * **UPDATE** cluster sync infrastructure to retain file permission defaults on the cluster. This change improves team collaboration, as default group settings will apply to the files on the cluster (#214). * **UPDATE** the declaration of required cluster resources. Moving away from a mechanism that is deprecated in Snakemake (#211). -* **UPDATE** default Snakemake profile to be activated automatically, for convenience (#264). +* **UPDATE** default Snakemake profile to be activated automatically, for convenience (#264, #268). * **UPDATE** default conda prefix directory including consistent handling of the path to eurocalliopelib (#264, #331). * **UPDATE** snakemake to v8.10.6 (#330), which ensures that conda environment builds ignore default package specifications (#289). diff --git a/docs/workflow/build-remote.md b/docs/workflow/build-remote.md index 043f91eb..0eac4133 100644 --- a/docs/workflow/build-remote.md +++ b/docs/workflow/build-remote.md @@ -1,7 +1,7 @@ # Build on a cluster You may want to build the model on a cluster. -While you can build Euro-Calliope on [any cluster that is supported by Snakemake](https://snakemake.readthedocs.io/en/v6.1.1/executing/cluster.html), our default configuration is targeted at, and tested on, ETH's Euler cluster. +While you can build Euro-Calliope on [any cluster that is supported by Snakemake](https://snakemake.readthedocs.io/en/v8.10.7/tutorial/additional_features.html#cluster-or-cloud-execution), our default configuration is targeted at, and tested on, ETH's Euler cluster. ## Build @@ -11,7 +11,7 @@ To build the model on Euler, use the following command: snakemake --profile profiles/euler ``` -If you want to run on another cluster, read [snakemake's documentation on cluster execution](https://snakemake.readthedocs.io/en/stable/executing/cluster.html) and take `profiles/euler` as a starting point. +If you want to run on another cluster, read [snakemake's documentation on cluster execution](https://snakemake.readthedocs.io/en/v8.10.7/tutorial/additional_features.html#cluster-or-cloud-execution) and take `profiles/euler` as a starting point. ## Work local, build on remote diff --git a/docs/workflow/customisation.md b/docs/workflow/customisation.md index 18fa8515..f2326482 100644 --- a/docs/workflow/customisation.md +++ b/docs/workflow/customisation.md @@ -13,19 +13,13 @@ You can, for example, change the temporal and spatial scope of the data and mode The configuration builds on Snakemake's configuration mechanism and consists of two parts: a default configuration `./config/default.yaml` and a schema declaring all configuration parameters `./config/schema.yaml`. To override configuration parameters, you can add another configuration file with just your updates or change parameter values on the command line when calling `snakemake`. -For details on how the configuration mechanism works, please read [Snakemake's documention](https://snakemake.readthedocs.io/en/v6.1.1/snakefiles/configuration.html). - -!!! warning - When you change configuration parameters, please consider this important caveat of Snakemake: - While Snakemake does understand that parameters have changed, it will not rerun rules with updated parameters by default (see also this [feature request](https://github.com/snakemake/snakemake/issues/976)). - To ensure that your configuration parameters are used, you must manually rerun all rules that use the updated parameter. - The easiest is to start the build process from scratch by running `snakemake clean`. +For details on how the configuration mechanism works, please read [Snakemake's documention](https://snakemake.readthedocs.io/en/v8.10.7/snakefiles/configuration.html). ## Adaptation Beyond configuration through parameters, you can adapt and extend the workflow in any possible way. You can adapt the data pre-processing steps and the way model files are generated, but you can also extend the model by adding your own model files or overrides. -Customising Euro-Calliope in this way requires a solid understanding of the workflow management system [Snakemake](https://snakemake.readthedocs.io/en/v6.1.1/index.html) that we use. +Customising Euro-Calliope in this way requires a solid understanding of the workflow management system [Snakemake](https://snakemake.readthedocs.io/en/v8.10.7/index.html) that we use. Whenever we applied Euro-Calliope in our research we made use of this option. Below you will find a list of publications in which we applied Euro-Calliope models. diff --git a/docs/workflow/overview.md b/docs/workflow/overview.md index ce06f184..d642c7ee 100644 --- a/docs/workflow/overview.md +++ b/docs/workflow/overview.md @@ -2,7 +2,7 @@ Euro-Calliope's workflow builds the models from raw data. In essence, it is composed out of three things: scripts that retrieve raw data, process data, and formulate the model; execution environments that explicitly define software dependencies and requirements (equivalent to the conditions in a lab); and the glue ensuring all scripts run in the correct execution environment and the right order. -Scripts are written in Python; the execution environments are [conda environments](https://docs.conda.io/en/latest/); and the glue holding everything together is the workflow management system [Snakemake](https://snakemake.readthedocs.io/en/v6.1.1/). +Scripts are written in Python; the execution environments are [conda environments](https://docs.conda.io/en/latest/); and the glue holding everything together is the workflow management system [Snakemake](https://snakemake.readthedocs.io/en/v8.10.7/). ## Rationale diff --git a/docs/workflow/troubleshooting.md b/docs/workflow/troubleshooting.md index b1e04813..fce1e232 100644 --- a/docs/workflow/troubleshooting.md +++ b/docs/workflow/troubleshooting.md @@ -7,7 +7,7 @@ running into these problems is completely normal and in many cases a solution is Here are some steps we advise you to go through: 1. Read through our frequently asked questions below. -2. If you did not find an answer to your question, check [Snakemake's documentation](https://snakemake.readthedocs.io/en/v6.1.1/). +2. If you did not find an answer to your question, check [Snakemake's documentation](https://snakemake.readthedocs.io/en/v8.10.7/). 3. If you still did not find an answer to your question, [open an issue](https://github.com/calliope-project/euro-calliope/issues/new/choose) on our issue tracker. ## Frequently Asked Questions diff --git a/environment.yaml b/environment.yaml index 2e1a5c31..3a482dae 100644 --- a/environment.yaml +++ b/environment.yaml @@ -6,5 +6,3 @@ dependencies: - python=3.11 - pycountry=18.12.8 - snakemake-minimal=8.10.7 -variables: - SNAKEMAKE_PROFILE: ./profiles/default diff --git a/profiles/conda/config.yaml b/profiles/conda/config.yaml index a7b8c064..3ecddb30 100644 --- a/profiles/conda/config.yaml +++ b/profiles/conda/config.yaml @@ -1,4 +1,3 @@ -software-deployment-method: conda +# These parameter are considered additional to ./profiles/default/config.yaml. +# Apply them using the `--profile="profiles/conda` cli option. conda-frontend: conda -cores: 2 -conda-prefix: ../envs/snakemake/euro-calliope/ From a9e68b0570c0ec8946612e597ce25be3e51acf13 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tim=20Tr=C3=B6ndle?= Date: Tue, 16 Apr 2024 13:00:58 +0200 Subject: [PATCH 2/5] Add workflow-profile to the documentation --- docs/workflow/build.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/workflow/build.md b/docs/workflow/build.md index 17280d8b..bdccbd30 100644 --- a/docs/workflow/build.md +++ b/docs/workflow/build.md @@ -42,6 +42,8 @@ Execute the workflow like so: snakemake ``` +This will use Snakemake run parameters from the default `workflow-profile` in `profiles/default`. You may change these parameters and/or use a user defined profile using `--profile=`; see [Snakemake's documentation on profiles](https://snakemake.readthedocs.io/en/stable/executing/cli.html#profiles). + ## Run The build step creates all individual components of Euro-Calliope, like technologies and time series. From 226b6a91c203e0eec4d22a7716948b7fc3087395 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tim=20Tr=C3=B6ndle?= Date: Tue, 16 Apr 2024 13:02:15 +0200 Subject: [PATCH 3/5] Fix consistency of Snakemake and Calliope versions in repo --- CHANGELOG.md | 2 +- CONTRIBUTING.md | 19 ++++++++++++------- Snakefile | 2 +- docs/model/customisation.md | 10 +++++----- docs/model/pre-built.md | 2 +- docs/workflow/build-remote.md | 4 ++-- docs/workflow/build.md | 2 +- docs/workflow/customisation.md | 4 ++-- docs/workflow/overview.md | 2 +- docs/workflow/troubleshooting.md | 2 +- lib/eurocalliopelib/docs/dag.py | 7 +++---- mkdocs.yaml | 5 ++++- requirements-docs.yaml | 5 +++-- 13 files changed, 37 insertions(+), 29 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8081102c..2b4a0c7c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -44,7 +44,7 @@ * **UPDATE** the declaration of required cluster resources. Moving away from a mechanism that is deprecated in Snakemake (#211). * **UPDATE** default Snakemake profile to be activated automatically, for convenience (#264, #268). * **UPDATE** default conda prefix directory including consistent handling of the path to eurocalliopelib (#264, #331). -* **UPDATE** snakemake to v8.10.6 (#330), which ensures that conda environment builds ignore default package specifications (#289). +* **UPDATE** snakemake to v8.10.7 (#330), which ensures that conda environment builds ignore default package specifications (#289). ### Fixed (models) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ccecdb51..35d3b69b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -118,13 +118,18 @@ Be aware that you can publish a release only if you are a maintainer of the GitH * [./lib/eurocalliopelib/__init__.py](./lib/eurocalliopelib/__init__.py) * [./templates/README.md](./templates/README.md) * [./docs/index.md](./docs/index.md) - 2. Verify consistent versions of Snakemake and Calliope. - Their versions are spread over the entire repository (including urls in the documentation). - Use search and replace to make sure all versions are consistent. - 3. Update the changelog and add the release date. - 4. Update the release date in [./CITATION.cff](./CITATION.cff). - 5. (If necessary) Update [./LICENSE.md](./LICENSE.md). - 6. (If necessary) Update `docs/img/spatial-scope-and-resolutions.png` by running `snakemake -s rules/doc.smk docs/img/spatial-scope-and-resolutions.png`. Inspect the result visually. Check it in if it changed; check out the old version if it did not change. The figure will change when the spatial scope or resolution has changed. + 2. Verify consistent versions of Snakemake. + * [./environment.yaml](./environment.yaml) + * [./requirements-docs.yaml](./requirements-docs.yaml) + 3. Verify consistent versions of Calliope. + * [./envs/test.yaml](./envs/test.yaml) + * [./templates/environment.yaml](./templates/environment.yaml) + * [./templates/models/example-model.yaml](./templates/models/example-model.yaml) + * [./mkdocs.yaml](./mkdocs.yaml) + 4. Update the changelog and add the release date. + 5. Update the release date in [./CITATION.cff](./CITATION.cff). + 6. (If necessary) Update [./LICENSE.md](./LICENSE.md). + 7. (If necessary) Update `docs/img/spatial-scope-and-resolutions.png` by running `snakemake -s rules/doc.smk docs/img/spatial-scope-and-resolutions.png`. Inspect the result visually. Check it in if it changed; check out the old version if it did not change. The figure will change when the spatial scope or resolution has changed. 2. Build the pre-builts and test everything using the `all_tests` rule. Make sure you start with a clean workflow folder: delete `./build` *and* `./data/automatic` should they exist. 3. Commit, open a pull request onto `develop`, and merge the release branch into both `develop` and `main` after successful review. 4. Add a `vX.Y.Z` release tag to `main`, push it, and add a release on GitHub. diff --git a/Snakefile b/Snakefile index af442bf6..f1f7e983 100644 --- a/Snakefile +++ b/Snakefile @@ -25,7 +25,7 @@ include: "./rules/nuclear.smk" include: "./rules/transport.smk" include: "./rules/sync.smk" include: "./rules/heat.smk" -min_version("7.8") +min_version("8.10") localrules: all, clean wildcard_constraints: resolution = "continental|national|regional" diff --git a/docs/model/customisation.md b/docs/model/customisation.md index c8a4243d..a863ec79 100644 --- a/docs/model/customisation.md +++ b/docs/model/customisation.md @@ -15,13 +15,13 @@ You have the following three options: With the Calliope model in your hands, you will be able to change any model parameter, any technology specifics, and the model definition to your liking. This kind of customisation can be useful to get to know the model and its parameters. To create reliable results, we advise making manual changes only to the model definition (`example-model.yaml`) as this makes it possible to trace those changes later. -A typical customisation here would be to change the solver from `gurobi` to an open-source solver, e.g. `cbc` (see [Calliope's documentation](https://calliope.readthedocs.io/en/v0.6.10/user/config_defaults.html#run-configuration)). +A typical customisation here would be to change the solver from `gurobi` to an open-source solver, e.g. `cbc` (see [Calliope's documentation](https://calliope.readthedocs.io/en/v{{ calliope_version }}/user/config_defaults.html#run-configuration)). We consider all Euro-Calliope model subcomponents (everything other than the model definition itself) as a toolbox from which you can choose to define your model -- see the [Import customisation option](./customisation.md#importing-modules). ## Importing modules The `example-model.yaml` definition file in each resolution sub-directory (e.g. `national/example-model.yaml`) specifies a list of other files to bring together to describe the model (under the `import` key). -This list can be changed by the modeller to select a combination of different files (see also [Calliope's documentation](https://calliope.readthedocs.io/en/v0.6.10/user/building.html#files-that-define-a-model)). +This list can be changed by the modeller to select a combination of different files (see also [Calliope's documentation](https://calliope.readthedocs.io/en/v{{ calliope_version }}/user/building.html#files-that-define-a-model)). These files represent "modules" of the model definition and contain everything necessary for a given technology or technology group to exist. For instance, `techs/supply/hydro.yaml` defines two technologies (under the `techs` key) which will convert river flows into electricity. It also places that technology in every relevant modelled location (under the `locations` key), along with any location-specific information that is needed; in this case, the maximum capacity of hydropower in that location. @@ -130,7 +130,7 @@ Here, we describe each module in terms of the technologies they contain (`callio This supply has high variable cost (see `tech-cost.yaml` parameter file) and no fixed cost. Due to its high cost, it will only be used when no other, less costly, option is available. - Calliope provides a built-in mechanism that is similar: [`ensure-feasibility`](https://calliope.readthedocs.io/en/v0.6.10/user/building.html#allowing-for-unmet-demand). + Calliope provides a built-in mechanism that is similar: [`ensure-feasibility`](https://calliope.readthedocs.io/en/v{{ calliope_version }}/user/building.html#allowing-for-unmet-demand). The benefit of using the `load-shedding` override over Calliope's built-in mechanism is that it is more targeted towards modelling shedding of electrical load and provides more flexibility -- for example in terms of the cost of shed load. @@ -204,7 +204,7 @@ Here, we describe each module in terms of the technologies they contain (`callio ## Overrides and scenarios -Calliope [overrides](https://calliope.readthedocs.io/en/v0.6.10/user/building.html#scenarios-and-overrides) enable models to be easily manipulated. +Calliope [overrides](https://calliope.readthedocs.io/en/v{{ calliope_version }}/user/building.html#scenarios-and-overrides) enable models to be easily manipulated. An override named `freeze-hydro-supply-capacities` can be used for example in this way: ``` bash @@ -223,7 +223,7 @@ For instance, `freeze-hydro-supply-capacities` and `freeze-hydro-storage-capacit You can also define your own overrides to manipulate any model component. We recommend you add these overrides into the model definition YAML file, to ensure they are easy to trace. -In Calliope, [scenarios](https://calliope.readthedocs.io/en/v0.6.7/user/building.html#scenarios-and-overrides) are groups of overrides and/or other scenarios. +In Calliope, [scenarios](https://calliope.readthedocs.io/en/v{{ calliope_version }}/user/building.html#scenarios-and-overrides) are groups of overrides and/or other scenarios. In Euro-Calliope, it can be helpful to define scenarios to help group similar overrides together. For instance, cost overrides from the Danish Energy Agency are defined in various files, since they are loaded in alongside the technologies they affect (the option to override offshore wind costs only exists when you load the `techs/supply/wind-offshore.yaml` module). You can pre-define scenarios in your model definition file, such as: diff --git a/docs/model/pre-built.md b/docs/model/pre-built.md index cbe54a36..e8754db2 100644 --- a/docs/model/pre-built.md +++ b/docs/model/pre-built.md @@ -3,7 +3,7 @@ If you want an easy entry into using and exploring Euro-Calliope, our pre-built models provide the jump start you need. The pre-built models are ordinary Calliope models that you can use like any other. If you are unfamiliar with Calliope, we'll show you below how to run the model. -After going through these first steps, we advise you to head over to [Calliope's documentation](https://calliope.readthedocs.io/en/v0.6.10/) to learn its basic usage. +After going through these first steps, we advise you to head over to [Calliope's documentation](https://calliope.readthedocs.io/en/v{{ calliope_version }}/) to learn its basic usage. ## Prepare diff --git a/docs/workflow/build-remote.md b/docs/workflow/build-remote.md index 0eac4133..f05a8475 100644 --- a/docs/workflow/build-remote.md +++ b/docs/workflow/build-remote.md @@ -1,7 +1,7 @@ # Build on a cluster You may want to build the model on a cluster. -While you can build Euro-Calliope on [any cluster that is supported by Snakemake](https://snakemake.readthedocs.io/en/v8.10.7/tutorial/additional_features.html#cluster-or-cloud-execution), our default configuration is targeted at, and tested on, ETH's Euler cluster. +While you can build Euro-Calliope on [any cluster that is supported by Snakemake](https://snakemake.readthedocs.io/en/v{{ snakemake_version }}/tutorial/additional_features.html#cluster-or-cloud-execution), our default configuration is targeted at, and tested on, ETH's Euler cluster. ## Build @@ -11,7 +11,7 @@ To build the model on Euler, use the following command: snakemake --profile profiles/euler ``` -If you want to run on another cluster, read [snakemake's documentation on cluster execution](https://snakemake.readthedocs.io/en/v8.10.7/tutorial/additional_features.html#cluster-or-cloud-execution) and take `profiles/euler` as a starting point. +If you want to run on another cluster, read [snakemake's documentation on cluster execution](https://snakemake.readthedocs.io/en/v{{ snakemake_version }}/tutorial/additional_features.html#cluster-or-cloud-execution) and take `profiles/euler` as a starting point. ## Work local, build on remote diff --git a/docs/workflow/build.md b/docs/workflow/build.md index bdccbd30..374ef728 100644 --- a/docs/workflow/build.md +++ b/docs/workflow/build.md @@ -55,7 +55,7 @@ It is a complete Calliope model and can be used like any other, for example like calliope run ./build/models/national/example-model.yaml ``` -For more information on how to use Calliope models, see [Calliope's documentation](https://calliope.readthedocs.io/en/v0.6.10/). +For more information on how to use Calliope models, see [Calliope's documentation](https://calliope.readthedocs.io/en/v{{ calliope_version }}/). ## Customise diff --git a/docs/workflow/customisation.md b/docs/workflow/customisation.md index f2326482..c3819e0c 100644 --- a/docs/workflow/customisation.md +++ b/docs/workflow/customisation.md @@ -13,13 +13,13 @@ You can, for example, change the temporal and spatial scope of the data and mode The configuration builds on Snakemake's configuration mechanism and consists of two parts: a default configuration `./config/default.yaml` and a schema declaring all configuration parameters `./config/schema.yaml`. To override configuration parameters, you can add another configuration file with just your updates or change parameter values on the command line when calling `snakemake`. -For details on how the configuration mechanism works, please read [Snakemake's documention](https://snakemake.readthedocs.io/en/v8.10.7/snakefiles/configuration.html). +For details on how the configuration mechanism works, please read [Snakemake's documention](https://snakemake.readthedocs.io/en/v{{ snakemake_version }}/snakefiles/configuration.html). ## Adaptation Beyond configuration through parameters, you can adapt and extend the workflow in any possible way. You can adapt the data pre-processing steps and the way model files are generated, but you can also extend the model by adding your own model files or overrides. -Customising Euro-Calliope in this way requires a solid understanding of the workflow management system [Snakemake](https://snakemake.readthedocs.io/en/v8.10.7/index.html) that we use. +Customising Euro-Calliope in this way requires a solid understanding of the workflow management system [Snakemake](https://snakemake.readthedocs.io/en/v{{ snakemake_version }}/index.html) that we use. Whenever we applied Euro-Calliope in our research we made use of this option. Below you will find a list of publications in which we applied Euro-Calliope models. diff --git a/docs/workflow/overview.md b/docs/workflow/overview.md index d642c7ee..366f91ee 100644 --- a/docs/workflow/overview.md +++ b/docs/workflow/overview.md @@ -2,7 +2,7 @@ Euro-Calliope's workflow builds the models from raw data. In essence, it is composed out of three things: scripts that retrieve raw data, process data, and formulate the model; execution environments that explicitly define software dependencies and requirements (equivalent to the conditions in a lab); and the glue ensuring all scripts run in the correct execution environment and the right order. -Scripts are written in Python; the execution environments are [conda environments](https://docs.conda.io/en/latest/); and the glue holding everything together is the workflow management system [Snakemake](https://snakemake.readthedocs.io/en/v8.10.7/). +Scripts are written in Python; the execution environments are [conda environments](https://docs.conda.io/en/latest/); and the glue holding everything together is the workflow management system [Snakemake](https://snakemake.readthedocs.io/en/v{{ snakemake_version }}/). ## Rationale diff --git a/docs/workflow/troubleshooting.md b/docs/workflow/troubleshooting.md index fce1e232..dbfe7c35 100644 --- a/docs/workflow/troubleshooting.md +++ b/docs/workflow/troubleshooting.md @@ -7,7 +7,7 @@ running into these problems is completely normal and in many cases a solution is Here are some steps we advise you to go through: 1. Read through our frequently asked questions below. -2. If you did not find an answer to your question, check [Snakemake's documentation](https://snakemake.readthedocs.io/en/v8.10.7/). +2. If you did not find an answer to your question, check [Snakemake's documentation](https://snakemake.readthedocs.io/en/v{{ snakemake_version }}/). 3. If you still did not find an answer to your question, [open an issue](https://github.com/calliope-project/euro-calliope/issues/new/choose) on our issue tracker. ## Frequently Asked Questions diff --git a/lib/eurocalliopelib/docs/dag.py b/lib/eurocalliopelib/docs/dag.py index 5ce3ccb3..9e718feb 100644 --- a/lib/eurocalliopelib/docs/dag.py +++ b/lib/eurocalliopelib/docs/dag.py @@ -4,21 +4,19 @@ import mkdocs import pydot -import snakemake from mkdocs.plugins import BasePlugin from mkdocs.structure.files import File +from snakemake import cli class DAGPlugin(BasePlugin): config_scheme = ( - ("path_to_snakefile", mkdocs.config.config_options.Type(str)), ("path_to_src_dir", mkdocs.config.config_options.Type(str)), ("path_to_png_relative_to_site", mkdocs.config.config_options.Type(str)), ) def on_files(self, files, config, **kwargs): """Generate DAG as png and add it to mkdocs files.""" - path_to_snakemake = Path.cwd() / self.config["path_to_snakefile"] path_to_png = ( Path.cwd() / self.config["path_to_src_dir"] @@ -27,7 +25,8 @@ def on_files(self, files, config, **kwargs): path_to_png.parent.mkdir(parents=True, exist_ok=True) graph_string = io.StringIO() with redirect_stdout(graph_string): - snakemake.snakemake(path_to_snakemake, printrulegraph=True) + parser, args = cli.parse_args("--rulegraph") + cli.args_to_api(args, parser) graph = pydot.graph_from_dot_data(graph_string.getvalue())[0] graph.write_png(path_to_png) diff --git a/mkdocs.yaml b/mkdocs.yaml index b8556f7b..dfa37088 100644 --- a/mkdocs.yaml +++ b/mkdocs.yaml @@ -30,8 +30,8 @@ markdown_extensions: permalink: "#" plugins: - search + - macros - dag: - path_to_snakefile: ./Snakefile path_to_src_dir: ./build/.docs path_to_png_relative_to_site: ./img/dag.png - schema: @@ -43,3 +43,6 @@ plugins: path_to_src_dir: . extra_css: [./css/extras.css] +extra: + snakemake_version: !!python/name:snakemake.__version__ + calliope_version: 0.6.10 diff --git a/requirements-docs.yaml b/requirements-docs.yaml index 3d541a69..12d1b7bb 100644 --- a/requirements-docs.yaml +++ b/requirements-docs.yaml @@ -3,14 +3,15 @@ channels: - conda-forge - bioconda dependencies: - - Python=3.8 + - Python=3.11 - mkdocs=1.2.2 + - mkdocs-macros-plugin=1.0.5 - jinja2=3.0.3 - graphviz=2.48.0 - pydot=1.4.2 - pycountry=18.12.8 - pymdown-extensions=9.9.2 - - snakemake-minimal=7.26.0 + - snakemake-minimal=8.10.7 - pip=21.0.1 - pip: - jsonschema2md==0.2.1 From 136959798177b06260d471687623b5f0a0a80bfd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tim=20Tr=C3=B6ndle?= Date: Thu, 2 May 2024 14:48:29 +0200 Subject: [PATCH 4/5] Fix bug introduced in adc2682 (#330) --- profiles/default/config.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/profiles/default/config.yaml b/profiles/default/config.yaml index d2d42461..32709ba5 100644 --- a/profiles/default/config.yaml +++ b/profiles/default/config.yaml @@ -1,3 +1,4 @@ software-deployment-method: conda +conda-frontend: mamba cores: 2 conda-prefix: ../envs/snakemake/euro-calliope/ From 8fc601f5d01c67c8edfd2e299f41b0211e712f9d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Tim=20Tr=C3=B6ndle?= Date: Thu, 2 May 2024 14:49:49 +0200 Subject: [PATCH 5/5] Add updates from review --- CONTRIBUTING.md | 1 + profiles/euler/config.yaml | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 35d3b69b..1ec07b8b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -121,6 +121,7 @@ Be aware that you can publish a release only if you are a maintainer of the GitH 2. Verify consistent versions of Snakemake. * [./environment.yaml](./environment.yaml) * [./requirements-docs.yaml](./requirements-docs.yaml) + * [./Snakefile](./Snakefile) 3. Verify consistent versions of Calliope. * [./envs/test.yaml](./envs/test.yaml) * [./templates/environment.yaml](./templates/environment.yaml) diff --git a/profiles/euler/config.yaml b/profiles/euler/config.yaml index fca98c3d..da6dc846 100644 --- a/profiles/euler/config.yaml +++ b/profiles/euler/config.yaml @@ -1,10 +1,10 @@ +# These parameter are considered additional to ./profiles/default/config.yaml. +# Apply them using the `--profile="profiles/euler` cli option. cluster: bsub -oo build/logs/{rule}.{wildcards}.log -W {resources.runtime} -n {resources.cores} -R "rusage[mem={resources.memory}]" -J {rule}.{wildcards} < jobs: 999 local-cores: 1 latency-wait: 60 use-envmodules: True -software-deployment-method: conda -conda-prefix: ../envs/snakemake/euro-calliope/ default-resources: [ "runtime=10", "cores=1",