Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: templates for modules and wrappers #29

Merged
merged 8 commits into from
Aug 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,5 @@ out/
output/
tmp/
results/
report/

!modules/_example_module/**
58 changes: 57 additions & 1 deletion docs/development_guide/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,5 +17,61 @@ If you don't, We recommend following `mamba`'s [installation advice](https://git
conda activate ec_modules
```

3. You are ready to go!
3. Create your module or wrapper using one of our standard [`copier`](https://github.com/copier-org/copier) templates.

??? example "Example: using a module template"

With the `ec_modules` environment activated, type:

```shell
copier copy modules/_template/ modules/
```

You'll be prompted with some questions. After answering them, `copier` will auto-generate the module for you!

```html
🎤 What is your module's name?
wind_offshore
🎤 Please give a brief sentence describing your module.
A module to estimate offshore-wind potentials for arbitrary subregions in Europe.
🎤 We auto-generate an MIT license for you. Please provide your full name.
E. Dantès
🎤 We auto-generate an MIT license for you. Please provide the name of your institution.
Morrel Technical Institute
🎤 We auto-generate an MIT license for you. Please provide an email address.
e.dantes@morrel-ti.edu
🎤 We auto-generate an MIT license for you. What year is this?
1815
```

??? example "Example: using a wrapper template"

Similar to the module example, call `copier` with the following:

```shell
copier copy wrappers/_template/ wrappers/
```

In the case of wrappers, there are some additional answers you must provide.

```html
🎤 What is the name of the tool you are designing a wrapper for?
gregor
🎤 Please provide a valid link to the tool's official website.
https://github.com/jnnr/gregor
🎤 What is the name of the wrapper?
snip
🎤 Please give a brief sentence describing your wrapper.
Snip a raster file into a smaller raster file.
🎤 We auto-generate an MIT license for you. Please provide your full name.
G. Samsa
🎤 We auto-generate an MIT license for you. Please provide the name of your institution.
Bekannt University
🎤 We auto-generate an MIT license for you. Please provide an email address.
g.s@un.bekannt.edu
🎤 We auto-generate an MIT license for you. What year is this?
1915
```

You are ready to go!
Please look into our [code conventions](conventions.md#code-conventions) and our requirements for developing [modules](modules.md) and [wrappers](wrappers.md) for more details.
55 changes: 0 additions & 55 deletions modules/_template/README.md

This file was deleted.

14 changes: 0 additions & 14 deletions modules/_template/config/README.md

This file was deleted.

7 changes: 0 additions & 7 deletions modules/_template/config/default.yaml

This file was deleted.

32 changes: 32 additions & 0 deletions modules/_template/copier.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
module_name:
type: str
help: What is your module's name?
placeholder: transport_road
validator: >-
{% if not (module_name | regex_search('^[a-z][a-z0-9_]+$')) %}
"Must be a single word in lowercase. Letters, digits and underscores are valid characters."
{% endif %}
module_description:
type: str
help: Please give a brief sentence describing your module.
placeholder: A module to estimate energy demand of road vehicles at a subnational level.
author_name:
type: str
help: We auto-generate an MIT license for you. Please provide your full name.
placeholder: Laura Patricia Orellana
author_institution:
type: str
help: We auto-generate an MIT license for you. Please provide the name of your institution.
placeholder: TU Delft
author_email:
type: str
help: We auto-generate an MIT license for you. Please provide an email address.
placeholder: yourname@university.edu
validator: >-
{% if not (author_email | regex_search('^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$')) %}
"This email does not seem valid... did you forget to add a dot?"
{% endif %}
year:
type: str
help: We auto-generate an MIT license for you. What year is this?
placeholder: "2024"
Binary file removed modules/_template/rulegraph.png
Binary file not shown.
23 changes: 0 additions & 23 deletions modules/_template/workflow/Snakefile

This file was deleted.

4 changes: 0 additions & 4 deletions modules/_template/workflow/envs/README.md

This file was deleted.

11 changes: 0 additions & 11 deletions modules/_template/workflow/profiles/README.md

This file was deleted.

5 changes: 0 additions & 5 deletions modules/_template/workflow/resources/README.md

This file was deleted.

9 changes: 0 additions & 9 deletions modules/_template/workflow/resources/internal.yaml

This file was deleted.

3 changes: 0 additions & 3 deletions modules/_template/workflow/rules/README.md

This file was deleted.

16 changes: 0 additions & 16 deletions modules/_template/workflow/rules/example.smk

This file was deleted.

5 changes: 0 additions & 5 deletions modules/_template/workflow/schemas/README.md

This file was deleted.

26 changes: 0 additions & 26 deletions modules/_template/workflow/schemas/config.schema.yaml

This file was deleted.

5 changes: 0 additions & 5 deletions modules/_template/workflow/scripts/README.md

This file was deleted.

7 changes: 0 additions & 7 deletions modules/_template/workflow/scripts/example.py

This file was deleted.

10 changes: 10 additions & 0 deletions modules/_template/{{module_name}}/AUTHORS.md.jinja
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Authors

This is the list of contributors to the '{{module_name}}' module for copyright purposes.

- {{author_name}}, {{author_institution}} <{{author_email}}>

This does not necessarily list everyone who has contributed to the '{{module_name}}' module code or documentation.
For a full contributor list, see:

<https://github.com/calliope-project/ec_modules/commits/main/modules/{{module_name}}>
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2021, AUTHORS
Copyright (c) {{year}}, AUTHORS

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
34 changes: 34 additions & 0 deletions modules/_template/{{module_name}}/README.md.jinja
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Easy Energy Modules - {{module_name}}

{{module_description}}

## Input-Ouput

Here is a brief IO diagram of the module's operation.

```mermaid
---
title: {{module_name}}
---
flowchart LR
D1[("`**Databases**
database1
...
`")] --> |Download| M
C1[/"`**User input**
shapefile.geojson
...
`"/] --> |Resources| M(({{module_name}}))
M --> O1("
output1.csv
")
M --> O2("
output2.nc
")
```

## DAG

Here is a brief example of the module's steps.

![DAG](rulegraph.png)
5 changes: 5 additions & 0 deletions modules/_template/{{module_name}}/config/default.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Module input files by users go here.
resources:
download: True
shapefile: "resources/shapefile.geojson"
# Any other user configuration goes below
Binary file added modules/_template/{{module_name}}/rulegraph.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
22 changes: 22 additions & 0 deletions modules/_template/{{module_name}}/workflow/Snakefile.jinja
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
import yaml

from snakemake.utils import min_version, validate

# Limit the snakemake version to a modern one.
min_version("8.10")

# Load the default configuration. This will be overridden by users.
configfile: "config/default.yaml"
# Validate the configuration using the schema file.
validate(config, workflow.source_path("schemas/config.schema.yaml"))

# Load internal settings separately so users cannot modify them.
with open(workflow.source_path("resources/internal_config.yaml"), "r") as f:
internal = yaml.safe_load(f)

# Add all your includes here.
include: "rules/downloads.smk"

rule all:
message: "Generate all outputs for '{{module_name}}'."
input:
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,5 @@ channels:
- conda-forge
- nodefaults
dependencies:
- curl=8.6.0
- curl=8.9.1
- unzip=6.0
- rsync=3.2.3
Loading