-
Notifications
You must be signed in to change notification settings - Fork 18
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #340 from irm-codebase/add-industry-module
Add industry module
- Loading branch information
Showing
16 changed files
with
635 additions
and
16 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -21,3 +21,5 @@ __pycache__ | |
# Snakemake | ||
.snakemake/ | ||
dag.pdf | ||
**/out/* | ||
**/tmp/* |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# About | ||
|
||
This module is dedicated to getting and processing energy demand for industry. | ||
|
||
Basic info of this module: | ||
|
||
- Main data sources: JRC IDEES and eurostat | ||
- Spatial resolution: national (because of JRC IDEES) | ||
- Temporal resolution: annual aggregated | ||
|
||
Some industries are specified as 'sub-module' in this module, such as iron and steel industry. This allows you to flexibly configure the decarbonisation level of each industry separately. All other industries are grouped into 'other industries'. | ||
|
||
For workflow users, i.e. non-developers, `config.yaml` is the main file to look into. | ||
Here, one can specify parameters such as the years to be included in the data processing, assumptions such as the share of recycled steel, and the specific industry sector to be included (such as iron and steel industry). | ||
The structure of the `config.yaml` file is: | ||
|
||
- inputs - the needed data sources from other modules in euro-calliope (for now, we assume this module depends on other modules and is not yet a stand-alone module) | ||
- outputs - the output of the whole industry module, usually files, possibly passed on to other modules in euro-calliope. | ||
- params - the parameters that affect the calculation process and result in this module. | ||
By changing the value of the parameters, each user can tailor the workflow to their own needs. | ||
- setup - anything that concerns the general data pipeline of the module. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
industry: | ||
inputs: | ||
path-energy-balances: build/data/annual-energy-balances.csv | ||
path-cat-names: config/energy-balances/energy-balance-category-names.csv | ||
path-carrier-names: config/energy-balances/energy-balance-carrier-names.csv | ||
path-jrc-industry-energy: build/data/jrc-idees/industry/processed-energy.nc | ||
path-jrc-industry-production: build/data/jrc-idees/industry/processed-production.nc | ||
outputs: | ||
placeholder-out1: | ||
placeholder-out2: | ||
params: | ||
steel: | ||
recycled-steel-share: 0.5 # % of recycled scrap steel for H-DRI |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
name: module-industry | ||
channels: | ||
- conda-forge | ||
- bioconda | ||
dependencies: | ||
- python=3.11 | ||
- ipdb=0.13.13 | ||
- numpy=1.23 | ||
- pandas=1.5.3 | ||
- pycountry=18.12.8 | ||
- snakemake-minimal=8.10.7 | ||
- netcdf4=1.6.5 | ||
- xarray=2022.9.0 | ||
- bottleneck=1.3.8 | ||
- pip=21.0.1 | ||
- pip: | ||
- styleframe==4.2 | ||
- -e ./lib |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
# Paths dependent on main Snakefile | ||
MODULE_PATH = "modules/industry" | ||
BUILD_PATH = f"{MODULE_PATH}/build" | ||
DATA_PATH = f"{MODULE_PATH}/raw_data" | ||
|
||
# Paths relative to this snakefile (snakemake behaviour is inconsitent) | ||
SCRIPT_PATH = "scripts" # scripts are called relative to this file | ||
CONDA_PATH = "./env_industry.yaml" | ||
|
||
# Ensure rules are defined in order. | ||
# Otherwise commands like "rules.rulename.output" won't work! | ||
rule steel_industry: | ||
message: "Calculate energy demand for the 'Iron and steel' sector in JRC-IDEES." | ||
conda: CONDA_PATH | ||
params: | ||
config_steel = config["params"]["steel"] | ||
input: | ||
path_energy_balances = config["inputs"]["path-energy-balances"], | ||
path_cat_names = config["inputs"]["path-cat-names"], | ||
path_carrier_names = config["inputs"]["path-carrier-names"], | ||
path_jrc_industry_energy = config["inputs"]["path-jrc-industry-energy"], | ||
path_jrc_industry_production = config["inputs"]["path-jrc-industry-production"], | ||
output: | ||
path_output = f"{BUILD_PATH}/annual_demand_steel.nc" | ||
script: f"{SCRIPT_PATH}/steel_industry.py" | ||
|
||
rule chemical_industry: | ||
message: "." | ||
conda: CONDA_PATH | ||
params: | ||
input: | ||
output: | ||
script: f"{SCRIPT_PATH}/chemicals.py" | ||
|
||
rule other_industry: | ||
message: "." | ||
conda: CONDA_PATH | ||
params: | ||
input: | ||
output: f"{BUILD_PATH}/other_industry.csv" | ||
script: f"{SCRIPT_PATH}/other_industry.py" | ||
|
||
# rule combine_and_scale: | ||
# message: "." | ||
# conda: CONDA_PATH | ||
# params: | ||
# input: | ||
# output: | ||
# script: | ||
|
||
# rule verify: | ||
# message: "." | ||
# params: | ||
# input: | ||
# output: | ||
# script: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,55 @@ | ||
$schema: https://json-schema.org/draft/2020-12/schema | ||
type: object | ||
additionalProperties: true | ||
properties: | ||
industry: | ||
description: Module subsection to ensure no name conflicts are possible between modules. | ||
type: object | ||
additionalProperties: false | ||
properties: | ||
inputs: | ||
type: object | ||
additionalProperties: false | ||
description: Inputs are paths of prerequired files. | ||
properties: | ||
path-energy-balances: | ||
type: string | ||
description: | | ||
Annual energy balance file. | ||
Columns [cat_code,carrier_code,unit,country,year,value]. | ||
path-cat-names: | ||
type: string | ||
description: | | ||
Category mapping file. | ||
Columns [cat_code,top_cat,sub_cat_contribution,sub_cat_1,sub_cat_2,jrc_idees]. | ||
path-carrier-names: | ||
type: string | ||
description: | | ||
Carrier mapping file. | ||
Columns [carrier_code,carrier_name,hh_carrier_name,com_carrier_name,ind_carrier_name,oth_carrier_name]. | ||
path-jrc-industry-energy: | ||
type: string | ||
description: | | ||
JRC processed industry energy demand .nc file. | ||
path-jrc-industry-production: | ||
type: string | ||
description: | | ||
JRC processed industrial production .nc file. | ||
outputs: | ||
type: object | ||
description: Outputs are paths for the files produced by the module. | ||
params: | ||
type: object | ||
additionalProperties: false | ||
description: Parameters allow users to configure module behaviour. | ||
properties: | ||
steel: | ||
type: object | ||
additionalProperties: false | ||
description: "Parameters specific to the 'Iron and steel' industry category." | ||
properties: | ||
recycled-steel-share: | ||
type: number | ||
description: "Share of recycled metal in the H-DRI steel process." | ||
minimum: 0 | ||
maximum: 1 |
Empty file.
Empty file.
Oops, something went wrong.