SWAT-pytools

A Python wrapper for executing and calibrating the Soil and Water Assessment Tool (SWAT) in Unix/macOS systems. A module for computing the 171 hydrologic indices reported by Henriksen et al. (2006) and the Magnificent Seven indices proposed by Archfield et al. (2014) is also included here. For single- and multi-objective calibration, we use the pymoo framework.

Installation

First, you need to have a Python 3 enviroment installed. We recommend using miniconda or anaconda. Check if conda is available in the command line:

conda --version

Create a new Python environment with Numpy, Pandas, and Matplotlib preinstalled and activate it:

conda create -n swatpy -y python==3.7 numpy pandas matplotlib
conda activate swatpy

If you are familiar with conda, you might prefer to use an existing environment that you already created.

Now, you can install the current SWAT-pytools version. cd to the directory where you normally save your repositories and execute these lines in a terminal:

git clone https://github.com/jshernandezs/swat-pytools
cd swat-pytools
pip install .

Usage

Once you download this package, you will find the following local directory tree:

swat-pytools
├── resources
│   ├── csv_files
│   ├── figFiles
│   ├── Models
│   └── Observed
├── src
│   └── swat_utilities
└── tests

In the resources directory, you must place all the necessary files for executing and calibrating SWAT, such as the model executables, zip files containing input SWAT text files (resources/Models directory), csv files containing observed time series (resources/Observed directory), and other optional files. Regarding SWAT executables, revisions 622, 670, and 681 are available for Unix in this repository, only version 622 is available for macOS.

Executing SWAT with Python

In this example, we are using the SWAT 622 version to run a model of the Honeyoey Creek - Pine Creek Watershed located in Michigan, US. The SWAT input text files, which normally are generated inside the TxtInOut folder when using ArcSWAT, are put together into a zip file that will be handled by the Python wrapper. In this case, we are using the Honeyoy_Model.zip file placed into the resources/Models directory.

We assume that a new Python script is created in the tests directory.

First, we import the libraries that we are going to use:

import os
import subprocess as sp
from swat_utilities.swat_config import ModelSetup

Then, we define a variable with the path to the zip file containing the SWAT text files:

model_file_path = os.path.abspath('../resources/Models/Honeyoy_Model.zip')

Now, we create the model object using the ModelSetup class:

swat_model = ModelSetup(model_file_path)

We must indicate which SWAT version we are going to use, which is the property swat_exec_name of the model object created above ('SWAT_Rev670' is the default), and where the executable file is located with the property swat_dir (by default the directory is '../resources'):

swat_model.swat_exec_name = 'SWAT_Rev622'
swat_model.swat_dir = os.path.abspath('../resources')

To see the execution progress when running the model, we set the property 'verbose_swat' as True:

swat_model.verbose_swat = True

To run the model, we need to execute the prepare_swat() method followed by the run_swat() method as follows:

swat_model.prepare_swat()
swat_model.run_swat()

prepare_swat() unzips the model text files, copy them with the SWAT executable to a temporal directory which can be changed with the temp_dir property of the model object (by default the directory is '/tmp/swat_runs'), and performs any needed modifications to the SWAT text files.

By default, the results are stored in the /tmp/output_swat/New_SWAT directory. The user can modify the output directory (i.e., '/tmp/output_swat') and the model folder containing the input and output SWAT text files (i.e., 'New_SWAT') by using the output_dir and new_model_name properties of the swat_model object, respectively.

Before running a new SWAT model, be sure that both temp_dir and output_dir do not contain a folder with the same name of your new SWAT model. For this reason, we recommend to run the remove_swat() at the end of any post-processing routine using the SWAT output.

For example, in the following lines, we create a new folder that will contain the output.rch and output.sub files generated by SWAT. Then, we remove the SWAT temporal and output directories generated during the model execution:

output_dir = '../resources/swat_output/Honeyoey_Model'
if not os.path.exists(output_dir):
    os.makedirs(output_dir)

sp.run(['mv', '/tmp/output_swat/New_SWAT/output.rch', os.path.abspath(output_dir)], check=True)
sp.run(['mv', '/tmp/output_swat/New_SWAT/output.sub', os.path.abspath(output_dir)], check=True)
swat_model.remove_swat()

Modifying SWAT model parameters

The user can modify the parameters of the model for: 1) the entire watershed, 2) specific subbasins, or 3) specific subbasins and hrus. There are four options to modifying a parameter value:

1. Replace the existing value by another value ('replace' option)
2. Multiply the existing value by 1 + a fraction ('multiply' option)
3. Multiply the existing value by a factor ('factor' option)
4. Add a constant to the existing value ('add' option)

The user must define a dictionary where the keys are the parameter names to change using the same nomenclature as the SWAT 2012 input/output documentation, and the values are three-element lists: the first element of the list is a numeric value, the second is the option for modifying the existing parameter value, and the third element is the file extension of the SWAT text file where the parameter is located. See an example below:

params = {
          'BIOMIX': [0.22, 'replace', 'mgt'],
          'CN2': [-0.21, 'multiply', 'mgt'],
          'CANMX': [1.67, 'replace', 'hru'],
          'ESCO': [0.70, 'replace', 'hru'],
          'EPCO': [0.0059, 'replace', 'hru'],
          'GW_DELAY': [6.11, 'replace', 'gw'],
          'ALPHA_BF': [0.83, 'replace', 'gw'],
          'GWQMN': [438, 'replace', 'gw'],
          'GW_REVAP': [0.16, 'replace', 'gw'],
          'REVAPMN': [438, 'replace', 'gw'],
          'RCHRG_DP': [0.50, 'replace', 'gw'],
          'CH_N2': [0.12, 'replace', 'rte'],
          'CH_K2': [6.45, 'replace', 'rte'],
          'SOL_AWC': [-0.21, 'multiply', 'sol'],
          'SURLAG': [1.10, 'replace', 'bsn']
          }

In this example, we want to replace the existing value of 'BIOMIX' by 0.22. This parameter can be found at any text file with extension 'mgt'. Meanwhile, we want to decrease 'CN2' by 21% (existing 'CN2' values will by multiplied by 0.79 = 1 + (-0.21)).

Note: In the current version of SWAT-pytools, databases (files .dat) are not supported yet. Therefore, biophysical parameters controlling evapotranspiration included in plant.dat cannot be calibrated using this package.

1) Entire Watershed

To modify the parameters at once for the entire watershed, the dictionary must be passed to the param property of the model object and then prepare and execute the model as follows:

swat_model.param = params
swat_model.prepare_swat()
swat_model.run_swat()

2) Specific subbasins

If the user wants to apply different parameter modifications to different sets of subbasins, a list of lists must be passed to the subbasins property of the model object. Each list contains the identifier numbers of the subbasins in which the parameters will be changed. Then, a list of dictionaries must be passed to the param property. This list must have the same size as the subbasins list. The first dictionary in param corresponds to the first list in subbasins and so forth. See the example below:

# we define two parameter dictionaries to be applied to two different sets of subbasins:

params1 = {          
          'CN2': [0.10, 'multiply', 'mgt'],
          'ESCO': [0.70, 'replace', 'hru']
          }
params2 = {          
          'CN2': [-0.21, 'multiply', 'mgt'],
          'ESCO': [0.90, 'replace', 'hru'],
          }
swat_model.params = [params1, params2]

# we define the sets of subbasins to be modified

subbasins1 = [1, 2, 3, 4, 5]
subbasins2 = [10, 11, 12, 13, 14]
swat_model.subbasins = [subbasins1, subbasins2]

swat_model.prepare_swat()
swat_model.run_swat()

Those subbasins not included in any set will not be modified at all.

3) Specific subbasins and hrus

In this case, an additional list of lists must be passed to the hrus property of the model object. Each list is also a list of lists with the hrus associated to each subbasin within the corresponding set of subasins. Still in this case, the param list must be as long as the subbasins list. See the example below (Note: this example cannot be run on the Honeyoey model since it only has 1 hru for each subbasin):

params1 = {          
          'CN2': [0.10, 'multiply', 'mgt'],
          'ESCO': [0.70, 'replace', 'hru']
          }
params2 = {          
          'CN2': [-0.21, 'multiply', 'mgt'],
          'ESCO': [0.90, 'replace', 'hru'],
          }
          
params3 = {          
          'CN2': [-0.15, 'multiply', 'mgt'],
          'ESCO': [0.60, 'replace', 'hru'],
          }
swat_model.params = [params1, params2, params3]

subbasins1 = [1, 2, 3]
subbasins2 = [1, 2]
subbasins3 = [10, 11, 12, 13, 14]
swat_model.subbasins = [subbasins1, subbasins2, subbasins3]

# we define the list of hrus

hrus1 = [[1, 3], [2, 3], []]
hrus2 = [[2, 4], [1, 4]]
hrus3 = []
swat_model.hrus = [hrus1, hrus2, hrus3]

Note that the third element in hrus1 is an empty list, meaning that the subbasin 3 will have the parameters changed for all of its hrus. Similarly, note that hrus3 is also an empty list, which means that all the hrus within the subbasins 10, 11, ..., 14 will have their parameters modified.

Also note that subbasins 1 and 2 belong to both subbasins1 and subbasins2. However, their corresponding lists of hrus are different under each set. This is an example of how to apply different parameter modifications to different hrus within the same subbasin.

SWAT model calibration

In the src folder, there are Python scrips showing how to set up single- and multi-objective optimization problems using pymoo.

Contact

Feel free to contact me if you have any questions:

J. Sebastian Hernandez-Suarez
herna505@msu.edu
Computational Ecohydrology Group (C-ECO)
Michigan State University
East Lansing, MI 48824, US