Routing extension for VIC image driver. #231

wietsefranssen · 2015-07-30T17:46:38Z

Affected files in the original VIC files are: ./drivers/image/Makefile and ./drivers/image/src/vic_image.c.

To change the routing scheme once they are implemented, only change the following line in the makefile
-> ROUT=rout_stub

Still only an empty routing module is implemented (rout_stub). See ./extensions/rout_stub

closes #24

Affected files in the original VIC files are: './drivers/image/Makefile' and './drivers/image/src/vic_image.c'. To change the routing scheme once they are implemented, only change the following line in the makefile -> ROUT=rout_stub Still only an empty routing module is implemented (rout_stub). See ./extensions/rout_stub

…e/routing_lohmann * 'develop' of https://github.com/UW-Hydro/VIC: update prefix string allocation size Cleanup ascii output file header. The major change here is that the header does not include a comment for the line containing the variable names. move out_data_file_struct and Error_struct to shared driver header (from vic_run)

bartnijssen · 2015-07-30T17:54:55Z

@jhamman : I'll let you go through this first. Let me know when I should take a look. Mare sure that code is formatted correctly (uncrustify)

jhamman · 2015-07-30T19:00:22Z

vic/extensions/rout_stub/include/rout.h

+#ifndef ROUT_STUB_H
+#define ROUT_STUB_H
+
+/******************************************************************************


Let's include this line here as well:

#define ROUT_EXT "rout_stub"

This will give any routines using rout.h some knowledge about which extension is active. This will become more important when you get to the other extensions.

Okay, I will do that

jhamman · 2015-07-30T19:17:39Z

Everything looks good so far here. There are a few small fixes. Starting with your next commit, run uncrustify on all the files you have changed before you commit.

bartnijssen · 2015-07-30T22:12:30Z

When running uncrustify use the configuration file in the repo under tools/code_format. That directory also includes a readme with instructions for running uncrustify.

…e/routing_lohmann_dev # By Joe Hamman (82) and others # Via Joe Hamman (22) and Bart Nijssen (16) * 'develop' of https://github.com/UW-Hydro/VIC: (90 commits) Fix links for forcing input documentation Update FAQ about running on OS X and tone down the CAPS SHOUTING remove other unused declarations remove find_average_layer from header files fixing up UW-Hydro#258 Updates to Model Overview and References * Clarified descriptions for partial veg cover fraction and time-varying veg parameters. * Added references to Bohn and Vivoni (2015). * Added schematic of partial veg cover fraction. * Numbered all figures in the model overview. * Clarified descriptions of a few other features. * Added references to Francini and Pacciani (1991) for soil evaporation and ARNO baseflow; added reference to Brooks and Corey (1964) for gravity-driven drainage. * Added links from all references to the References.md page where links were not present. * Added references to Bohn and Vivoni (2015), Francini and Pacciani (1991), and Brooks and Corey (1964) to the References.md page. * Alphabetized references on the References.md page. Rewrite of the VIC front page README update links in readme to new website dont put badge in readme header clean up readme files a bit add comment to python driver setup.py about logging add doxygen config file add readme to docs dir cleanup mkdocs.yml Removed docs/Development/ModelDevelopment.md from mkdocs.yml Minor edits to documentation set log level in setup.py fix a few bugs in vic_time.c, add unit tests for main functions update outline levels in model overview remove goes radiation doc link (since it doesn't exisit) ... Conflicts: vic/drivers/image/Makefile

…e/routing_lohmann_dev * 'develop' of https://github.com/UW-Hydro/VIC: (51 commits) update git ignore fix bash syntax debugging multi os python build update python miniconda for osx and linux updates to python builds, including osx builds for all python tests update nc and mpi paths in image driver build build image driver on osx move out_data structures to shared driver move zero_output_list to driver/shared remove brew doctor brew doctor debugging homebrew install print path in classic driver builds update osx build fix macos build update macos build update mac os build add brew_installs functionality travis update simplier macos build for classic ...

*Made mapping: rout-outlet-index to VIC-index *Filled: OUT_DISCHARGE variable

bartnijssen · 2015-09-24T00:59:04Z

@wietsefranssen : Hi Wietse - can you give an update one where things are at?

wietsefranssen · 2015-09-24T13:27:02Z

Sorry for the long quietness!
You will hear from me next Monday or Tuesday.

…e/routing_lohmann_dev * 'develop' of https://github.com/UW-Hydro/VIC: Made the reference to Liang et al (1994) in readme.md more explicit. Moved ATBD reference into main references section. Emphasized Liang et al 1994 as primary reference. refactor comparisons to use type separate functions remove unused measure_h update comparisons module initial commit for VIC.5 docs update fix warnings on soome compilers that localtime_r is declared implicitly add comparisons module remove log_ncerr macro and replace with log_err, added helpful error messages fix zero dim array error raised by PGI compilers fix failing test for python 3, decode bytestring fix typo in vic_mpi_support.c previous commit fixes know failing test fix log to file in image driver when running mpi, now brodcasting filenames struct

outlet index of VIC gridcell check

…re.c, state_metadata.c and vic_driver_shared_all.h to the extensions folders: rvic and stub

wietsefranssen · 2017-09-19T09:38:10Z

@jhamman @bartnijssen.
We separated the state restore (and state save) functions as suggested by @bartnijssen (Mar 4 2017) in order to avoid having the variable 'STATE_ROUT_RING' in the statefile while 'ROUT_STUB' is enabled (no routing).
Also 'statefile.md' and 'ReleaseNotes.md' have been adapted.

The Output state variables (enum) are defined in 'vic_driver_shared_all.h'. This headerfile will be compiled by all drivers. This means that we needed to find a way to only include 'STATE_ROUT_RING' when the extension RVIC is enabled ('N_STATE_VARS' should be equal to the exact number of state variables that are written to the statefile). We experimented with various options including moving 'vic_driver_shared_all.h' to the extension folders. However this caused compilation issues for the other drivers. The solution we implemented is using pre-processor directive '#ifdef EXTENSION_RVIC' in 'vic_driver_shared_all.h'. Note that the definition of 'EXTENSION_RVIC' cannot be placed in 'rout.h' because 'vic_driver_shared_all.h' needs to be called first. Therefore we added a header file 'image_extension_name.h' with a definition of the extension name in every extension (eg. '#define EXTENSION_STUB' for the STUB extension).

jhamman · 2017-09-19T15:29:22Z

@wietsefranssen - it looks like a few files (vic_restore.c, vic_store.c) may have been accidentally moved to the routing extension. Can you take a look into that and see if that can be fixed.

I think we can avoid any #ifdef preprocessor calls in the c code but we'll wait to look at that until after the issue described above is fixed.

wietsefranssen · 2017-09-25T12:57:23Z

@jhamman @bartnijssen
We did a first attempt to move the vic_store.c back to its original position. Please have a look if this is the way forward. If this is the case we will proceed with vic_restore.c.

shared_image/src. This included some refactoring for gather_put_nc_field_double() and get_scatter_nc_field_double()

test suite

should only depend on shared_image)

Final fixes for the routing extension

jhamman

I had just a few minor semanitc comments but this looks ready to go. Thanks @wietsefranssen
and @isupit for sticking with this. Thanks @yixinmao for helping bring this across the finish line.

jhamman · 2017-10-20T18:03:38Z

docs/Development/ReleaseNotes.md

@@ -107,6 +107,10 @@ To check which release of VIC you are running:

 	The VIC image and CESM drivers now may be optionally compiled with OPENMP to enable shared memory thread parallelization. This option should improve the parallel scaling of these drivers by reducing the number of MPI messages and increasing message size.

+8. Added routing extensions ROUT_STUB and ROUT_RVIC for the VIC image driver ([GH#231](https://github.com/UW-Hydro/VIC/pull/231))
+
+    The VIC image driver can be optionally compiled with ROUT_RVIC to enable routing in image mode (ROUT_STUB is the default extension which means no routing). With ROUT_RVIC enabled, the output variable ``OUT_DISCHARGE`` is available, and there will also be an extra state variable ``STATE_ROUT_RING`` stored in the state file.


insert "streamflow" before "routing" here and in line 110.

jhamman · 2017-10-20T18:27:21Z

vic/drivers/image/Makefile

+# |----------  |---------------------------------   |
+# | rout_stub  | Stub routing model (no routing)    |
+# | rout_rvic  | Use RVIC routing scheme            |
+ROUT=rout_stub


should this be ROUT := rout_stub to allow it to be set on the command line via make ROUT=rout_rvic?

ROUT=rout_rvic is also working with ROUT=rout_stub, but for future additions it would probably better to use your suggestion. We included it.

jhamman · 2017-10-20T18:30:23Z

vic/drivers/shared_image/include/vic_driver_shared_image.h

@@ -27,6 +27,7 @@
 #ifndef VIC_DRIVER_SHARED_IMAGE_H
 #define VIC_DRIVER_SHARED_IMAGE_H

+#include <image_extension_name.h>


should we call this rout_extension_name.h?

This indeed streamlines better with @bartnijssen latest additions.
Done.

jhamman · 2017-10-20T18:30:43Z

vic/drivers/shared_image/include/vic_image_log.h

@@ -66,7 +66,7 @@
 #define check_nc_status(A, M, ...) if (A != NC_NOERR) {log_ncerr(A, M, \
                                                                 ## __VA_ARGS__); \
                                                       errno = 0; exit( \
-                                                           EXIT_FAILURE);}
+                                                           EXIT_FAILURE); }


is this change required?

This change was done by run_crustify

jhamman · 2017-10-20T18:34:27Z

vic/drivers/shared_image/src/vic_restore.c

@@ -882,6 +883,9 @@ vic_restore(void)
        }
    }

+    // routing ring
+    vic_restore_extension(&(filenames.init_state), state_metadata);


rename function to vic_restore_rout_extension

done. also changed vic_store_extension into vic_store_rout_extension

jhamman · 2017-10-20T18:34:52Z

vic/drivers/shared_image/src/vic_store.c

@@ -1269,6 +1270,10 @@ vic_store(dmy_struct *dmy_state,
        }
    }

+    // store extension variables
+    vic_store_extension(&nc_state_file);


rename to vic_store_rout_extension

done (see above sorry ;) )

jhamman · 2017-10-20T18:35:26Z

vic/drivers/shared_image/src/vic_store.c

@@ -1324,9 +1329,13 @@ set_nc_state_file_info(nc_file_struct *nc_state_file)
    nc_state_file->time_size = NC_UNLIMITED;
    nc_state_file->veg_size = options.NVEGTYPES;

+    // set ids and dimension sizes of the extension variables
+    set_nc_state_file_info_extension(nc_state_file);


rename to set_nc_state_file_info_rout_extension

jhamman · 2017-10-20T18:35:56Z

vic/drivers/shared_image/src/vic_store.c

        if (options.LAKES) {
            status = nc_def_dim(nc_state_file->nc_id, "lake_node",
                                nc_state_file->lake_node_size,
                                &(nc_state_file->lake_node_dimid));
            check_nc_status(status, "Error defining lake_node in %s", filename);
        }

+        // add extension dimensions
+        initialize_state_file_extension(filename, nc_state_file);


rename to inialize_state_file_rout_extension

yixinmao · 2017-10-20T22:35:48Z

Thanks @jhamman and @bartnijssen for review. Also, another thing to update before merging this PR is to sync with the latest develop branch.

bartnijssen · 2017-10-20T23:54:48Z

VICTORY 🎉

wietsefranssen · 2017-10-23T09:08:33Z

GREAT!! Thanks to all!

wietsefranssen · 2017-10-23T13:25:15Z

@ALL We included the changes suggested by @jhamman. and made a new pull request.

wietsefranssen added 2 commits July 30, 2015 19:35

jhamman reviewed Jul 30, 2015
View reviewed changes

First attempt to make the RVIC routing extension. Not finished/okay yet.

fef012a

jhamman mentioned this pull request Aug 5, 2015

Adds CN to classic driver. Still includes verbose option. #174

Closed

wietsefranssen added 2 commits August 26, 2015 11:47

Initial commit for lohman routing (dev).

058c0c6

jhamman assigned wietsefranssen Sep 13, 2015

jhamman modified the milestones: 5.1, someday Sep 13, 2015

jhamman added image driver feature extensions labels Sep 13, 2015

jhamman modified the milestones: 5.1, someday Sep 13, 2015

wietsefranssen added 3 commits September 14, 2015 12:44

Small update (eg Makefile)

34433fe

*Created: OUT_DISCHARGE variable for NetCDF

c06ae0a

*Made mapping: rout-outlet-index to VIC-index *Filled: OUT_DISCHARGE variable

wietsefranssen added 4 commits September 28, 2015 10:23

uncrusified routing related files

ad88b4c

tiny updates

1dd197f

source index of VIC gridcell check

f3358e9

outlet index of VIC gridcell check

wietsefranssen added 6 commits September 18, 2017 10:40

solved merge conflict in vic_store.c

b2f8737

removed rout_ring for initstate file by moving vic_restore.c, vic_sto…

f6a593f

…re.c, state_metadata.c and vic_driver_shared_all.h to the extensions folders: rvic and stub

added empty vic_driver_shared_all.h to shared_all folder

461796c

test travis

adf31aa

new extension header included

22b2775

Added changelog info and statefile info

cb8c7ce

first attempt to move the vic_store.c file back to its original position

54eabab

ymao added 10 commits October 13, 2017 12:02

Clean up vic_store.c and related extension

23cfe54

Moved vic_restore.c back to "shared_image/src" and cleaned up extension

1ef2e9b

Moved state_metadata.c back to shared_image/src and cleaned up extension

3eb5296

Update routing-related documentation

7b776c9

Clean up and small bug fix for vic_restore.c

ec77031

Moved gather_field_double() and scatter_field_double() to

feae8ee

shared_image/src. This included some refactoring for gather_put_nc_field_double() and get_scatter_nc_field_double()

Added routing extension outputs (state & discharge hist output) in the

2549773

test suite

Minor updates for routing doc

9580e78

Ran uncrustify

dd4f1af

Removed unused variables in vic_mpi_support.c

f061e51

yixinmao mentioned this pull request Oct 16, 2017

Final fixes for the routing extension wietsefranssen/VIC#8

Merged

3 tasks

ymao and others added 2 commits October 17, 2017 15:17

Remove dependence of extension header file rout.h on image driver (it

d3868d9

should only depend on shared_image)

Merge pull request #8 from yixinmao/routing_update

90c6a68

Final fixes for the routing extension

jhamman approved these changes Oct 20, 2017

View reviewed changes

bartnijssen approved these changes Oct 20, 2017

View reviewed changes

bartnijssen merged commit f76d299 into UW-Hydro:develop Oct 20, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Routing extension for VIC image driver. #231

Routing extension for VIC image driver. #231

wietsefranssen commented Jul 30, 2015 •

edited by jhamman

Loading

bartnijssen commented Jul 30, 2015

jhamman Jul 30, 2015

wietsefranssen Jul 30, 2015

jhamman commented Jul 30, 2015

bartnijssen commented Jul 30, 2015

bartnijssen commented Sep 24, 2015

wietsefranssen commented Sep 24, 2015

wietsefranssen commented Sep 19, 2017 •

edited

Loading

jhamman commented Sep 19, 2017

wietsefranssen commented Sep 25, 2017

jhamman left a comment

jhamman Oct 20, 2017

wietsefranssen Oct 23, 2017

jhamman Oct 20, 2017

wietsefranssen Oct 23, 2017

jhamman Oct 20, 2017

wietsefranssen Oct 23, 2017

jhamman Oct 20, 2017

yixinmao Oct 20, 2017

jhamman Oct 20, 2017

wietsefranssen Oct 23, 2017

jhamman Oct 20, 2017

wietsefranssen Oct 23, 2017

jhamman Oct 20, 2017

wietsefranssen Oct 23, 2017

jhamman Oct 20, 2017

wietsefranssen Oct 23, 2017

yixinmao commented Oct 20, 2017

bartnijssen commented Oct 20, 2017

wietsefranssen commented Oct 23, 2017

wietsefranssen commented Oct 23, 2017

Routing extension for VIC image driver. #231

Routing extension for VIC image driver. #231

Conversation

wietsefranssen commented Jul 30, 2015 • edited by jhamman Loading

bartnijssen commented Jul 30, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jhamman commented Jul 30, 2015

bartnijssen commented Jul 30, 2015

bartnijssen commented Sep 24, 2015

wietsefranssen commented Sep 24, 2015

wietsefranssen commented Sep 19, 2017 • edited Loading

jhamman commented Sep 19, 2017

wietsefranssen commented Sep 25, 2017

jhamman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yixinmao commented Oct 20, 2017

bartnijssen commented Oct 20, 2017

wietsefranssen commented Oct 23, 2017

wietsefranssen commented Oct 23, 2017

wietsefranssen commented Jul 30, 2015 •

edited by jhamman

Loading

wietsefranssen commented Sep 19, 2017 •

edited

Loading