AORC Example/Benchmark - WIP

[mikejohnson51]

Set up testing

1. Download AORC data from: https://hydrology.nws.noaa.gov/aorc-historic/

• The publicly available data is 4km split up by River Forecasting Centers. We'll test those in HUC01 (NERFC)

2. I downloaded one year/month and built a one day, Temperature composite file (24 time_slices)

ncrcat *nc4 -O merged_one_day.nc  

During Hydrofabric Creation:

3. We can read in catchment set, and build a weight grid.

4. This process is only needed once per input source and be executed in the creation of the hydrofabric.

✓ [master*]> cats = sf::read_sf(gpkg, "catchments")
✓ [master*]> w = zonal::weighting_grid(merged, cats, "ID" )
✓ [master*]> head(w)

$weight_map
              ID grid_id          w
    1:  cat-3739     156 0.01666190
    2:  cat-3739     159 0.13445126
    3:  cat-3739     160 0.19213705
    4:  cat-3739     355 0.19502342
    5:  cat-3739     356 0.29357070
   ---                             
76971: cat-39604   42830 0.05861758
76972: cat-39182   42831 0.14088909
76973: cat-39603   42832 0.00001536
76974: cat-39202   43029 0.43438056
76975: cat-39202   43030 0.00640736

$dims
xmin xmax ymin ymax 
 173  374  218    5 

I would advocated storing the weight files as a parquet file alongside the hydrofabrics for rapid cloud-based and/or partial read access. This is how we are dealing with the parameters sets for LSTM, CFE, and NOAH-OWP.

Real-time forcing Reads

6. With a weight grid, summarized forcings can be computed. This process is effectively only impacted by the speed in which a computer can extract the needed data from the forcing files.

✓  [master*]> system.time({
   xx = zonal::execute_zonal(file = m, w = w, join = FALSE) |> 
     data.table::setnames(c("ID", paste0("TEMP", 1:24)))
 })
   user  system elapsed 
  0.749   0.085   0.874 

✓ [master*]> dplyr::glimpse(xx)
Rows: 14,632
Columns: 25
$ ID     <chr> "cat-100", "cat-10001", "cat-1000…
$ TEMP1  <dbl> 262.7, 265.5, 266.1, 267.0, 268.5…
$ TEMP2  <dbl> 262.6, 265.6, 265.7, 266.9, 267.7…
$ TEMP3  <dbl> 261.5, 265.5, 265.6, 266.6, 267.5…
$ TEMP4  <dbl> 261.6, 265.1, 265.2, 266.6, 267.4…
$ TEMP5  <dbl> 261.1, 264.8, 264.6, 266.4, 267.2…
$ TEMP6  <dbl> 260.1, 264.7, 264.9, 266.7, 267.2…
$ TEMP7  <dbl> 260.3, 264.6, 264.8, 266.4, 267.2…
$ TEMP8  <dbl> 260.3, 265.1, 265.1, 266.7, 267.1…
$ TEMP9  <dbl> 259.7, 265.2, 265.1, 267.1, 266.9…
$ TEMP10 <dbl> 259.8, 264.5, 264.4, 266.4, 266.6…
$ TEMP11 <dbl> 259.5, 264.6, 264.5, 266.6, 266.8…
$ TEMP12 <dbl> 259.1, 264.7, 264.6, 267.3, 267.2…
$ TEMP13 <dbl> 258.8, 265.6, 265.4, 268.5, 267.3…
$ TEMP14 <dbl> 260.5, 266.3, 265.8, 268.7, 267.7…
$ TEMP15 <dbl> 264.3, 267.0, 266.5, 268.5, 268.9…
$ TEMP16 <dbl> 267.2, 269.1, 268.8, 269.2, 270.3…
$ TEMP17 <dbl> 268.3, 269.7, 269.3, 271.0, 271.4…
$ TEMP18 <dbl> 269.5, 272.7, 272.4, 272.5, 273.3…
$ TEMP19 <dbl> 270.4, 272.8, 272.7, 272.8, 274.6…
$ TEMP20 <dbl> 271.9, 272.6, 272.5, 272.6, 274.4…
$ TEMP21 <dbl> 270.5, 271.4, 271.4, 271.6, 273.1…
$ TEMP22 <dbl> 269.1, 269.6, 269.7, 269.5, 271.2…
$ TEMP23 <dbl> 267.3, 268.5, 268.7, 268.2, 269.4…
$ TEMP24 <dbl> 265.8, 267.7, 267.7, 268.0, 268.8…