RE Optimization w/ Dynamic Surrogates

[dguittet]

Addresses issue:

Wind + PEM Hybrid Optimization for December Milestone

Requires IDAES branch for grid integration changes: IDAES/idaes-pse#1019

Files for running conceptual design w/ dynamic surrogates:

• dispatches.case_studies.renewables_case/RE_surrogate_optimization.py
• dispatches.case_studies.renewables_case/SurrogateDesignResults.ipynb: shows cost curve, and optimal size surfaces of surrogate model vs prescient sweep; upcoming analysis on comparison with double loop revenue results

Files for running Double Loop with parameterized bid curve:

• workflow.parametrized_bidder.py: class interface for a bidder, implementation of a DataFrame-based PerfectForecaster
• dispatches.case_studies.renewables_case.PEM_parametrized_bidder.py: implementation of bidder using Wind + PEM bid structure from Prescient sweeps
• dispatches.case_studies.renewables_case/wind_PEM_double_loop.py: implementation of a grid-integration wrapper for the Wind + PEM flowsheet model
• dispatches.case_studies.renewables_case/run_double_loop_PEM.py: script to run the double loop

Summary:

Wind size is fixed to the original plant size (847 MW for 303_WIND_1) since the parameter sweep didn't vary the wind plant size

Optimal design:

Wind capacity = 847.0 MW
PEM capacity = 247.70494311964453MW
Plant bid = $29.711626518965524
Plant Revenue Annual = $24927.081507569157
Plant NPV = $-1252263849.6855943

TODO

• Re-train and save models using Tensorflow 2.12 2.10 (@Xinhe-Chen)
• Test that newly saved model works with Windows (@lbianchi-lbl)
• If it works: update this PR with the new saved model

Legal Acknowledgement

By contributing to this software project, I agree to the following terms and conditions for my contribution:

1. I agree my contributions are submitted under the license terms described in the LICENSE.md and COPYRIGHT.md file at the top level of this directory.
2. I represent I am authorized to make the contributions and grant the license. If my employer has rights to intellectual property that includes these contributions, I represent that I have received permission to make contributions and grant the required license on behalf of that employer.

[codecov]

Codecov Report

Patch coverage: 72.10% and project coverage change: +0.07 🎉

Comparison is base (a2a7bb9) 89.69% compared to head (a6fe777) 89.76%.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #172      +/-   ##
==========================================
+ Coverage   89.69%   89.76%   +0.07%     
==========================================
  Files          67       74       +7     
  Lines        8303     8854     +551     
==========================================
+ Hits         7447     7948     +501     
- Misses        856      906      +50     

Impacted Files	Coverage Δ
.../tests/test_multiperiod_wind_battery_doubleloop.py	100.00% <ø> (ø)
dispatches/unit_models/solar_pv.py	91.66% <ø> (ø)
dispatches/unit_models/tests/test_solar_pv.py	100.00% <ø> (ø)
...n_market_surrogates/dynamic/Train_NN_Surrogates.py	28.75% <28.75%> (ø)
...arket_surrogates/dynamic/Time_Series_Clustering.py	37.67% <37.80%> (ø)
...tudies/renewables_case/tests/test_surrogate_opt.py	73.91% <73.91%> (ø)
dispatches/workflow/parametrized_bidder.py	75.75% <75.75%> (ø)
...bles_case/RE_surrogate_optimization_steadystate.py	91.56% <91.56%> (ø)
...se_studies/renewables_case/wind_PEM_double_loop.py	92.85% <92.85%> (ø)
...t_surrogates/dynamic/tests/test_Simulation_Data.py	97.90% <97.90%> (ø)
... and 8 more

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[bknueven]

I left a few minor comments.

There seems to be a bit of duplicated code in this PR, but I guess its not a huge concern.

[lbianchi-lbl]

There's still the issue of loading Keras serialized models that causes one of the tests to fail on Windows. We can look into this (possibly through .gitattributes?) but it doesn't have to be done in this PR necessarily.

[lbianchi-lbl]

@lbianchi-lbl will try to see if .gitattributes is enough to resolve the Windows failures.

[lbianchi-lbl]

@dguittet just want to confirm this is the problem that affects model loading on Windows:

After checking out this PR's head branch and installing in a fresh Conda environment, I ran the following:

(test-dispatches-172)
lbian@DESKTOP-BMTV081 MINGW64 ~/proj/dispatches/dispatches (re_surrogate_opt)
$ python -c 'from dispatches.case_studies.renewables_case.RE_surrogate_optimization_steadystate import *; load_surrogate_model(re_nn_dir)'

And this is what I get:

C:\Users\lbian\miniconda3\envs\test-dispatches-172\lib\site-packages\sklearn\base.py:318: UserWarning: Trying to unpickle estimator KMea
ns from version 1.2.1 when using version 1.2.2. This might lead to breaking code or invalid results. Use at your own risk. For more info
 please refer to:
https://scikit-learn.org/stable/model_persistence.html#security-maintainability-limitations
  warnings.warn(
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\lbian\proj\dispatches\dispatches\dispatches\case_studies\renewables_case\RE_surrogate_optimization_steadystate.py", lin
e 66, in load_surrogate_model
    nn_rev = keras.models.load_model(re_nn_dir / "revenue" / "RE_revenue_2_25")
  File "C:\Users\lbian\miniconda3\envs\test-dispatches-172\lib\site-packages\keras\saving\saving_api.py", line 212, in load_model
    return legacy_sm_saving_lib.load_model(
  File "C:\Users\lbian\miniconda3\envs\test-dispatches-172\lib\site-packages\keras\utils\traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "C:\Users\lbian\miniconda3\envs\test-dispatches-172\lib\site-packages\keras\saving\legacy\saved_model\load.py", line 112, in load

    raise IOError(
OSError: Cannot parse keras metadata at path C:\Users\lbian\proj\dispatches\dispatches\dispatches\case_studies\renewables_case\data\stea
dy_state_surrogate\revenue\RE_revenue_2_25\keras_metadata.pb: Received error: Error parsing message

Running the same command on Linux seems to be working just fine.

[lbianchi-lbl]

@Xinhe-Chen:

• Try to see if you can find the version of Tensorflow used to save/generate this file
• Try (in a separate environment on your system) to see if you can load the serialized model used the python -c command in my comment above

[Xinhe-Chen]

My tensorflow version is 2.9.1.
I did not have that IOError when I ran it in a fresh environment.

[lbianchi-lbl]

Yesterday I changed the Keras models to those that @Xinhe-Chen retrained using Tensorflow 2.10, and pinned that version in setup.py. Currently, both Linux and Windows are failing:

• On Linux, deserializing (loading) the models work fine (as before), but the numeric results are different than the expectations in the tests:
  
• On Windows, loading the model does not fail outright as before, but the produced object seems to have a different type than expected, which causes the tests to fail
  • Using Linux as a reference, I've added tests that specifically check the types of the surrogates after deserialization (loading):
    
  • Looking at the warnings they suggest the possibility that the deserialization process is somehow only partially successful:
    

[lbianchi-lbl]

Finally merging this after re-training the model (@Xinhe-Chen) and figuring out the root cause of the remaining Windows error (@dguittet), which was fixed by marking those files as binary with .gitattributes.