Merge pull request #242 from NREL/develop

FLASC v2.2

.github/workflows/deploy-pages.yaml

@@ -29,6 +29,7 @@ jobs:
     - name: Copy examples to docs
       working-directory: ${{runner.workspace}}/flasc/
       run: |
+        rsync -av --mkpath  examples_artificial_data/01_raw_data_processing/ docs/examples/01_raw_data_processing
         rsync -av --mkpath  examples_artificial_data/03_energy_ratio/ docs/examples/03_energy_ratio
         rsync -av --mkpath  examples_artificial_data/floris_input_artificial/ docs/examples/floris_input_artificial
         ls docs/examples

.pre-commit-config.yaml

@@ -13,14 +13,15 @@ repos:
   - id: check-symlinks
   - id: mixed-line-ending
 
+repos:
 - repo: https://github.com/astral-sh/ruff-pre-commit
   # Ruff version.
-  rev: v0.1.7
+  rev: v0.7.3
   hooks:
     # Run the linter.
     - id: ruff
-      types_or: [ python, pyi, jupyter ]
+      types_or: [ python, pyi ]
       args: [ --fix ]
     # Run the formatter.
     - id: ruff-format
-      types_or: [ python, pyi, jupyter ]
+      types_or: [ python, pyi ]

docs/_toc.yml

@@ -15,18 +15,20 @@ parts:
     # - file: data_processing
     - file: flasc_data_format
     - file: energy_ratio
-    - file: total_uplift
+    - file: energy_change
     - file: licensing
 
   # - caption: Developer Reference
   #   chapters:
   #   - file: contributing
   #   - file: development
   #   - file: testing
+  - caption: Examples Data Processing
+    chapters:
+    - file: examples/01_raw_data_processing/03_northing_calibration_hoger
 
-  - caption: Examples
+  - caption: Examples Energy Ratio
     chapters:
-    # - file: flascdataframe
     - file: examples/03_energy_ratio/00_demo_energy_ratio_syntax
     - file: examples/03_energy_ratio/01_demo_energy_ratio_options
 

docs/energy_change.md

@@ -0,0 +1,32 @@
+# Calculating the change in energy production
+
+To investigate the change in energy production from a field test of a technology such as wind farm control, the FLASC repository was originally built around [energy ratio analysis](energy_ratio) with a focus on observing patterns in changes in energy production and visually comparing those with those changes expected from wake models.
+
+FLASC now however includes three methods for quantifying the change in energy production.
+
+## Total Uplift Power Ratio
+
+`total_uplift_power_ratio` uses a similar input as the [energy ratio](energy_ratio) methods but returns a single value representing the total uplift, rather than uplift binned by wind direction.  The method is named power_ratio because the change in energy production is computed using the mean per-wind-condition-bin power ratios.  The change in ratio per bin is then combined with the mean base power and frequency to estimate change in energy production.
+
+Currently the main example usage of the total uplift function is at the end of [smarteole example 06](https://github.com/NREL/flasc/blob/main/examples_smarteole/06_wake_steering_energy_ratio_analysis.ipynb).  Documentation of the function itself is available in the [API documentation](https://nrel.github.io/flasc/_autosummary/flasc.analysis.total_uplift_power_ratio.compute_total_uplift.html#flasc.analysis.total_uplift_power_ratio.compute_total_uplift).  Uncertainty of the results can be computed via bootstrapping.
+
+The method was developed by Eric Simley and implemented by Paul Fleming and Misha Sinner of NREL.  
+
+## Wind-Up
+
+FLASC further includes methods for calculating change in energy production using the [wind-up](https://github.com/resgroup/wind-up) module.   [wind-up](https://github.com/resgroup/wind-up) is a tool to assess yield uplift of wind turbines developed by Alex Clerc of RES and available open-source on GitHub.  Using translation methods in the [FlascDataFrame](flasc_data_format), the methods and analysis of wind-up can be invoked from FLASC.
+
+[smarteole example 09](https://github.com/NREL/flasc/blob/main/examples_smarteole/09_wind-up_wake_steering_uplift_analysis.ipynb), calculates the change in energy production (as in [smarteole example 06](https://github.com/NREL/flasc/blob/main/examples_smarteole/06_wake_steering_energy_ratio_analysis.ipynb)) using wind-up.
+
+## Expected Power Analysis
+
+The final included methodology for calculating change in energy production is the module `expected_power_analysis`.  This module implements the calculations of change in energy production described in [AWC validation methodology](https://publications.tno.nl/publication/34637216/LWOb3s/TNO-2020-R11300.pdf), by Stoyan Kanev of TNO.  The method was implemented into python/FLASC by Paul Fleming and Eric Simley of NREL referring to the above publication by Stoyan Kanev.  The method is named `expected_power_analysis` within FLASC to denote its calculation of the expected farm power as the weighted sum of the per-bin expected powers, rather than power ratios, and using this to calculate the change in energy production.
+
+Specifically, this module computes the total uplift along with the confidence interval of the total uplift estimate by implementing Equations 4.11 - 4.29 in the abovementioned TNO report. To determine the expected wind farm power for each wind direction/wind speed bin the expected power of each individual turbine is summed for the bin. One advantage of this method is that by computing expected power at the turbine level before summing, the method does not require that all test turbines are operating normally at each timestamp. Total wind farm energy is then computed by summing the expected farm power values weighted by their frequencies of occurrence over all wind condition bins. 
+
+The module provides two approaches for quantifying uncertainty in the total uplift. First, bootstrapping can be used similar to the first two methods. The second option approximates uncertainty in the total uplift following the approach in the abovementioned TNO report by propagating the standard errors of the expected wind farm power in each wind condition bin for the two control modes following analytic expressions derived by linearizing the total uplift formula. Benefits of this approach include higher computational efficiency compared to bootstrapping, which relies on computing the uplift for many different iterations. However, challenges with computing the required variances and covariances of wind turbine power can arise for bins with very little data (though this is accounted for automatically by approximating the missing terms).
+
+The approach is different from the above approaches in several ways (refer to [AWC validation methodology](https://publications.tno.nl/publication/34637216/LWOb3s/TNO-2020-R11300.pdf) for full description).  First, as mentioned above, the uncertainty of the result can be computed directly from the variance and co-variances of the turbine powers instead of relying on the more computationally expensive bootstrapping approach.  Additionally, the method does not normalize the power of the test turbines by reference powers.  Therefore, the method may be more sensitive to wind speed variations within bins and other atmospheric conditions that would otherwise be partially controlled for through normalization by reference powers. To account for this sensitivity we suggest using smaller wind speed bins and wind speed estimates based on measured turbine performance (rather than nacelle anemometry) when calculating the expected power.  However, by avoiding normalization by a reference power signal, this method does not require all test turbines to be operating normally at each sample, increasing the amount of usable data, especially when large wind farms are being analyzed.
+
+Similar to above, an example using the smarteole data provides usage example, see [smarteole example 10](https://github.com/NREL/flasc/blob/main/examples_smarteole/examples_smarteole/10_uplift_with_expected_power.ipynb)
+

examples_artificial_data/03_energy_ratio/00_demo_energy_ratio_syntax.ipynb

@@ -33,7 +33,7 @@
     "\n",
     "from flasc import FlascDataFrame\n",
     "from flasc.analysis import energy_ratio as erp\n",
-    "from flasc.analysis.energy_ratio_input import EnergyRatioInput"
+    "from flasc.analysis.analysis_input import AnalysisInput"
    ]
   },
   {
@@ -220,7 +220,7 @@
    "source": [
     "# Energy Ratio Input\n",
     "\n",
-    "In the new syntax, the first step in computing an energy is building an EnergyRatioInput object.  This is analagous to the construction of an energy ratio suite object in prior versions.  \n",
+    "In the new syntax, the first step in computing an energy is building an AnalysisInput object.  This is analagous to the construction of an energy ratio suite object in prior versions.  \n",
     "\n",
     "The inputs to provide are the list of dataframes, and the names of those dataframes.  An optional 3rd input is how many blocks to divide the data into, this will be used in block bootstrapping and is done here to save time.  Setting the number of blocks equal to the number of rows in the dataframe with the smallest number of rows approximates non-block boostreapping/"
    ]
@@ -231,7 +231,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "er_in = EnergyRatioInput(\n",
+    "a_in = AnalysisInput(\n",
     "    [df_baseline_noisy_pd, df_wakesteering_noisy_pd], [\"baseline\", \"wakesteering\"], num_blocks=10\n",
     ")"
    ]
@@ -242,7 +242,7 @@
    "source": [
     "# Computing the energy ratio\n",
     "\n",
-    "Computing the energy ratio is now done by passing the above er_in into the compute_energy_ratio function.  The energy ratio expects to be instructed which sets of turbines to average to produce the wind speed, wind direction, reference power and test power.  However, there is also the option to use pre-defined values, in which case the columns 'wd', 'ws' and/or 'pow_ref' need to be already defined in the dataframes.\n",
+    "Computing the energy ratio is now done by passing the above a_in into the compute_energy_ratio function.  The energy ratio expects to be instructed which sets of turbines to average to produce the wind speed, wind direction, reference power and test power.  However, there is also the option to use pre-defined values, in which case the columns 'wd', 'ws' and/or 'pow_ref' need to be already defined in the dataframes.\n",
     "\n",
     "In this example since wd,ws,and pow_ref are defined, we'll use that option here"
    ]
@@ -270,7 +270,7 @@
    "outputs": [],
    "source": [
     "er_out = erp.compute_energy_ratio(\n",
-    "    er_in,\n",
+    "    a_in,\n",
     "    test_turbines=[2],\n",
     "    use_predefined_ref=True,\n",
     "    use_predefined_wd=True,\n",
@@ -369,12 +369,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
     "er_out = erp.compute_energy_ratio(\n",
-    "    er_in,\n",
+    "    a_in,\n",
     "    test_turbines=[2],\n",
     "    use_predefined_ref=True,\n",
     "    use_predefined_wd=True,\n",
@@ -429,12 +429,12 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
     "er_out = erp.compute_energy_ratio(\n",
-    "    er_in,\n",
+    "    a_in,\n",
     "    test_turbines=[2],\n",
     "    use_predefined_ref=True,\n",
     "    use_predefined_wd=True,\n",

examples_artificial_data/03_energy_ratio/01_demo_energy_ratio_options.ipynb

@@ -30,7 +30,7 @@
     "\n",
     "from flasc import FlascDataFrame\n",
     "from flasc.analysis import energy_ratio as erp\n",
-    "from flasc.analysis.energy_ratio_input import EnergyRatioInput\n",
+    "from flasc.analysis.analysis_input import AnalysisInput\n",
     "from flasc.data_processing import dataframe_manipulations as dfm"
    ]
   },
@@ -206,7 +206,7 @@
    "outputs": [],
    "source": [
     "# Build the energy ratio input\n",
-    "er_in = EnergyRatioInput([df], [\"baseline\"], num_blocks=10)"
+    "a_in = AnalysisInput([df], [\"baseline\"], num_blocks=10)"
    ]
   },
   {
@@ -242,7 +242,7 @@
     "# Calculate and plot the energy ratio of turbine 2 with respect to\n",
     "# turbine 0, using turbine 0's measurements of wind speed and wind direction\n",
     "er_out = erp.compute_energy_ratio(\n",
-    "    er_in, test_turbines=[2], ref_turbines=[0], ws_turbines=[0], wd_turbines=[0], N=50\n",
+    "    a_in, test_turbines=[2], ref_turbines=[0], ws_turbines=[0], wd_turbines=[0], N=50\n",
     ")\n",
     "er_out.plot_energy_ratios()"
    ]
@@ -280,7 +280,7 @@
     "# Reverse the above calculation showing the energy ratio of T0 / T2,\n",
     "# letting T1 supply wind speed and direction\n",
     "er_out = erp.compute_energy_ratio(\n",
-    "    er_in, test_turbines=[0], ref_turbines=[2], ws_turbines=[1], wd_turbines=[1], N=50\n",
+    "    a_inest_turbines=[0], ref_turbines=[2], ws_turbines=[1], wd_turbines=[1], N=50\n",
     ")\n",
     "er_out.plot_energy_ratios()"
    ]
@@ -314,11 +314,11 @@
    "source": [
     "# Overplot the energy ratios of turbine 2 and 3, with respect to the averages of turbines 0 and 1\n",
     "er_out_2 = erp.compute_energy_ratio(\n",
-    "    er_in, test_turbines=[2], ref_turbines=[0, 1], ws_turbines=[0, 1], wd_turbines=[0, 1], N=50\n",
+    "    a_in, test_turbines=[2], ref_turbines=[0, 1], ws_turbines=[0, 1], wd_turbines=[0, 1], N=50\n",
     ")\n",
     "\n",
     "er_out_3 = erp.compute_energy_ratio(\n",
-    "    er_in, test_turbines=[3], ref_turbines=[0, 1], ws_turbines=[0, 1], wd_turbines=[0, 1], N=50\n",
+    "    a_in, test_turbines=[3], ref_turbines=[0, 1], ws_turbines=[0, 1], wd_turbines=[0, 1], N=50\n",
     ")\n",
     "\n",
     "fig, axarr = plt.subplots(3, 1, sharex=True, figsize=(8, 11))\n",
@@ -506,7 +506,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": null,
    "metadata": {},
    "outputs": [
     {
@@ -536,10 +536,10 @@
    "source": [
     "# Now use the predefined values in the calculation of the average of turbines 2 and 3\n",
     "\n",
-    "er_in = EnergyRatioInput([df], [\"baseline\"], num_blocks=10)\n",
+    "a_in = AnalysisInput([df], [\"baseline\"], num_blocks=10)\n",
     "\n",
     "er_out = erp.compute_energy_ratio(\n",
-    "    er_in,\n",
+    "    a_in,\n",
     "    test_turbines=[2, 3],\n",
     "    use_predefined_ref=True,\n",
     "    use_predefined_wd=True,\n",

examples_artificial_data/03_energy_ratio/02_energy_ratio_vs_wd_for_single_df.py

@@ -4,7 +4,7 @@
 import pandas as pd
 
 from flasc.analysis import energy_ratio as er
-from flasc.analysis.energy_ratio_input import EnergyRatioInput
+from flasc.analysis.analysis_input import AnalysisInput
 from flasc.data_processing import dataframe_manipulations as dfm
 from flasc.utilities import floris_tools as fsatools
 from flasc.utilities.utilities_examples import load_floris_artificial as load_floris
@@ -67,11 +67,11 @@ def load_data():
     df = dfm.set_pow_ref_by_turbines(df, turbine_numbers=[0, 6])
 
     # # Initialize energy ratio object for the dataframe
-    er_in = EnergyRatioInput([df], ["baseline"])
+    a_in = AnalysisInput([df], ["baseline"])
 
     # Get energy ratio without uncertainty quantification
     er_out = er.compute_energy_ratio(
-        er_in,
+        a_in,
         test_turbines=[1],
         use_predefined_ref=True,
         use_predefined_wd=True,
@@ -91,7 +91,7 @@ def load_data():
     # Get energy ratio with uncertainty quantification
     # using N=20 bootstrap samples and 5-95 percent conf. bounds.
     er_out = er.compute_energy_ratio(
-        er_in,
+        a_in,
         test_turbines=[1],
         use_predefined_ref=True,
         use_predefined_wd=True,
@@ -107,9 +107,9 @@ def load_data():
 
     # Get energy ratio with uncertainty quantification
     # using N=20 bootstrap samples and without block bootstrapping.
-    er_in_noblocks = EnergyRatioInput([df], ["baseline"], num_blocks=len(df))
+    a_in_noblocks = AnalysisInput([df], ["baseline"], num_blocks=len(df))
     er_out = er.compute_energy_ratio(
-        er_in_noblocks,
+        a_in_noblocks,
         test_turbines=[1],
         use_predefined_ref=True,
         use_predefined_wd=True,

examples_artificial_data/03_energy_ratio/03_energy_ratio_vs_wd_for_multiple_dfs.py

@@ -5,7 +5,7 @@
 from floris.utilities import wrap_360
 
 from flasc.analysis import energy_ratio as er
-from flasc.analysis.energy_ratio_input import EnergyRatioInput
+from flasc.analysis.analysis_input import AnalysisInput
 from flasc.data_processing import dataframe_manipulations as dfm
 from flasc.utilities import floris_tools as fsatools
 from flasc.utilities.utilities_examples import load_floris_artificial as load_floris
@@ -69,13 +69,13 @@ def load_data():
     # Initialize the energy ratio input object and add dataframes
     # separately. We will add the original data and the manipulated
     # dataset.
-    er_in = EnergyRatioInput([df, df2], ["Original data", "Data with wd bias of 7.5 degrees"])
+    a_in = AnalysisInput([df, df2], ["Original data", "Data with wd bias of 7.5 degrees"])
 
     # Calculate the energy ratios for test_turbines = [1] for a subset of
     # wind directions with uncertainty quantification using 50 bootstrap
     # samples
     er_out = er.compute_energy_ratio(
-        er_in,
+        a_in,
         test_turbines=[1],
         use_predefined_ref=True,
         use_predefined_wd=True,
@@ -91,7 +91,7 @@ def load_data():
 
     # Look at another test turbine with the same masked datasets
     er_out = er.compute_energy_ratio(
-        er_in,
+        a_in,
         test_turbines=[3],
         use_predefined_ref=True,
         use_predefined_wd=True,

examples_artificial_data/03_energy_ratio/04_energy_ratio_in_turbine_array.py

@@ -8,7 +8,7 @@
 from floris.utilities import wrap_360
 
 from flasc.analysis import energy_ratio as er
-from flasc.analysis.energy_ratio_input import EnergyRatioInput
+from flasc.analysis.analysis_input import AnalysisInput
 from flasc.data_processing import dataframe_manipulations as dfm
 from flasc.utilities.utilities_examples import load_floris_artificial as load_floris
 
@@ -69,7 +69,7 @@ def _calculate_energy_ratios(df, test_turbines, aligned_wd, N=1):
     df = dfm.filter_df_by_ws(df, [6, 10])
 
     # Finally, construct the energy ratio input with the dataframe
-    er_in = EnergyRatioInput([df], ["baseline"])
+    a_in = AnalysisInput([df], ["baseline"])
 
     # Now, we calculate the energy ratio for each turbine for the one wind
     # direction and wind speed bin. We save those values to
@@ -78,7 +78,7 @@ def _calculate_energy_ratios(df, test_turbines, aligned_wd, N=1):
     for ti in test_turbines:
         # Get energy ratios
         er_out = er.compute_energy_ratio(
-            er_in,
+            a_in,
             test_turbines=[ti],
             use_predefined_ref=True,
             use_predefined_wd=True,

examples_artificial_data/03_energy_ratio/06_wake_steering_example.py

@@ -5,7 +5,7 @@
 
 from flasc import FlascDataFrame
 from flasc.analysis import energy_ratio as er
-from flasc.analysis.energy_ratio_input import EnergyRatioInput
+from flasc.analysis.analysis_input import AnalysisInput
 from flasc.utilities.utilities_examples import load_floris_artificial as load_floris
 from flasc.visualization import plot_binned_mean_and_ci
 
@@ -136,7 +136,7 @@
     color_palette = sns.color_palette("Paired", 4)[::-1]
 
     # Initialize the energy ratio input object
-    er_in = EnergyRatioInput(
+    a_in = AnalysisInput(
         [df_baseline, df_wakesteering, df_baseline_noisy, df_wakesteering_noisy],
         ["Baseline", "WakeSteering", "Baseline (Noisy)", "WakeSteering (Noisy)"],
     )
@@ -145,7 +145,7 @@
     # With respect to reference turbine [0]
     # datasets with uncertainty quantification using 50 bootstrap samples
     er_out = er.compute_energy_ratio(
-        er_in,
+        a_in,
         test_turbines=[2],
         use_predefined_ref=True,
         use_predefined_wd=True,