Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Example 01 is crashing #149

Closed
1 task done
paulf81 opened this issue Dec 15, 2023 · 1 comment
Closed
1 task done

[BUG] Example 01 is crashing #149

paulf81 opened this issue Dec 15, 2023 · 1 comment
Assignees
Labels
bug Something isn't working v1.5
Milestone

Comments

@paulf81
Copy link
Collaborator

paulf81 commented Dec 15, 2023

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

The example is examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb is crashing on my machine when I try to run it. The error occurs in cell 5, pasting error below.

Think on the one hand should fix the issue and then also think a little bit more about including some example running within CI a la Issue #109. What do you think @misi9170 ?

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
[/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb) Cell 9 line 2
    [248](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=247)     df_pow_curve = ws_pow_filtering.pw_curve_df
    [250](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=249)     return df, df_pow_curve
--> [253](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=252) df_scada_filtered, df_pow_curve = filter_by_ws_pow_curve(
    [254](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=253)     df=df_scada_formatted.copy(),
    [255](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=254)     plot_figures=plot_figures,
    [256](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=255)     save_figures=save_figures,
    [257](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=256)     use_bokeh_for_plots=use_bokeh_plotting_library,
    [258](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=257) )

[/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb) Cell 9 line 1
     [93](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=92) ws_pow_filtering.filter_by_condition(
     [94](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=93)     condition=(ws_pow_filtering.df["ws_{:03d}".format(ti)] > 4.0)
     [95](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=94)     & (ws_pow_filtering.df["pow_{:03d}".format(ti)] < 1.0),
   (...)
     [98](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=97)     verbose=True,
     [99](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=98) )
    [101](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=100) # Other common filters here are based on turbine-specific/OEM-specific flags. For example,
    [102](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=101) # a 'run counter' may indicate how many seconds of a 10-minute period the turbine was
    [103](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=102) # reporting valid measurements. By filtering for the condition that that value is 600, or
   (...)
    [117](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=116) # In this case, we have an operational_status flag. If that has a 'False' value, then we
    [118](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=117) # mark those measurements as faulty.
--> [119](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=118) ws_pow_filtering.filter_by_condition(
    [120](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=119)     condition=(ws_pow_filtering.df["is_operation_normal_{:03d}".format(ti)] is False),
    [121](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=120)     label="Self-flagged (is_operation_normal==False)",
    [122](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=121)     ti=ti,
    [123](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=122)     verbose=True,
    [124](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=123) )
    [126](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=125) # Filter for sensor-stuck faults
    [127](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=126) ws_pow_filtering.filter_by_sensor_stuck_faults(
    [128](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=127)     columns=["wd_{:03d}".format(ti), "ws_{:03d}".format(ti)],
    [129](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=128)     ti=ti,
   (...)
    [132](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=131)     plot=False,
    [133](vscode-notebook-cell:/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/00_filter_ws_power_curves.ipynb#X11sZmlsZQ%3D%3D?line=132) )

File [~/Projects/FLORIS/flasc/flasc/turbine_analysis/ws_pow_filtering.py:244](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/Projects/FLORIS/flasc/flasc/turbine_analysis/ws_pow_filtering.py:244), in ws_pw_curve_filtering.filter_by_condition(self, condition, label, ti, verbose, apply_filters_to_df)
    [242](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/Projects/FLORIS/flasc/flasc/turbine_analysis/ws_pow_filtering.py:242) # Mark data as faulty on the dataframe
    [243](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/Projects/FLORIS/flasc/flasc/turbine_analysis/ws_pow_filtering.py:243) N_pre = [dff.df_get_no_faulty_measurements(df_in, tii) for tii in ti]
--> [244](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/Projects/FLORIS/flasc/flasc/turbine_analysis/ws_pow_filtering.py:244) df_out = dff.df_mark_turbdata_as_faulty(df=df_in, cond=condition, turbine_list=ti)
    [246](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/Projects/FLORIS/flasc/flasc/turbine_analysis/ws_pow_filtering.py:246) # Print the reduction in useful data to the console, if verbose
    [247](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/Projects/FLORIS/flasc/flasc/turbine_analysis/ws_pow_filtering.py:247) if verbose:

File [~/Projects/FLORIS/flasc/flasc/dataframe_operations/dataframe_filtering.py:138](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/Projects/FLORIS/flasc/flasc/dataframe_operations/dataframe_filtering.py:138), in df_mark_turbdata_as_faulty(df, cond, turbine_list, exclude_columns)
    [135](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/Projects/FLORIS/flasc/flasc/dataframe_operations/dataframe_filtering.py:135) for ti in turbine_list:
    [136](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/Projects/FLORIS/flasc/flasc/dataframe_operations/dataframe_filtering.py:136)     cols = [s for s in df.columns if s[-4::] == ('_%03d' % ti)
    [137](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/Projects/FLORIS/flasc/flasc/dataframe_operations/dataframe_filtering.py:137)             and s not in exclude_columns]
--> [138](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/Projects/FLORIS/flasc/flasc/dataframe_operations/dataframe_filtering.py:138)     df.loc[cond, cols] = None  # Delete measurements
    [140](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/Projects/FLORIS/flasc/flasc/dataframe_operations/dataframe_filtering.py:140) return df

File [~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:885](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:885), in _LocationIndexer.__setitem__(self, key, value)
    [882](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:882) self._has_valid_setitem_indexer(key)
    [884](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:884) iloc = self if self.name == "iloc" else self.obj.iloc
--> [885](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:885) iloc._setitem_with_indexer(indexer, value, self.name)

File [~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1795](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1795), in _iLocIndexer._setitem_with_indexer(self, indexer, value, name)
   [1791](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1791) for i, idx in enumerate(indexer):
   [1792](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1792)     if isinstance(idx, dict):
   [1793](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1793)         # reindex the axis to the new value
   [1794](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1794)         # and set inplace
-> [1795](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1795)         key, _ = convert_missing_indexer(idx)
   [1797](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1797)         # if this is the items axes, then take the main missing
   [1798](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1798)         # path first
   [1799](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1799)         # this correctly sets the dtype and avoids cache issues
   [1800](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1800)         # essentially this separates out the block that is needed
   [1801](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1801)         # to possibly be modified
   [1802](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1802)         if self.ndim > 1 and i == info_axis:
   [1803](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1803)             # add the new item, and set the value
   [1804](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1804)             # must have all defined axes if we have a scalar
   [1805](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1805)             # or a list-like on the non-info axes if we have a
   [1806](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:1806)             # list-like

File [~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:2607](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:2607), in convert_missing_indexer(indexer)
   [2604](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:2604)     indexer = indexer["key"]
   [2606](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:2606)     if isinstance(indexer, bool):
-> [2607](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:2607)         raise KeyError("cannot use a single bool to index into setitem")
   [2608](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:2608)     return indexer, True
   [2610](https://file+.vscode-resource.vscode-cdn.net/Users/pfleming/Projects/FLORIS/flasc/examples_artificial_data/01_raw_data_processing/~/opt/anaconda3/envs/floris/lib/python3.10/site-packages/pandas/core/indexing.py:2610) return indexer, False

KeyError: 'cannot use a single bool to index into setitem'

Expected Behavior

No response

Steps To Reproduce

No response

Environment

- OS:
- pip environment (can be retrieved with `pip list`):

Anything else?

No response

@paulf81 paulf81 added bug Something isn't working v1.5 labels Dec 15, 2023
@paulf81 paulf81 added this to the v1.5 milestone Dec 15, 2023
@paulf81 paulf81 self-assigned this Dec 15, 2023
@paulf81
Copy link
Collaborator Author

paulf81 commented Dec 15, 2023

Addressed by #154

@paulf81 paulf81 closed this as completed Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working v1.5
Projects
None yet
Development

No branches or pull requests

1 participant