FEMS Requests combined to avoid timeouts #1262

sthhs · 2023-11-18T12:41:05Z

My FEMS (Fenecon Home 10, Version 2022 and later) is very sensitive with answering requests at the REST/JSON interface. Too many requests lead to very long answering times (30 s or more) and therefore to timeouts in the OpenWB. In the OpenEMS community you can find that there is indeed a limitation for not overloading the FEMS. The impact on the OpenWB is shown on the attached picture.

With my modifications (combining of the requests in inverter.py, bat.py, counter.py) and increasing the timeout value in counter.py to 6 almost all of the peaks (occurences of timeouts that lead to the wrong calculation of the consumption) disappear. They aren't gone totally, but reduced to a very small quantity.

sthhs · 2023-11-19T09:03:39Z

Here are more pictures.
Without the modification:

Corresponding error message:

and with:

LKuemmel · 2023-11-21T12:03:45Z

packages/modules/devices/fems/bat.py

+       # response = session.get(
+       #     "http://" + self.ip_address +
+       #     ":8084/rest/channel/_sum/(GridActivePower|ProductionActivePower|ConsumptionActivePower)",
+       #     timeout=2).json()
+       # for singleValue in response:
+       #     address = singleValue["address"]
+       #     if (address == "_sum/GridActivePower"):
+       #         grid = scale_metric(singleValue['value'], singleValue.get('unit'), 'W')
+       #     elif address == "_sum/ProductionActivePower":
+       #         pv = scale_metric(singleValue['value'], singleValue.get('unit'), 'W')
+       #     elif address == "_sum/ConsumptionActivePower":
+       #         haus = scale_metric(singleValue['value'], singleValue.get('unit'), 'W')
+


Suggested change

# response = session.get(

# "http://" + self.ip_address +

# ":8084/rest/channel/_sum/(GridActivePower|ProductionActivePower|ConsumptionActivePower)",

# timeout=2).json()

# for singleValue in response:

# address = singleValue["address"]

# if (address == "_sum/GridActivePower"):

# grid = scale_metric(singleValue['value'], singleValue.get('unit'), 'W')

# elif address == "_sum/ProductionActivePower":

# pv = scale_metric(singleValue['value'], singleValue.get('unit'), 'W')

# elif address == "_sum/ConsumptionActivePower":

# haus = scale_metric(singleValue['value'], singleValue.get('unit'), 'W')

Please remove, GitHub is already tracking changes

LKuemmel · 2023-11-21T12:04:02Z

packages/modules/devices/fems/counter.py

+            # Grid total energy sums
+            # response = session.get(
+            #     'http://'+self.ip_address+':8084/rest/channel/_sum/Grid.+ActiveEnergy',
+            #     timeout=1).json()
+
+            # for singleValue in response:
+            #     address = singleValue['address']
+            #     if (address == '_sum/GridBuyActiveEnergy'):
+            #         imported = scale_metric(singleValue['value'], singleValue.get('unit'), 'Wh')
+            #     elif (address == '_sum/GridSellActiveEnergy'):
+            #         exported = scale_metric(singleValue['value'], singleValue.get('unit'), 'Wh')


Suggested change

# Grid total energy sums

# response = session.get(

# 'http://'+self.ip_address+':8084/rest/channel/_sum/Grid.+ActiveEnergy',

# timeout=1).json()

# for singleValue in response:

# address = singleValue['address']

# if (address == '_sum/GridBuyActiveEnergy'):

# imported = scale_metric(singleValue['value'], singleValue.get('unit'), 'Wh')

# elif (address == '_sum/GridSellActiveEnergy'):

# exported = scale_metric(singleValue['value'], singleValue.get('unit'), 'Wh')

LKuemmel · 2023-11-21T12:04:29Z

packages/modules/devices/fems/inverter.py

+
+        # power = scale_metric(response["value"], response.get("unit"), 'W') * -1
+
+        # response = session.get(
+        #     'http://'+self.ip_address+':8084/rest/channel/_sum/ProductionActiveEnergy',
+        #     timeout=2).json()
+        # exported = scale_metric(response["value"], response.get("unit"), 'Wh')


Suggested change

# power = scale_metric(response["value"], response.get("unit"), 'W') * -1

# response = session.get(

# 'http://'+self.ip_address+':8084/rest/channel/_sum/ProductionActiveEnergy',

# timeout=2).json()

# exported = scale_metric(response["value"], response.get("unit"), 'Wh')

sthhs · 2023-11-21T16:51:32Z

Changes completed

LKuemmel

Bitte noch die Flake8-Änderungen vornehmen, damit der Test durchläuft.

sthhs · 2023-12-01T15:23:53Z

Danke für den Hinweis, die Korrekturen sind gemacht.

LKuemmel

Der Test muss noch angepasst werden. Hast Du eine Beispiel-Antwort für die neue Abfrage?

sthhs · 2023-12-01T16:04:25Z

Die Antworten von meinem FEMS stehen jeweils unter der Abfrage:

inverter.py:
http://'+self.ip_address+':8084/rest/channel/_sum/(ProductionActivePower|ProductionActiveEnergy)

[ { "address": "_sum/ProductionActiveEnergy", "type": "LONG", "accessMode": "RO", "text": "", "unit": "Wh_Σ", "value": 18988374 }, { "address": "_sum/ProductionActivePower", "type": "INTEGER", "accessMode": "RO", "text": "Total production; always positive", "unit": "W", "value": -9 } ]

counter.py:
'http://' + self.ip_address +':8084/rest/channel/(meter0|_sum)/' + '(ActivePower.*|VoltageL.|Frequency|Grid.+ActiveEnergy)'

[ { "address": "meter0/ActivePowerL3", "type": "INTEGER", "accessMode": "RO", "text": "", "unit": "W", "value": 6141 }, { "address": "meter0/ActivePower", "type": "INTEGER", "accessMode": "RO", "text": "", "unit": "W", "value": 17189 }, { "address": "meter0/ActivePowerL1", "type": "INTEGER", "accessMode": "RO", "text": "", "unit": "W", "value": 5575 }, { "address": "meter0/ActivePowerL2", "type": "INTEGER", "accessMode": "RO", "text": "", "unit": "W", "value": 5473 }, { "address": "meter0/VoltageL1", "type": "INTEGER", "accessMode": "RO", "text": "", "unit": "mV", "value": 229500 }, { "address": "meter0/VoltageL2", "type": "INTEGER", "accessMode": "RO", "text": "", "unit": "mV", "value": 229300 }, { "address": "meter0/VoltageL3", "type": "INTEGER", "accessMode": "RO", "text": "", "unit": "mV", "value": 229300 }, { "address": "meter0/Frequency", "type": "INTEGER", "accessMode": "RO", "text": "", "unit": "mHz", "value": 50000 }, { "address": "_sum/GridSellActiveEnergy", "type": "LONG", "accessMode": "RO", "text": "", "unit": "Wh_Σ", "value": 11752059 }, { "address": "_sum/GridBuyActiveEnergy", "type": "LONG", "accessMode": "RO", "text": "", "unit": "Wh_Σ", "value": 1088853 } ]
bat.py:
"http://" + self.ip_address + ":8084/rest/channel/(ess0|_sum)/" + "(Soc|DcChargeEnergy|DcDischargeEnergy|GridActivePower|ProductionActivePower|ConsumptionActivePower)"

[ { "address": "_sum/ConsumptionActivePower", "type": "INTEGER", "accessMode": "RO", "text": "", "unit": "W", "value": 17183 }, { "address": "_sum/ProductionActivePower", "type": "INTEGER", "accessMode": "RO", "text": "Total production; always positive", "unit": "W", "value": -9 }, { "address": "_sum/GridActivePower", "type": "INTEGER", "accessMode": "RO", "text": "Grid exchange power. Negative values for sell-to-grid; positive for buy-from-grid", "unit": "W", "value": 17164 }, { "address": "ess0/Soc", "type": "INTEGER", "accessMode": "RO", "text": "", "unit": "%", "value": 6 }, { "address": "ess0/DcChargeEnergy", "type": "LONG", "accessMode": "RO", "text": "", "unit": "Wh_Σ", "value": 3013394 }, { "address": "ess0/DcDischargeEnergy", "type": "LONG", "accessMode": "RO", "text": "", "unit": "Wh_Σ", "value": 3202108 } ]

LKuemmel · 2024-01-02T15:27:15Z

Sorry, ich habe mich lange nicht gemeldet. Ich habe die Tests erstellt bzw angepasst und war so frei, sie in Deinen Branch zu pushen. Danke für deine Beispieldaten!

dj3mu · 2024-01-11T20:14:28Z

packages/modules/devices/fems/bat.py

@@ -20,7 +20,8 @@ def update(self, session: Session) -> None:
        else:
            data = "ess2"
        response = session.get(
-            "http://" + self.ip_address + ":8084/rest/channel/"+data+"/(Soc|DcChargeEnergy|DcDischargeEnergy)",
+            "http://" + self.ip_address + ":8084/rest/channel/(" + data + "|_sum)/" +


Sorry, but this change breaks queries on older FEMS versions.

At least on my FEMS v 2022.12.5 a query with Regex in any "segment" other than the last one is NOT supported. FEMS just returns:

requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://<IP>:8084/rest/channel/(meter0%7C_sum)/(ActivePower.*%7CVoltageL.%7CFrequency%7CGrid.+ActiveEnergy)

I also see the optimization done here. But I might not be the only one with older version. Actually I have no control on if and when Fenecon updates my FEMS.

So I have no other idea but reverting this!

At least older FEMS seem not to support Regex in any path fragment other than the very last one.

Abfragen zusammengefasst

fde782f

LKuemmel requested changes Nov 21, 2023

View reviewed changes

LKuemmel added this to the 2.1.3 milestone Nov 21, 2023

Comments with old version removed

be57015

sthhs requested a review from LKuemmel November 21, 2023 16:51

LKuemmel approved these changes Nov 29, 2023

View reviewed changes

LKuemmel requested changes Dec 1, 2023

View reviewed changes

Flake8 Korrekturen

1bbea38

sthhs requested a review from LKuemmel December 1, 2023 15:24

LKuemmel requested changes Dec 1, 2023

View reviewed changes

add inverter and bat test, fix counter test

7648b3a

LKuemmel approved these changes Jan 2, 2024

View reviewed changes

LKuemmel merged commit 3736fc0 into openWB:master Jan 2, 2024
1 check passed

dj3mu reviewed Jan 11, 2024

View reviewed changes

dj3mu mentioned this pull request Jan 11, 2024

FEMS querying broken at least on FEMS <= 2022.12.5 #1347

Closed

dj3mu added a commit to YourCharge/openWBcore that referenced this pull request Jan 11, 2024

Partially revert PR openWB#1262

0131707

At least older FEMS seem not to support Regex in any path fragment other than the very last one.

dj3mu mentioned this pull request Jan 11, 2024

Partially revert PR #1262 #1348

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FEMS Requests combined to avoid timeouts #1262

FEMS Requests combined to avoid timeouts #1262

sthhs commented Nov 18, 2023

sthhs commented Nov 19, 2023 •

edited

Loading

LKuemmel Nov 21, 2023

LKuemmel Nov 21, 2023

LKuemmel Nov 21, 2023

sthhs commented Nov 21, 2023

LKuemmel left a comment

sthhs commented Dec 1, 2023

LKuemmel left a comment

sthhs commented Dec 1, 2023 •

edited

Loading

LKuemmel commented Jan 2, 2024

dj3mu Jan 11, 2024

FEMS Requests combined to avoid timeouts #1262

FEMS Requests combined to avoid timeouts #1262

Conversation

sthhs commented Nov 18, 2023

sthhs commented Nov 19, 2023 • edited Loading

LKuemmel Nov 21, 2023

Choose a reason for hiding this comment

LKuemmel Nov 21, 2023

Choose a reason for hiding this comment

LKuemmel Nov 21, 2023

Choose a reason for hiding this comment

sthhs commented Nov 21, 2023

LKuemmel left a comment

Choose a reason for hiding this comment

sthhs commented Dec 1, 2023

LKuemmel left a comment

Choose a reason for hiding this comment

sthhs commented Dec 1, 2023 • edited Loading

LKuemmel commented Jan 2, 2024

dj3mu Jan 11, 2024

Choose a reason for hiding this comment

sthhs commented Nov 19, 2023 •

edited

Loading

sthhs commented Dec 1, 2023 •

edited

Loading