No data returned causes the dashes

[webmake]

Describe the bug
Sometimes Foxess decides to return an empty data, so we can see no data at that time (image below). Not sure how to fix that getRaw would retry to fetch data again when data is empty.

To Reproduce
It is easier to see when the refresh time (timedelta) set to 1min

Expected behavior
To see a continuous line without breaks

Screenshots

Debug logs:

{"errno":0,"result":[{"variable":"ambientTemperation","unit":"℃","name":"AmbientTemperature","data":[]},{"variable":"batChargePower","unit":"kW","name":"Charge Power","data":[]},{"variable":"batCurrent","unit":"A","name":"BatCurrent","data":[]},{"variable":"batDischargePower","unit":"kW","name":"Discharge Power","data":[]},{"variable":"batTemperature","unit":"℃","name":"batTemperature","data":[]},{"variable":"batVolt","unit":"V","name":"BatVolt","data":[]},{"variable":"boostTemperation","unit":"℃","name":"BoostTemperature","data":[]},{"variable":"chargeTemperature","unit":"℃","name":"ChargeTemperature","data":[]},{"variable":"dspTemperature","unit":"℃","name":"DSPTemperature","data":[]},{"variable":"epsCurrentR","unit":"A","name":"EPS-RCurrent","data":[]},{"variable":"epsCurrentS","unit":"A","name":"EPS-SCurrent","data":[]},{"variable":"epsCurrentT","unit":"A","name":"EPS-TCurrent","data":[]},{"variable":"epsPower","unit":"kW","name":"EPSPower","data":[]},{"variable":"epsPowerR","unit":"kW","name":"EPS-RPower","data":[]},{"variable":"epsPowerS","unit":"kW","name":"EPS-SPower","data":[]},{"variable":"epsPowerT","unit":"kW","name":"EPS-TPower","data":[]},{"variable":"epsVoltR","unit":"V","name":"EPS-RVolt","data":[]},{"variable":"epsVoltS","unit":"V","name":"EPS-SVolt","data":[]},{"variable":"epsVoltT","unit":"V","name":"EPS-TVolt","data":[]},{"variable":"feedinPower","unit":"kW","name":"Feed-in Power","data":[]},{"variable":"generationPower","unit":"kW","name":"Output Power","data":[]},{"variable":"gridConsumptionPower","unit":"kW","name":"GridConsumption Power","data":[]},{"variable":"invBatCurrent","unit":"A","name":"InvBatCurrent","data":[]},{"variable":"invBatPower","unit":"kW","name":"invBatPower","data":[]},{"variable":"invBatVolt","unit":"V","name":"InvBatVolt","data":[]},{"variable":"invTemperation","unit":"℃","name":"InvTemperation","data":[]},{"variable":"loadsPower","unit":"kW","name":"Load Power","data":[]},{"variable":"loadsPowerR","unit":"kW","name":"LoadsRPower","data":[]},{"variable":"loadsPowerS","unit":"kW","name":"LoadsSPower","data":[]},{"variable":"loadsPowerT","unit":"kW","name":"LoadsTPower","data":[]},{"variable":"meterPower","unit":"kW","name":"MeterPower","data":[]},{"variable":"meterPower2","unit":"kW","name":"Meter2Power","data":[]},{"variable":"meterPowerR","unit":"kW","name":"MeterRPower","data":[]},{"variable":"meterPowerS","unit":"kW","name":"MeterSPower","data":[]},{"variable":"meterPowerT","unit":"kW","name":"MeterTPower","data":[]},{"variable":"PowerFactor","unit":"","name":"PowerFactor","data":[]},{"variable":"pv1Current","unit":"A","name":"PV1Current","data":[]},{"variable":"pv1Power","unit":"kW","name":"PV1Power","data":[]},{"variable":"pv1Volt","unit":"V","name":"PV1Volt","data":[]},{"variable":"pv2Current","unit":"A","name":"PV2Current","data":[]},{"variable":"pv2Power","unit":"kW","name":"PV2Power","data":[]},{"variable":"pv2Volt","unit":"V","name":"PV2Volt","data":[]},{"variable":"pv3Current","unit":"A","name":"PV3Current","data":[]},{"variable":"pv3Power","unit":"kW","name":"PV3Power","data":[]},{"variable":"pv3Volt","unit":"V","name":"PV3Volt","data":[]},{"variable":"pv4Current","unit":"A","name":"PV4Current","data":[]},{"variable":"pv4Power","unit":"kW","name":"PV4Power","data":[]},{"variable":"pv4Volt","unit":"V","name":"PV4Volt","data":[]},{"variable":"pvPower","unit":"kW","name":"PVPower","data":[]},{"variable":"RCurrent","unit":"A","name":"RCurrent","data":[]},{"variable":"ReactivePower","unit":"kVar","name":"ReactivePower","data":[]},{"variable":"RFreq","unit":"Hz","name":"RFreq","data":[]},{"variable":"RPower","unit":"kW","name":"RPower","data":[]},{"variable":"RVolt","unit":"V","name":"RVolt","data":[]},{"variable":"SCurrent","unit":"A","name":"SCurrent","data":[]},{"variable":"SFreq","unit":"Hz","name":"SFreq","data":[]},{"variable":"SoC","unit":"%","name":"SoC","data":[]},{"variable":"SPower","unit":"kW","name":"SPower","data":[]},{"variable":"SVolt","unit":"V","name":"SVolt","data":[]},{"variable":"TCurrent","unit":"A","name":"TCurrent","data":[]},{"variable":"TFreq","unit":"Hz","name":"TFreq","data":[]},{"variable":"TPower","unit":"kW","name":"TPower","data":[]},{"variable":"TVolt","unit":"V","name":"TVolt","data":[]}]}

[muhkuh666]

Will there be a fix? I expierence random no data returns also

[jfzeidler78]

I see a similar issue quite often on all signals. I'm not sure but it seems like this happens when 'raw': {} occurs instead of having content inside.
My problem is, that it happens almost half of the datasets every day, so calculating values based on inverter data gives incorrect data.

just an idea for a temporary workaround (best solution is of course to get the missing data): fill in the missing data with the last valid value instead of keeping it empty.

Example with the issue:
2023-05-18 09:31:13.059 DEBUG (MainThread) [custom_components.foxess.sensor] {'report': {'feedin': 0, 'generation': 1.7999999999999545, 'gridConsumption': 0.20000000000000107, 'chargeEnergyToTal': 2.8999999999999986, 'dischargeEnergyToTal': 2.6000000000000014, 'loads': 2.0999999999999943}, 'reportDailyGeneration': {'index': 18, 'value': 2.200000000000017}, 'raw': {}, 'online': True, 'addressbook': {'errno': 0, 'result': {REMOVED_PERSONAL_DATA 'feedinDate': '2023-05-08 15:35:28 CEST+0200', 'hardwareVersion': '', 'softVersion': {'master': '1.74', 'slave': '1.03', 'manager': '1.62', 'afci': ''}, 'protocolVersion': 'D1.15.01'}}}

Example without issue:
2023-05-18 09:41:15.038 DEBUG (MainThread) [custom_components.foxess.sensor] {'report': {'feedin': 0, 'generation': 1.7999999999999545, 'gridConsumption': 0.20000000000000107, 'chargeEnergyToTal': 3.200000000000003, 'dischargeEnergyToTal': 2.6000000000000014, 'loads': 2.0999999999999943}, 'reportDailyGeneration': {'index': 18, 'value': 2.200000000000017}, 'raw': {'ambientTemperation': 52.3, 'batChargePower': 4.251, 'batCurrent': 14.2, 'batDischargePower': 0, 'batTemperature': 27.9, 'batVolt': 308.8, 'boostTemperation': 0, 'chargeEnergyToTal': 48.5, 'chargeTemperature': 0, 'dischargeEnergyToTal': 62.1, 'dspTemperature': 0, 'epsCurrentR': 0.4, 'epsCurrentS': 0.4, 'epsCurrentT': 0.3, 'epsPower': -0.019000000000000003, 'epsPowerR': -0.005, 'epsPowerS': -0.017, 'epsPowerT': 0.003, 'epsVoltR': 239, 'epsVoltS': 238.9, 'epsVoltT': 240.7, 'feedin': 177.6, 'feedin2': 0, 'feedinPower': 0, 'generation': 193.3, 'generationPower': -1.271, 'gridConsumption': 9.4, 'gridConsumption2': 0, 'gridConsumptionPower': 0.006999999999999992, 'input': 208.8, 'invBatCurrent': -13.6, 'invBatPower': -4.251, 'invBatVolt': 308.3, 'invTemperation': 42.2, 'loads': 63.3, 'loadsPower': -1.266, 'loadsPowerR': -0.315, 'loadsPowerS': -0.465, 'loadsPowerT': -0.486, 'meterPower': 0.006999999999999992, 'meterPower2': 1.5890000000000002, 'meterPowerR': 0.129, 'meterPowerS': -0.037, 'meterPowerT': -0.085, 'PowerFactor': 0, 'pv1Current': 3.4, 'pv1Power': 1.652, 'pv1Volt': 485.8, 'pv2Current': 3.3, 'pv2Power': 1.478, 'pv2Volt': 448, 'pv3Current': 0, 'pv3Power': 0, 'pv3Volt': 0, 'pv4Current': 0, 'pv4Power': 0, 'pv4Volt': 0, 'pvPower': 3.13, 'RCurrent': 2, 'ReactivePower': 0, 'RFreq': 49.99, 'RPower': -0.444, 'RVolt': 238.9, 'SCurrent': 1.9, 'SFreq': 0, 'SoC': 96, 'SPower': -0.427, 'SVolt': 238.7, 'TCurrent': 1.8, 'TFreq': 0, 'TPower': -0.4, 'TVolt': 238.5}, 'online': True, 'addressbook': {'errno': 0, 'result': {REMOVED_PERSONAL_DATA 'feedinDate': '2023-05-08 15:35:28 CEST+0200', 'hardwareVersion': '', 'softVersion': {'master': '1.74', 'slave': '1.03', 'manager': '1.62', 'afci': ''}, 'protocolVersion': 'D1.15.01'}}}

[webmake]

I found in the code:
# If data is a non-empty list, pop the last value off the list, otherwise return the previously found value
maybe there was an intention to keep previous value, which now doesn't work, since data is returned but raw is empty

[jfzeidler78]

I found in the code: # If data is a non-empty list, pop the last value off the list, otherwise return the previously found value maybe there was an intention to keep previous value, which now doesn't work, since data is returned but raw is empty

I assume the same. Unfortunally I cannot write phyton :-(

could it have an effect on this issue to replace
if item["data"]: allData['raw'][variableName] = item["data"].pop().get("value",None)

with something like this?
if item["data"]: if item["data.rawData"]: allData['raw'][variableName] = item["data"].pop().get("value",None)

(probably, the syntax isn't correct)

Better would of course having a solution for the root cause of this missing values. But just to show you how bad the result is at my place:

[webmake]

The root cause is unsolvable, here is what I got from Foxess:

Hello,

We usually don’t give API for end customers. But difficult to say what’s causing those empty lists.

Kind regards,

[mstevanak]

I'm having the same behavior of FoxESS integration. This way it is unsuable because it produces wrong values when calculating kWh from kW variables.