Set Limit over MQTT stop responding

[DejanBukovec]

What happened?

Hi,
I have 3 HM-800 inverters.
Im make in Home Assistant automation which every 15 seconds calculate how many power solar system need produce to cover current power usage(Zero export).
DTU Settings:
DTU Poll Interval: 5 seconds
MQTT Publish Interval: 5 seconds

When I enable automation for 1 inverter everything work ok. And all inverters receive/update every cca. 15-30 seconds.
When I enable automation also for second inverter then problem start. It set limit on second inverter but newer receive any data/update from it. Im wait around 200-300 seconds and DTU didn't receive any updates from inverters...

To Reproduce Bug

Enable automation in Home Assistant which set Non persistent power limit over MQTT for 3 HM-800 inverters.

Expected Behavior

Setting Limit over MQTT do not stop receiving data from inverters...

Maybe solution is to add commands received by MQTT into cache and update inverters on next pool timeout to avoid loosing inverter data? Or add this option inside inverter options if sending commands need to be imediate or at pool interval....

Install Method

Pre-Compiled binary from GitHub

What git-hash/version of OpenDTU?

v23.6.21

Relevant log/trace output

16:30:51.319 > All missing
16:30:51.319 > Nothing received, resend whole request
16:30:51.319 > TX RealTimeRunData Channel: 23 --> 15 81 81 43 60 80 18 28 40 80 0B 00 64 99 A1 1B 00 00 00 00 00 00 00 00 AC EA 4C 
16:30:51.863 > RX Period End
16:30:51.863 > All missing
16:30:51.863 > Nothing received, resend whole request
16:30:51.863 > TX RealTimeRunData Channel: 40 --> 15 81 81 43 60 80 18 28 40 80 0B 00 64 99 A1 1B 00 00 00 00 00 00 00 00 AC EA 4C 
16:30:52.407 > RX Period End
16:30:52.407 > All missing
16:30:52.407 > Nothing received, resend whole request
16:30:52.407 > TX RealTimeRunData Channel: 61 --> 15 81 81 43 60 80 18 28 40 80 0B 00 64 99 A1 1B 00 00 00 00 00 00 00 00 AC EA 4C 
16:30:52.463 > Interrupt received
16:30:52.523 > RX Channel: 23 --> 95 81 81 43 60 81 81 43 60 01 00 01 01 81 02 AC 0A 47 01 81 02 AD 0A 4F 00 00 9C | -80 dBm
16:30:52.628 > Interrupt received
16:30:52.932 > RX Channel: 23 --> 95 81 81 43 60 81 81 43 60 02 08 59 00 00 08 4C 06 F2 06 F7 09 04 13 8A 13 9A 9A | -80 dBm
16:30:52.984 > Interrupt received
16:30:53.036 > RX Channel: 75 --> 95 81 81 43 60 81 81 43 60 83 00 00 00 D9 03 E8 02 11 00 10 02 C9 EC | -80 dBm
16:30:53.085 > RX Period End
16:30:53.085 > Success
16:30:55.723 > Fetch inverter: 1141818XXXXX
16:30:55.723 > Request SystemConfigPara
16:30:56.007 > TX RealTimeRunData Channel: 75 --> 15 81 80 96 46 80 18 28 40 80 0B 00 64 99 A1 20 00 00 00 00 00 00 00 00 9D CD 93 
16:30:56.414 > Interrupt received
16:30:56.473 > RX Channel: 61 --> 95 81 80 96 46 81 80 96 46 01 00 01 01 7D 02 A2 0A 03 01 7D 02 9D 09 F7 00 00 5D | -80 dBm
16:30:56.620 > Interrupt received
16:30:56.681 > RX Channel: 40 --> 95 81 80 96 46 81 80 96 46 02 0C 1F 00 00 0C 2E 0A A6 0A AE 09 0B 13 89 13 05 20 | -80 dBm
16:30:56.924 > Interrupt received
16:30:56.981 > RX Channel: 23 --> 95 81 80 96 46 81 80 96 46 83 00 00 00 D2 03 E8 02 34 00 10 CA 16 D5 | -80 dBm
16:30:57.109 > RX Period End
16:30:57.109 > Success
16:30:57.170 > TX SystemConfigPara Channel: 3 --> 15 81 80 96 46 80 18 28 40 80 05 00 64 99 A1 20 00 00 00 00 00 00 00 00 53 C2 5C 
16:30:57.221 > RX Period End
16:30:57.221 > All missing
16:30:57.221 > Nothing received, resend whole request
16:30:57.221 > TX SystemConfigPara Channel: 23 --> 15 81 80 96 46 80 18 28 40 80 05 00 64 99 A1 20 00 00 00 00 00 00 00 00 53 C2 5C 
16:30:57.539 > RX Period End
16:30:57.539 > All missing
16:30:57.539 > Nothing received, resend whole request
16:30:57.539 > TX SystemConfigPara Channel: 40 --> 15 81 80 96 46 80 18 28 40 80 05 00 64 99 A1 20 00 00 00 00 00 00 00 00 53 C2 5C 
16:30:57.849 > RX Period End
16:30:57.849 > All missing
16:30:57.849 > Nothing received, resend whole request
16:30:57.849 > TX SystemConfigPara Channel: 61 --> 15 81 80 96 46 80 18 28 40 80 05 00 64 99 A1 20 00 00 00 00 00 00 00 00 53 C2 5C 
16:30:57.902 > RX Period End
16:30:57.902 > All missing
16:30:57.902 > Nothing received, resend whole request
16:30:57.902 > TX SystemConfigPara Channel: 75 --> 15 81 80 96 46 80 18 28 40 80 05 00 64 99 A1 20 00 00 00 00 00 00 00 00 53 C2 5C 
16:30:57.951 > Interrupt received
16:30:58.157 > RX Channel: 23 --> 95 81 80 96 46 81 80 96 46 81 00 01 03 E8 00 00 03 E8 FF FF FF FF 01 68 7D F8 F9 | -80 dBm
16:30:58.209 > RX Period End
16:30:58.209 > Success
16:31:00.212 > Received MQTT message on topic: solar/1141818XXXXX/cmd/limit_nonpersistent_absolute
16:31:00.212 > Limit Non-Persistent: 800 W
16:31:00.269 > TX ActivePowerControl Channel: 3 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30 
16:31:02.299 > RX Period End
16:31:02.299 > All missing
16:31:02.299 > Nothing received, resend whole request
16:31:02.299 > TX ActivePowerControl Channel: 23 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30 
16:31:04.337 > RX Period End
16:31:04.337 > All missing
16:31:04.337 > Nothing received, resend whole request
16:31:04.337 > TX ActivePowerControl Channel: 40 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30 
16:31:06.378 > RX Period End
16:31:06.378 > All missing
16:31:06.378 > Nothing received, resend whole request
16:31:06.378 > TX ActivePowerControl Channel: 61 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30 
16:31:06.436 > Interrupt received
16:31:06.756 > RX Channel: 3 --> D1 81 81 43 60 81 81 43 60 81 00 00 0B 00 14 07 48 | -80 dBm
16:31:08.544 > RX Period End
16:31:08.544 > Success
16:31:08.544 > Fetch inverter: 1141818XXXXX
16:31:08.600 > TX RealTimeRunData Channel: 75 --> 15 81 81 20 14 80 18 28 40 80 0B 00 64 99 A1 2C 00 00 00 00 00 00 00 00 9D 98 2F 
16:31:08.658 > Interrupt received
16:31:08.909 > RX Channel: 61 --> 95 81 81 20 14 81 81 20 14 01 00 01 01 7B 01 78 05 91 01 75 02 95 09 A3 00 00 4B | -80 dBm
16:31:09.183 > Interrupt received
16:31:09.242 > RX Channel: 23 --> 95 81 81 20 14 81 81 20 14 02 0A B1 00 00 0E 7B 02 18 02 23 09 05 13 89 0E 79 83 | -80 dBm
16:31:09.519 > Interrupt received
16:31:09.830 > RX Channel: 40 --> 95 81 81 20 14 81 81 20 14 83 00 00 00 A1 03 E8 01 F0 00 0E 3F 63 FF | -80 dBm
16:31:09.883 > RX Period End
16:31:09.883 > Success
16:31:13.391 > Fetch inverter: 1141818XXXXX
16:31:13.512 > TX RealTimeRunData Channel: 3 --> 15 81 81 43 60 80 18 28 40 80 0B 00 64 99 A1 31 00 00 00 00 00 00 00 00 CD 0D E0 
16:31:13.984 > RX Period End
16:31:13.984 > All missing
16:31:13.984 > Nothing received, resend whole request
16:31:13.984 > TX RealTimeRunData Channel: 23 --> 15 81 81 43 60 80 18 28 40 80 0B 00 64 99 A1 31 00 00 00 00 00 00 00 00 CD 0D E0 
16:31:14.529 > RX Period End
16:31:14.529 > All missing
16:31:14.529 > Nothing received, resend whole request
16:31:14.529 > TX RealTimeRunData Channel: 40 --> 15 81 81 43 60 80 18 28 40 80 0B 00 64 99 A1 31 00 00 00 00 00 00 00 00 CD 0D E0 
16:31:15.071 > RX Period End
16:31:15.071 > All missing
16:31:15.071 > Nothing received, resend whole request
16:31:15.071 > TX RealTimeRunData Channel: 61 --> 15 81 81 43 60 80 18 28 40 80 0B 00 64 99 A1 31 00 00 00 00 00 00 00 00 CD 0D E0 
16:31:15.225 > Received MQTT message on topic: solar/1141818XXXXX/cmd/limit_nonpersistent_absolute
16:31:15.225 > Limit Non-Persistent: 800 W
16:31:15.282 > Interrupt received
16:31:15.356 > RX Channel: 3 --> 95 81 81 43 60 81 81 43 60 02 08 5B 00 00 08 4E 06 F4 06 F9 09 07 13 89 13 8A 82 | -80 dBm
16:31:15.411 > Interrupt received
16:31:15.665 > RX Channel: 3 --> 95 81 81 43 60 81 81 43 60 83 00 04 00 D9 03 E8 02 10 00 10 1B A9 90 | -80 dBm
16:31:15.975 > RX Period End
16:31:15.975 > Middle missing
16:31:15.975 > Request retransmit: 1
16:31:15.975 > TX RequestFrame Channel: 75 --> 15 81 81 43 60 80 18 28 40 81 47 
16:31:16.264 > RX Period End
16:31:16.264 > Middle missing
16:31:16.264 > Request retransmit: 1
16:31:16.264 > TX RequestFrame Channel: 3 --> 15 81 81 43 60 80 18 28 40 81 47 
16:31:16.331 > RX Period End
16:31:16.331 > Middle missing
16:31:16.331 > Request retransmit: 1
16:31:16.331 > TX RequestFrame Channel: 23 --> 15 81 81 43 60 80 18 28 40 81 47 
16:31:16.392 > RX Period End
16:31:16.392 > Middle missing
16:31:16.392 > Request retransmit: 1
16:31:16.392 > TX RequestFrame Channel: 40 --> 15 81 81 43 60 80 18 28 40 81 47 
16:31:16.447 > RX Period End
16:31:16.448 > Middle missing
16:31:16.448 > Request retransmit: 1
16:31:16.448 > TX RequestFrame Channel: 61 --> 15 81 81 43 60 80 18 28 40 81 47 
16:31:16.567 > RX Period End
16:31:16.567 > Middle missing
16:31:16.567 > Retransmit timeout
16:31:16.629 > TX ActivePowerControl Channel: 75 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30 
16:31:16.689 > Interrupt received
16:31:16.893 > RX Channel: 23 --> D1 81 81 43 60 81 81 43 60 81 00 00 0B 00 14 07 48 | -80 dBm
16:31:18.625 > RX Period End
16:31:18.625 > Success
16:31:18.625 > Fetch inverter: 1141818XXXXX
16:31:18.736 > TX RealTimeRunData Channel: 3 --> 15 81 80 96 46 80 18 28 40 80 0B 00 64 99 A1 36 00 00 00 00 00 00 00 00 FD 2B 03 
16:31:19.212 > RX Period End
16:31:19.212 > All missing
16:31:19.212 > Nothing received, resend whole request
16:31:19.212 > TX RealTimeRunData Channel: 23 --> 15 81 80 96 46 80 18 28 40 80 0B 00 64 99 A1 36 00 00 00 00 00 00 00 00 FD 2B 03 
16:31:19.760 > RX Period End
16:31:19.760 > All missing
16:31:19.760 > Nothing received, resend whole request
16:31:19.760 > TX RealTimeRunData Channel: 40 --> 15 81 80 96 46 80 18 28 40 80 0B 00 64 99 A1 36 00 00 00 00 00 00 00 00 FD 2B 03 
16:31:20.304 > RX Period End
16:31:20.304 > All missing
16:31:20.304 > Nothing received, resend whole request
16:31:20.304 > TX RealTimeRunData Channel: 61 --> 15 81 80 96 46 80 18 28 40 80 0B 00 64 99 A1 36 00 00 00 00 00 00 00 00 FD 2B 03 
16:31:20.352 > Interrupt received
16:31:20.579 > RX Channel: 40 --> 95 81 80 96 46 81 80 96 46 01 00 01 01 7D 02 9F 09 F8 01 7D 02 9B 09 EB 00 00 82 | -80 dBm
16:31:20.888 > Interrupt received
16:31:21.193 > RX Channel: 23 --> 95 81 80 96 46 81 80 96 46 02 0C 21 00 00 0C 30 0A A8 0A B0 09 11 13 89 12 EF E1 | -80 dBm
16:31:21.241 > Interrupt received
16:31:21.315 > RX Channel: 3 --> 95 81 80 96 46 81 80 96 46 83 00 04 00 D1 03 E8 02 33 00 10 B3 8D 37 | -80 dBm
16:31:21.504 > RX Period End
16:31:21.504 > Success
16:31:23.614 > Fetch inverter: 1141818XXXXX
16:31:23.668 > TX RealTimeRunData Channel: 75 --> 15 81 81 20 14 80 18 28 40 80 0B 00 64 99 A1 3B 00 00 00 00 00 00 00 00 6D 73 23 
16:31:23.755 > Interrupt received
16:31:23.977 > RX Channel: 61 --> 95 81 81 20 14 81 81 20 14 01 00 01 01 7B 01 78 05 90 01 75 02 94 09 A1 00 00 49 | -80 dBm
16:31:24.273 > Interrupt received
16:31:24.577 > RX Channel: 40 --> 95 81 81 20 14 81 81 20 14 02 0A B2 00 00 0E 7C 02 19 02 24 09 00 13 89 0E 76 8B | -80 dBm
16:31:24.882 > Interrupt received
16:31:25.192 > RX Channel: 40 --> 95 81 81 20 14 81 81 20 14 83 00 00 00 A1 03 E8 01 F0 00 0E 1A AD 14 | -80 dBm
16:31:25.258 > RX Period End
16:31:25.258 > Success
16:31:28.616 > Fetch inverter: 1141818XXXXX
16:31:28.877 > TX RealTimeRunData Channel: 3 --> 15 81 81 43 60 80 18 28 40 80 0B 00 64 99 A1 40 00 00 00 00 00 00 00 00 9F 65 AB 
16:31:29.285 > RX Period End
16:31:29.285 > All missing
16:31:29.285 > Nothing received, resend whole request
16:31:29.285 > TX RealTimeRunData Channel: 23 --> 15 81 81 43 60 80 18 28 40 80 0B 00 64 99 A1 40 00 00 00 00 00 00 00 00 9F 65 AB 
16:31:29.757 > RX Period End
16:31:29.757 > All missing
16:31:29.757 > Nothing received, resend whole request
16:31:29.757 > TX RealTimeRunData Channel: 40 --> 15 81 81 43 60 80 18 28 40 80 0B 00 64 99 A1 40 00 00 00 00 00 00 00 00 9F 65 AB 
16:31:30.100 > Received MQTT message on topic: solar/1141818XXXXX/cmd/limit_nonpersistent_absolute
16:31:30.100 > Limit Non-Persistent: 800 W
16:31:30.302 > RX Period End
16:31:30.302 > All missing
16:31:30.302 > Nothing received, resend whole request
16:31:30.302 > TX RealTimeRunData Channel: 61 --> 15 81 81 43 60 80 18 28 40 80 0B 00 64 99 A1 40 00 00 00 00 00 00 00 00 9F 65 AB 
16:31:30.351 > Interrupt received
16:31:30.409 > RX Channel: 40 --> 95 81 81 43 60 81 81 43 60 01 00 01 01 81 02 A9 0A 3B 01 81 02 AB 0A 42 00 00 EE | -80 dBm
16:31:30.457 > Interrupt received
16:31:30.536 > RX Channel: 3 --> 95 81 81 43 60 81 81 43 60 02 08 5C 00 00 08 4F 06 F5 06 FA 09 06 13 89 13 81 8C | -80 dBm
16:31:30.837 > RX Period End
16:31:30.837 > Last missing
16:31:30.837 > Request retransmit: 3
16:31:30.837 > TX RequestFrame Channel: 75 --> 15 81 81 43 60 80 18 28 40 83 45 
16:31:30.974 > RX Period End
16:31:30.974 > Last missing
16:31:30.974 > Request retransmit: 3
16:31:30.974 > TX RequestFrame Channel: 3 --> 15 81 81 43 60 80 18 28 40 83 45 
16:31:31.333 > RX Period End
16:31:31.333 > Last missing
16:31:31.333 > Request retransmit: 3
16:31:31.333 > TX RequestFrame Channel: 23 --> 15 81 81 43 60 80 18 28 40 83 45 
16:31:31.505 > RX Period End
16:31:31.505 > Last missing
16:31:31.505 > Request retransmit: 3
16:31:31.505 > TX RequestFrame Channel: 40 --> 15 81 81 43 60 80 18 28 40 83 45 
16:31:31.563 > RX Period End
16:31:31.563 > Last missing
16:31:31.563 > Request retransmit: 3
16:31:31.563 > TX RequestFrame Channel: 61 --> 15 81 81 43 60 80 18 28 40 83 45 
16:31:31.610 > RX Period End
16:31:31.610 > Last missing
16:31:31.610 > Retransmit timeout
16:31:31.946 > TX ActivePowerControl Channel: 75 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30 
16:31:32.000 > Interrupt received
16:31:32.254 > RX Channel: 40 --> D1 81 81 43 60 81 81 43 60 81 00 00 0B 00 14 07 48 | -80 dBm
16:31:33.725 > RX Period End
16:31:33.725 > Success
16:31:33.786 > Fetch inverter: 1141818XXXXX
16:31:34.096 > TX RealTimeRunData Channel: 3 --> 15 81 80 96 46 80 18 28 40 80 0B 00 64 99 A1 46 00 00 00 00 00 00 00 00 3F 4E D4 
16:31:34.325 > RX Period End
16:31:34.325 > All missing
16:31:34.325 > Nothing received, resend whole request
16:31:34.325 > TX RealTimeRunData Channel: 23 --> 15 81 80 96 46 80 18 28 40 80 0B 00 64 99 A1 46 00 00 00 00 00 00 00 00 3F 4E D4 
16:31:34.877 > RX Period End
16:31:34.877 > All missing
16:31:34.877 > Nothing received, resend whole request
16:31:34.877 > TX RealTimeRunData Channel: 40 --> 15 81 80 96 46 80 18 28 40 80 0B 00 64 99 A1 46 00 00 00 00 00 00 00 00 3F 4E D4 
16:31:35.421 > RX Period End
16:31:35.421 > All missing
16:31:35.421 > Nothing received, resend whole request
16:31:35.421 > TX RealTimeRunData Channel: 61 --> 15 81 80 96 46 80 18 28 40 80 0B 00 64 99 A1 46 00 00 00 00 00 00 00 00 3F 4E D4 
16:31:35.474 > Interrupt received
16:31:35.631 > RX Channel: 23 --> 95 81 80 96 46 81 80 96 46 02 0C 22 00 00 0C 31 0A A9 0A B1 09 0C 13 89 12 E8 F9 | -80 dBm
16:31:36.042 > Interrupt received
16:31:36.095 > RX Channel: 3 --> 95 81 80 96 46 81 80 96 46 83 00 00 00 D1 03 E8 02 34 00 10 6A 5B 3B | -80 dBm
16:31:36.145 > RX Period End
16:31:36.145 > Middle missing
16:31:36.145 > Request retransmit: 1
16:31:36.145 > TX RequestFrame Channel: 75 --> 15 81 80 96 46 80 18 28 40 81 B5 
16:31:36.196 > Interrupt received
16:31:36.250 > RX Channel: 23 --> 95 81 80 96 46 81 80 96 46 01 00 01 01 7D 02 9E 09 F4 01 7D 02 9A 09 E8 00 00 8D | -80 dBm
16:31:36.300 > RX Period End
16:31:36.300 > Success
16:31:38.961 > Fetch inverter: 1141818XXXXX
16:31:39.012 > TX RealTimeRunData Channel: 3 --> 15 81 81 20 14 80 18 28 40 80 0B 00 64 99 A1 4B 00 00 00 00 00 00 00 00 AF 16 F4 
16:31:39.557 > RX Period End
16:31:39.557 > All missing
16:31:39.557 > Nothing received, resend whole request
16:31:39.557 > TX RealTimeRunData Channel: 23 --> 15 81 81 20 14 80 18 28 40 80 0B 00 64 99 A1 4B 00 00 00 00 00 00 00 00 AF 16 F4 
16:31:40.102 > RX Period End
16:31:40.102 > All missing
16:31:40.102 > Nothing received, resend whole request
16:31:40.102 > TX RealTimeRunData Channel: 40 --> 15 81 81 20 14 80 18 28 40 80 0B 00 64 99 A1 4B 00 00 00 00 00 00 00 00 AF 16 F4 
16:31:40.651 > RX Period End
16:31:40.651 > All missing
16:31:40.651 > Nothing received, resend whole request
16:31:40.651 > TX RealTimeRunData Channel: 61 --> 15 81 81 20 14 80 18 28 40 80 0B 00 64 99 A1 4B 00 00 00 00 00 00 00 00 AF 16 F4 
16:31:40.857 > Interrupt received
16:31:41.164 > RX Channel: 23 --> 95 81 81 20 14 81 81 20 14 01 00 01 01 7B 01 77 05 8D 01 76 02 92 09 9A 00 00 65 | -80 dBm
16:31:41.215 > Interrupt received
16:31:41.266 > RX Channel: 3 --> 95 81 81 20 14 81 81 20 14 02 0A B2 00 00 0E 7D 02 19 02 25 09 07 13 8A 0E 6D 94 | -80 dBm
16:31:41.319 > Interrupt received
16:31:41.548 > RX Channel: 3 --> 95 81 81 20 14 81 81 20 14 83 00 03 00 A0 03 E8 01 F0 00 0E 96 D0 E7 | -80 dBm
16:31:41.601 > RX Period End
16:31:41.601 > Success

Anything else?

No response

[tbnobody]

If you are updating your inverter limit so often you should increase your polling interval (which is currently 5 seconds). Other way would be to send the limit updates with more delay
Keep in mind, that the original DTU polls inverter data every 15minutes....

[DejanBukovec]

Im set right now interval to 30 seconds(DTU Poll Interval). MQTT Im leave at 5 seconds.
Now data age go to around 90 seconds until it update. Does that it try update every 90 seconds one inverter if I use 3 inverters?
Because all of 3 inverters go to around 90 seconds data age before new values are shown...
This increase to 30 seconds interval also produce me problems in HA. Now entities became unavailible for around 60 seconds... They go one by one offline and came back...

[tbnobody]

To understand the logic I have to explain some things:

• You can only send the request and receive the response for one inverter at a time
• This means if a request for e.g. statistic data is sent to the inverter the DTU has to wait until it receives is response (or a defined timeout occours) before sending the next request or command
• This means, there exists a FiFo queue for sending requests and commands
• The software takes the first entry in the queue sends it via the RF interface and waits for an answer
• If you send mqtt limits to the DTU, these command requests are immediatly inserted at the end of the queue (independet of the filling level of the queue)
• The statistic request commands are inserted into the queue every polling interval but only if the queue is EMPTY.
• That means if you send new limit commands before the software has a chance to work through all elements in the queue no new statistic commands will be inserted.

Why was this implemented in this way? Commands like limit or restart have higher priority in this way and never get lost.
And why not inserting statistic commands every poll interval? Because if someone would set the poll interval too short the queue would fill up until all the memory is consumed and the ESP would just crash.

So now it's in the response of the user not to send the limit commands too often. (which was also a feature request to leave this in the users hand)

[DejanBukovec]

Im sorry but I understand your explaination or something do not work correctly...

Lets test. I have 3 Inverters enabled.

Situation 1:
Poll Interval: 5 seconds
MQTT Publish Interval: 5 seconds
HA Automation: Disabled

So there is no Limit or other commands send to inverters.
OpenDTU WEB: Each inverters "Data Age" go to around 15 seconds(15-18 seconds) to update statistic. So this is not 5 second interval but 15 seconds.
HomeAssistant: Work ok and there is no unavailible entities.

Situation 2:
Poll Interval: 15 seconds
MQTT Publish Interval: 5 seconds
HA Automation: Disabled

So there is no Limit or other commands send to inverters.
OpenDTU WEB: Each inverters "Data Age" go to around 45 seconds(45-48 seconds) to update statistic. So this is not 5 second interval but 45 seconds.
HomeAssistant: Do not work ok there is unavailible entities for few seconds(2-4 seconds) and then came back.

Situation 3:
Poll Interval: 5 seconds
MQTT Publish Interval: 5 seconds
HA Automation: Enabled. Every 15 seconds send No Persistent limit to 1 Inverter

So there is only one Limit command every 15 seconds send to only one inverter. No other commands send to inverters.
OpenDTU WEB: Each inverters "Data Age" go to around 15-30 seconds to update statistic. So this is again not 5 second interval but 15 seconds + Send command delay.
HomeAssistant: Work ok and there is no unavailible entities.

Situation 4:
Poll Interval: 5 seconds
MQTT Publish Interval: 5 seconds
HA Automation: Enabled. Every 15 seconds send No Persistent limit to 2 Inverters

So there is only one Limit command every 15 seconds send to only two inverters. No other commands send to inverters.
OpenDTU WEB: Each inverters "Data Age" go to around 100+ seconds to update statistic. Firt need 60 seconds then slowlly increase to 90 seconds then to 110 seconds then 150 seconds. After few minutes(5-10 minutes then in 200 seconds do not receive data...
HomeAssistant: Do not work ok there is unavailible entities for most of time.

Log when DTU need more than 200 seconds to get data:

RX Period End
19:14:04.016 > All missing
19:14:04.016 > Nothing received, resend whole request
19:14:04.016 > TX ActivePowerControl Channel: 61 --> 51 81 81 20 14 80 18 28 40 81 0B 00 1F 40 00 00 60 07 27
19:14:06.055 > RX Period End
19:14:06.056 > All missing
19:14:06.056 > Nothing received, resend whole request
19:14:06.056 > TX ActivePowerControl Channel: 75 --> 51 81 81 20 14 80 18 28 40 81 0B 00 1F 40 00 00 60 07 27
19:14:06.283 > Interrupt received
19:14:06.333 > RX Channel: 23 --> D1 81 81 20 14 81 81 20 14 81 00 00 0B 00 14 07 48 | -80 dBm
19:14:08.219 > RX Period End
19:14:08.219 > Success
19:14:08.267 > TX ActivePowerControl Channel: 3 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30
19:14:10.305 > RX Period End
19:14:10.305 > All missing
19:14:10.305 > Nothing received, resend whole request
19:14:10.305 > TX ActivePowerControl Channel: 23 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30
19:14:12.345 > RX Period End
19:14:12.345 > All missing
19:14:12.345 > Nothing received, resend whole request
19:14:12.345 > TX ActivePowerControl Channel: 40 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30
19:14:14.390 > RX Period End
19:14:14.390 > All missing
19:14:14.390 > Nothing received, resend whole request
19:14:14.390 > TX ActivePowerControl Channel: 61 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30
19:14:14.887 > Received MQTT message on topic: solar/1141818XXXX1/cmd/limit_nonpersistent_absolute
19:14:14.936 > Limit Non-Persistent: 800 W
19:14:15.091 > Received MQTT message on topic: solar/1141818XXXX2/cmd/limit_nonpersistent_absolute
19:14:15.091 > Limit Non-Persistent: 800 W
19:14:16.428 > RX Period End
19:14:16.429 > All missing
19:14:16.429 > Nothing received, resend whole request
19:14:16.429 > TX ActivePowerControl Channel: 75 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30
19:14:18.621 > RX Period End
19:14:18.621 > All missing
19:14:18.621 > Nothing received, resend count exeeded
19:14:18.670 > TX ActivePowerControl Channel: 3 --> 51 81 81 20 14 80 18 28 40 81 0B 00 1F 40 00 00 60 07 27
19:14:20.712 > RX Period End
19:14:20.712 > All missing
19:14:20.712 > Nothing received, resend whole request
19:14:20.712 > TX ActivePowerControl Channel: 23 --> 51 81 81 20 14 80 18 28 40 81 0B 00 1F 40 00 00 60 07 27
19:14:22.752 > RX Period End
19:14:22.752 > All missing
19:14:22.752 > Nothing received, resend whole request
19:14:22.752 > TX ActivePowerControl Channel: 40 --> 51 81 81 20 14 80 18 28 40 81 0B 00 1F 40 00 00 60 07 27
19:14:24.793 > RX Period End
19:14:24.793 > All missing
19:14:24.793 > Nothing received, resend whole request
19:14:24.793 > TX ActivePowerControl Channel: 61 --> 51 81 81 20 14 80 18 28 40 81 0B 00 1F 40 00 00 60 07 27
19:14:26.834 > RX Period End
19:14:26.834 > All missing
19:14:26.834 > Nothing received, resend whole request
19:14:26.834 > TX ActivePowerControl Channel: 75 --> 51 81 81 20 14 80 18 28 40 81 0B 00 1F 40 00 00 60 07 27
19:14:26.885 > Interrupt received
19:14:26.936 > RX Channel: 40 --> D1 81 81 20 14 81 81 20 14 81 00 00 0B 00 14 07 48 | -80 dBm
19:14:29.016 > RX Period End
19:14:29.017 > Success
19:14:29.072 > TX ActivePowerControl Channel: 3 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30
19:14:29.946 > Received MQTT message on topic: solar/1141818XXXX1/cmd/limit_nonpersistent_absolute
19:14:29.946 > Limit Non-Persistent: 800 W
19:14:30.103 > Received MQTT message on topic: solar/1141818XXXX2/cmd/limit_nonpersistent_absolute
19:14:30.103 > Limit Non-Persistent: 800 W
19:14:31.106 > RX Period End
19:14:31.106 > All missing
19:14:31.106 > Nothing received, resend whole request
19:14:31.106 > TX ActivePowerControl Channel: 23 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30
19:14:33.148 > RX Period End
19:14:33.148 > All missing
19:14:33.148 > Nothing received, resend whole request
19:14:33.148 > TX ActivePowerControl Channel: 40 --> 51 81 81 43 60 80 18 28 40 81 0B 00 1F 40 00 00 60 07 30

System looks ok:

If there is FIFO for commands is not better idea to only keep last limit command if there are 5, 10 or more commands in row? Looks like sending limit by FIFO way every 15 seconds work ok for only one inverter, but for 2 do not work and it go to infinity loop...

[tbnobody]

So this is not 5 second interval but 15 seconds.

Makes sense with 5 inverters

• Poll inverter 1
• wait 5sec
• poll inverter 2
• wait 5sec
• poll inverter 3
• wait 5 sec
• poll inverter 1
• etc...

And this applies only if the queue is empty. if e.g. 4 seconds after polling inverter 1 a limit command comes in then the limit command is handled first before polling inverter 2. and as I see you are inserting 2 limit commands in a row. that means that both limit commands have to be sent first.
And if your connection is bad and your limit command is not successfully sent then its not removed from the queue but it will retry sending this command.

If there is FIFO for commands is not better idea to only keep last limit command if there are 5, 10 or more commands in row?

it depends... What if one command is turn off and the other one is turn on?

[DejanBukovec]

Ok now I understand more :) Then in DTU settings "Poll Interval" is not pool interval but delay between inverter pool requests :)
Pool interval will mean:
wait 5 sec
poll inverter 1
poll inverter 2
poll inverter 3
wait 5 sec
poll inverter 1
...

How long usually need set power command to be completed? In current test it looks like one request is completed in 15 seconds but two are not. DTU is around 3-4 meters away from inverters. NRF Power has been set to high now Im set it to Minimum if there will be some difference...

Is there some reason why this is not set to 1 second? Then it will imediatelly continue after set commands...

Im think about this when talking about FIFO optiimzations
For example there is queue:
set inverter 1 power 100 - This command is currently sending
set inverter 2 power 105
set inverter 3 power 102
set inverter 2 power off
set inverter 1 power 205
set inverter 3 power 102
set inverter 2 power on
set inverter 1 power 300
set inverter 2 power 450
set inverter 3 power 200

After optimization and when command "set inverter 1 power 100" will finish it will became:
set inverter 2 power off
set inverter 2 power on
set inverter 1 power 300
set inverter 2 power 450
set inverter 3 power 200

So all other comands like turn on/off or reboot will stay in queue. but if for some inverter there is already in queue some power limit command it will delete old one and enter new one in queue... Afcourse persistent and non persistent(This are currently problematic) need to be seperated...

This also can be option in inverter settings to queue only last set power command....

I will test more combinations tomorrow because right now there is no sun :)

[DejanBukovec]

Now using v23.6.26 firmware.

Situation 5:
Poll Interval: 5 seconds
MQTT Publish Interval: 5 seconds
HA Automation: Enabled. Every 30 seconds send No Persistent limit to 2 Inverters

So there is only one Limit command every 30 seconds send to two inverters. No other commands send to inverters.
OpenDTU WEB: Each inverters "Data Age" go to around 15 seconds(15-18 seconds) to update statistic.
HomeAssistant: Work ok and there is no unavailible entities.

Situation 6:
Poll Interval: 5 seconds
MQTT Publish Interval: 5 seconds
HA Automation: Enabled. Every 30 seconds send No Persistent limit to 3 Inverters

So there is only one Limit command every 30 seconds send to three inverters. No other commands send to inverters.
OpenDTU WEB: Each inverters "Data Age" go to around 15-25 seconds to update statistic.
HomeAssistant: Work ok and there is no unavailible entities.

Situation 7:
Poll Interval: 3 seconds
MQTT Publish Interval: 5 seconds
HA Automation: Enabled. Every 20 seconds send No Persistent limit to 3 Inverters

So there is only one Limit command every 30 seconds send to three inverters. No other commands send to inverters.
OpenDTU WEB: Each inverters "Data Age" go to around 9 seconds(9-12 seconds) to update statistic.
HomeAssistant: Work ok and there is no unavailible entities.

For now Situation 7 work best...

I think that current pooling is not optimized and there can be a lot of improvements...
Main problem is that if pool interval is high(10-30 seconds) at pool interval timeout it check for queue and if there are some commands then it wait again complete pool interval and not shorter one...
For example we have pool interval 15 seconds. If command need 1 second to complete then instead of waiting 1,5 second and recheck if queue is empty it wait another 15 seconds and in this time MQTT again queue one command and queue is not empty... So we fall into loop...
Better aproach will be to temporary decrease pool interval of inverter(Only not updated inverter) to 1-2 seconds if queue is not empty...

Im never program arduino but maybe it can be done by simple fix in Hoymiles.cpp where you check isQueueEmpty() one else statement to decrease _pollInterval value to minimum...

[DejanBukovec]

@tbnobody Did you think to do some check if "same" command(with different value) is already in queue that to inverter is send only last one?
Im new at this but by looking code you will probably need change "std::queue" do something else which offer "find" and check if similar command exist in queue for inverter and delete old one and insert(push) new command into queue...

[kjmwwi]

I have the same setup and the same issues. I increased the publishing intervall of the new powerlimit via mqtt to 33seconds. This is working fine over the day.
But HA and Node Red are sending also new limits over night => so they fill up the mqtt-queue. In the morning when the inverter starts I do not get a respone of the inverters until I rebbot opendtu.

[stefan123t]

@DejanBukovec as Thomas explained the commands are queued in your scenarios.
The normal time for a RealTimeRunData_Debug | 0x0B command is minimum 250-300 ms if the connection is good.
If you are having larger inverters like 2-in-1 or 4-in-1 models with more per-channel detail data or
if you have connection issues this may take considerably longer.
Besides the RealTimeRunData_Debug command the OpenDTU has to query some other commands in between.

For the Active Power Limit | 0x0B command it will not only transmit the new power limit but also has to wait several seconds (15-20 seconds) until the inverter responds with the new value to the SystemConfigPara | 0x05 command which we need to show the new value has been accepted.

So it is only natural that in case you have filled up your FiFo queue with Active Power Limit and System Config Para commands, that this will eventually prevent your OpenDTU from querying new RealTimeRunData_Debug data from the inverters.
This is the effect you have described in Scenario 4.

Please double check the Dynamic Power Limiter (DPL) function in the daughter project OpenDTU-OnBattery.
I have described a possible PM+DPL setup with a Shelly Pro3EM as Power Meter (PM) here in issue #272.

@tbnobody nevertheless I second Dejan's idea of finding older entries with Active Power Limit | 0x0B for the same inverter, which could be skipped or overwritten when a new Command has been received for the same inverter with a new value. Thus preventing the queue from growing or even overflowing too easily.

[DejanBukovec]

Hi,
It is not actual anymore for me...
Changing FiFo to something else and keep only last power command for specific inverter is still actual... Maybe instead of using power control in command queue use standard array where can be stored only one power value per inverter and one value 0/1 if need to be send... This array will have priority over command queue...

[stefan123t]

@tbnobody as Dejan mentioned we should consider a different scheme regarding the ActivePowerLimit.

As this takes quite some time to send to the inverter and wait for its update/confirmation, we may be faster to compare the current ActivePowerLimit on the OpenDTU before we actually try to send any command via NRF/CMT.

As Dejan noted only the last requested value / command should be executed. i am unsure if we have to distinguish / prioritize between Temporary ( aka. non-persistent ;) ) and Persistent PowerLimit values too ?

[tbnobody]

see #2356