Scripting: MQTT Subscribe - How to decompose JSON

[jkwim]

I am trying to process 3 values from the MQTT JSON payload like this:

stat/tasmota/SENSOR = {"Time":"2022-10-27T19:54:24","ActPw":"588.00","BatV":"54.00","GridV":"233.9"}

I tried following based on examples:

>D
bat=0
grid=0
actpw=0

>B

if mqtts>0
then
=#subs
endif

#subs
+>Subscribe grid, stat/tasmota/SENSOR, GridV
+>Subscribe bat, stat/tasmota/SENSOR, BatV
+>Subscribe actpw, stat/tasmota/SENSOR, ActPw



>S
if mqtts>0
then
=#subs
endif
print S: grid=%grid% bat=%bat% actpw=%actpw%

However the script does not seem to pick all three values.

It picks only one value. That is also random, sometimes it picks GridV, othertimes it picks one of the other two. Always one value get correctly populated.

From the mosquitto broker logs, I can see that complete JSON payload (78 bytes) being sent to the device.

What am I doing wrong here?

[jkwim]

Can anybody help me please?

[jkwim]

Any chance that you can change the format on the other device to publish each value on a different topic ?

As a temporary workaround I can. I prefer to get it in a single message in order to reduce the load and traffic on network. I am generating the message every 10 secs.

Will try that.

BTW How would the Rule Engine work in a similar situation?
Does it support picking multiple values/events from same subscription?

[jkwim]

Wonder whether there is a possibility of picking the values in to an array instead of a single variable with one subscription.

[sfromis]

BTW How would the Rule Engine work in a similar situation?

What I mentioned above was with the rule engine, not with scripting. As the Subscribe command is the same for both, I'd not expect divergent behavior anyway.

[barbudor]

If you have an ESP32, Berry would handle json

[sfromis]

Did a small test with an older binary, "Version":"10.1.0.3(tasmota)","BuildDateTime":"2021-12-27T22:16:36", same behavior with rules. Thus no indication of some recent change around this.

[gemu2015]

i am on travel for several weeks without sufficient internet access
will be back at end of november

[hk2k2]

Hello! As i have the same Problem here on Rules and also on Scripting.. Are there any news to this problem? Many Thanks! Br hk2k2

[hk2k2]

Yeah many Thanks! First Test with scripting works! (2 Values from JSON extracted)

Any fix for Rules too?? Thank you and HAPPY NEW YEAR!

EDIT:

here some debug messages if you need:

18:44:03.547 Script: MQTT Topic Smartmeter/Normaltarif/out, Event {"1_8_0":25494733,"2_8_0":1112272,"3_8_1":95775,"4_8_1":7113218,"1_7_0":1281,"2_7_0":0,"3_7_0":0,"4_7_0":722,"saldo":1281,"1_128_0":0}
18:44:03.549 Script: Match MQTT message Topic Smartmeter/Normaltarif/out with subscription topic Smartmeter/Normaltarif/out and key 1_8_0
18:44:03.551 Script: setting script var >sm180="25494733"

18:44:03.552 Script: Match MQTT message Topic Smartmeter/Normaltarif/out with subscription topic Smartmeter/Normaltarif/out and key saldo
18:44:03.553 Script: setting script var >saldo="1281"

[gemu2015]

see pr, should be fixed in rules too.

happy new year!

[hk2k2]

Hello, i´m sorry, but rules won´t work :(

My Ruleset:

ON mqtt#connected do backlog0 subscribe sm170, Smartmeter/Normaltarif/out, 1_7_0;subscribe saldo, Smartmeter/Normaltarif/out, saldo;subscribe sm180, Smartmeter/Normaltarif/out, 1_8_0; endon
on event#sm170 do var1=%value% endon
on event#sm180 do var2=%value% endon
on event#saldo do var3=%value% endon

Result is, only the LAST SUBSCRIBED Value works! it my case, i only got "sm180" event triggered.

and here some WEBLOG 4 data:

15:45:24.039 WIF: DNS resolved '192.168.40.1' (192.168.40.1) in 1 ms
15:45:24.047 MQT: Connected
15:45:24.051 MQT: tele/tasmota_E52AA3/LWT = Online (retained)
15:45:24.053 MQT: cmnd/tasmota_E52AA3/POWER = 
15:45:24.054 MQT: Subscribe to cmnd/tasmota_E52AA3/#
15:45:24.055 MQT: Subscribe to cmnd/tasmotas/#
15:45:24.056 MQT: Subscribe to cmnd/DVES_E52AA3_fb/#
15:45:24.058 MQT: tele/tasmota_E52AA3/INFO1 = {"Info1":{"Module":"Sonoff Basic","Version":"12.3.1.3(DISPLAY)","FallbackTopic":"cmnd/DVES_E52AA3_fb/","GroupTopic":"cmnd/tasmotas/"}}
15:45:24.070 MQT: tele/tasmota_E52AA3/INFO2 = {"Info2":{"WebServerMode":"Admin","Hostname":"tasmota-E52AA3-2723","IPAddress":"192.168.40.202"}}
15:45:24.082 MQT: tele/tasmota_E52AA3/INFO3 = {"Info3":{"RestartReason":"Software/System restart","BootCount":19}}
15:45:24.092 MQT: stat/tasmota_E52AA3/RESULT = {"POWER":"OFF"}
15:45:24.097 MQT: stat/tasmota_E52AA3/POWER = OFF
15:45:24.229 RUL: MQTT#CONNECTED performs "backlog0 subscribe sm170, Smartmeter/Normaltarif/out, 1_7_0;subscribe saldo, Smartmeter/Normaltarif/out, saldo;subscribe sm180, Smartmeter/Normaltarif/out, 1_8_0;"
15:45:24.237 SRC: Rule
15:45:24.239 CMD: Grp 0, Cmd 'BACKLOG', Idx 0, Len 153, Pld -99, Data 'subscribe sm170, Smartmeter/Normaltarif/out, 1_7_0;subscribe saldo, Smartmeter/Normaltarif/out, saldo;subscribe sm180, Smartmeter/Normaltarif/out, 1_8_0;'
15:45:24.243 SRC: Backlog
15:45:24.244 CMD: Grp 0, Cmd 'SUBSCRIBE', Idx 1, Len 40, Pld -99, Data 'sm170, Smartmeter/Normaltarif/out, 1_7_0'
15:45:24.246 MQT: Subscribe to Smartmeter/Normaltarif/out/#
15:45:24.252 MQT: stat/tasmota_E52AA3/RESULT = {"Subscribe":"SM170,Smartmeter/Normaltarif/out/#,1_7_0"}
15:45:24.258 SRC: Backlog
15:45:24.260 CMD: Grp 0, Cmd 'SUBSCRIBE', Idx 1, Len 40, Pld -99, Data 'saldo, Smartmeter/Normaltarif/out, saldo'
15:45:24.262 MQT: Subscribe to Smartmeter/Normaltarif/out/#
15:45:24.265 MQT: stat/tasmota_E52AA3/RESULT = {"Subscribe":"SALDO,Smartmeter/Normaltarif/out/#,saldo"}
15:45:24.273 SRC: Backlog
15:45:24.274 CMD: Grp 0, Cmd 'SUBSCRIBE', Idx 1, Len 40, Pld -99, Data 'sm180, Smartmeter/Normaltarif/out, 1_8_0'
15:45:24.276 MQT: Subscribe to Smartmeter/Normaltarif/out/#
15:45:24.281 MQT: stat/tasmota_E52AA3/RESULT = {"Subscribe":"SM180,Smartmeter/Normaltarif/out/#,1_8_0"}
15:45:24.294 RUL: EVENT#SM180 performs "var2=25516825"
15:45:24.295 SRC: Rule
15:45:24.296 CMD: Grp 0, Cmd 'VAR', Idx 2, Len 9, Pld -99, Data '=25516825'
15:45:24.301 MQT: stat/tasmota_E52AA3/RESULT = {"Var2":"25516824.000"}
15:45:24.431 RUL: EVENT#SM180 performs "var2=25516825"
15:45:24.432 SRC: Rule
15:45:24.433 CMD: Grp 0, Cmd 'VAR', Idx 2, Len 9, Pld -99, Data '=25516825'
15:45:24.438 MQT: stat/tasmota_E52AA3/RESULT = {"Var2":"25516824.000"}
15:45:24.531 RUL: EVENT#SM180 performs "var2=25516825"
15:45:24.532 SRC: Rule
15:45:24.533 CMD: Grp 0, Cmd 'VAR', Idx 2, Len 9, Pld -99, Data '=25516825'
15:45:24.538 MQT: stat/tasmota_E52AA3/RESULT = {"Var2":"25516824.000"}

Thank You! br hk2k2

[gemu2015]

rules had the same error then scripts: json payload is destroyed when decoded and had to be preserved when iterating through the subscribed items. that's what i did exactly with the rules subroutine which is very similar to scripting. i see however that in contrast to scripts rules generate a single event message only and that will be the last key found.
sorry this has to be solved by @arends.

[hk2k2]

Ok Thank You.
I will try to workaround with scripting in my special usecase.. but i hope rules would be fixed too .

@arends ? do you mean @arendst ?

br hk2k2

[gemu2015]

yes, typo

[jkwim]

Thanks @gemu2015 for following this up. I have been following the thread its just that I need to setup my test environment again to verify this. I decided to live with separate MQTT messages for the time being however I want to test this out asap.

Thanks @hk2k2 for continuing the discussion.

[jkwim]

And HAPPY NEW YEAR! to both of you.

[jkwim]

What version of Tasmota should I use with this? I tried compiling 12.1.1.2 but failed with following error: ``` Linking .pio/build/tasmota-smartmeter/firmware.elf /home/ubuntu/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: .pio/build/tasmota-smartmeter/src/tasmota.ino.cpp.o:(.text._Z14Run_script_subPKcaP5GVARS+0xa4): undefined reference to `sml_options' /home/ubuntu/.platformio/packages/toolchain-xtensa/bin/../lib/gcc/xtensa-lx106-elf/4.8.2/../../../../xtensa-lx106-elf/bin/ld: .pio/build/tasmota-smartmeter/src/tasmota.ino.cpp.o:(.irom.text.xdrv_interface.ino.21.1989+0x24): undefined reference to `Xdrv10(unsigned char)' collect2: error: ld returned 1 exit status *** [.pio/build/tasmota-smartmeter/firmware.elf] Error 1 ========================= [FAILED] Took 311.61 seconds ========================= ```

[jkwim]

Something in the form of:

tele/tasmotaxxx/SENSOR = {"Time":"2023-01-06T21:42:03","ENERGY":{"TotalStartTime":"2022-11-20T19:13:51","Total":237.476,"Yesterday":0.014,"Today":5.087,"Period":0,"Power":0,"ApparentPower":0,"ReactivePower":0,"Factor":0.00,"Voltage":0,"Current":0.000}}

All parameters that I am interested in are in the container ENERGY.

[gemu2015]

if you subscribe this it works: (replace # with .) at least with scripts
Subscribe actpw, stat/tasmota/SENSOR, ENERGY.Total

[jkwim]

Ah! Ha! I never imagined a difference in the syntax.

It works now!

Thank you all!

This is great!. I now can have my project completed.

Actually what I am doing is using this RGB Smart Plug as the monitoring/controlling device. It is simply a device with a RGB light and a button.

So I can use it to display certain status of devices and even send control signals to some of them using button presses.

This can be just moved around and plugged in to any outlet within WiFi coverage.

[jkwim]

@hk2k2,
We should be able to close this discussion. Do you agree?

[sfromis]

Discussions are not "closed", they just stay dormant if no one posts.

[jkwim]

Ok. Thanks. Will test.

[hk2k2]

yes. i tried with latest DEV too :)
i´m not so familiar with github, regarding the still existing problem with rules ... should i (or you) open a ISSUE for this??
thanks! br hk2k2

[gemu2015]

looking at the rule sources i think it is not trivial to fix this. anyhow i am not the owner of rules and therefore you may open an issue for that problem.

[arendst]

I've never looked at the rule MQTT subscribe/unsubscribe before but from a quick chk I see you are trying to subscribe three times the same MQTT topic (Smartmeter/Normaltarif/out) to three different events (sm170, saldo, sm180) based on three different payloads (1_7_0, saldo, 1_8_0).

MQTT initially looks for a topic. Once detected it starts to act on the contents of the payload. So as you actually only subscribe to Smartmeter/Normaltarif/out it's obvious only the last subscribe command with event sm180 will execute that event; the previous subscribes were simply overwritten by the last same topic subscribe.

The payload to act upon looks like this (I think):

{"1_8_0":25494733,"2_8_0":1112272,"3_8_1":95775,"4_8_1":7113218,"1_7_0":1281,"2_7_0":0,"3_7_0":0,"4_7_0":722,"saldo":1281,"1_128_0":0}

and you want to store the values of fieldpair 1_7_0 into var1, saldo into var2 and 1_8_0 into var3.

Pls verify if the latter is the case you're requesting.

[sfromis]

Someone was suggesting it to already be possible to use a single subscribe for multiple fields from the same message.
#11636 (comment)
Not tested that myself.

[arendst]

Yes that's the solution. Only problem is that the current buffer size is too small, 100 characters for events. I'll change it to 256 characters to support a JSON payload of 256 characters like above.

In a test using the above JSON message and command subscribe sm170, Smartmeter/Normaltarif/out with a rule like rule2 on event#sm170#1_7_0 do var1 %value% endon on event#sm170#saldo do var2 %value% endon on event#sm170#1_8_0 do var3 %value% endon I'll get a response like:

14:50:38.660-028 CMD: rule2 on event#sm170#1_7_0 do var1 %value% endon on event#sm170#saldo do var2 %value% endon on event#sm170#1_8_0 do var3 %value% endon
14:50:38.666-028 SRC: Serial
14:50:38.670-028 CMD: Grp 0, Cmd 'RULE', Idx 2, Len 128, Pld -99, Data 'on event#sm170#1_7_0 do var1 %value% endon on event#sm170#saldo do var2 %value% endon on event#sm170#1_8_0 do var3 %value% endon'
14:50:38.688-028 RUL: Stored uncompressed, would compress from 128 to 57 (-55%)
14:50:38.700-025 MQT: stat/demo/RESULT = {"Rule2":{"State":"OFF","Once":"OFF","StopOnError":"OFF","Length":128,"Free":383,"Rules":"on event#sm170#1_7_0 do var1 %value% endon on event#sm170#saldo do var2 %value% endon on event#sm170#1_8_0 do var3 %value% endon"}}
14:50:44.137-028 CMD: rule2 1
14:50:44.139-028 SRC: Serial
14:50:44.142-028 CMD: Grp 0, Cmd 'RULE', Idx 2, Len 1, Pld 1, Data '1'
14:50:44.152-025 MQT: stat/demo/RESULT = {"Rule2":{"State":"ON","Once":"OFF","StopOnError":"OFF","Length":128,"Free":383,"Rules":"on event#sm170#1_7_0 do var1 %value% endon on event#sm170#saldo do var2 %value% endon on event#sm170#1_8_0 do var3 %value% endon"}}
14:50:50.269-027 RUL: EVENT#SM170#1_7_0 performs "var1 1281"
14:50:50.271-027 SRC: Rule
14:50:50.274-027 CMD: Grp 0, Cmd 'VAR', Idx 1, Len 4, Pld 1281, Data '1281'
14:50:50.281-024 MQT: stat/demo/RESULT = {"Var1":"1281"}
14:50:50.294-026 RUL: EVENT#SM170#SALDO performs "var2 1281"
14:50:50.296-026 SRC: Rule
14:50:50.299-026 CMD: Grp 0, Cmd 'VAR', Idx 2, Len 4, Pld 1281, Data '1281'
14:50:50.305-026 MQT: stat/demo/RESULT = {"Var2":"1281"}
14:50:50.319-027 RUL: EVENT#SM170#1_8_0 performs "var3 25494733"
14:50:50.321-027 SRC: Rule
14:50:50.324-027 CMD: Grp 0, Cmd 'VAR', Idx 3, Len 8, Pld 25494733, Data '25494733'
14:50:50.330-024 MQT: stat/demo/RESULT = {"Var3":"25494733"}

which looks to me the OP wanted.

[arendst]

Dev release now supports payloads up to 256 characters accomodating the above shown solution.

[sfromis]

While stepping up to 256 helps a lot, multisensor devices can easily get much longer.
Not an issue to me, as I preferentially use Berry on ESP32, if I do not manage it backend.

[arendst]

The stepping up is only needed for EXTERNAL JSON messages received by a Tasmota device (for displaying). Normal sensor data is send using a buffer the size of available RAM so not an issue.

[sfromis]

Multisensor messages are external when seen from other Tasmota devices. No go. But works with Berry, so I'm not really affected....

[jkwim]

I also have a device which sends 264 bytes. I am using a second message with less number of parameters so that I can fit inside the 256 byte limitation on the receiving side. Kind of duplicating the message but since I have to do it only in one device I am ok with that.

[arendst]

Tasmota byte size hasn't change but I did add recursive search through the subscribed topics allowing to use the same subscribe topic with different eventnames and/or fields in rules like in scripts just now.

[hk2k2]

@arendst ,

Pls verify if the latter is the case you're requesting.

yes, exact what i try to reach:)

@sfromis @arendst
just read your postings, and i will try your suggested variant as soon as possible and give feedback! Thank You very much! br hk2k2

[hk2k2]

Wonderful!

Rule1
ON mqtt#connected do backlog0 subscribe sm, Smartmeter/Normaltarif/out endon 

Rule2
on event#sm#1_7_0 do var1 sm170:%value% endon 
on event#sm#1_8_0 do var2 sm180:%value% endon 
on event#sm#saldo do var3 saldo:%value% endon

result is:

19:20:11.212 SRC: WebConsole from 192.168.10.197
19:20:11.214 CMD: Grp 0, Cmd 'WEBLOG', Idx 1, Len 1, Pld 3, Data '3'
19:20:11.217 MQT: stat/tasmota_E52AA3/RESULT = {"WebLog":3}
19:20:11.445 RUL: EVENT#SM#1_7_0 performs "var1 sm170:1437"
19:20:11.446 SRC: Rule
19:20:11.447 CMD: Grp 0, Cmd 'VAR', Idx 1, Len 10, Pld -99, Data 'sm170:1437'
19:20:11.452 MQT: stat/tasmota_E52AA3/RESULT = {"Var1":"sm170:1437"}
19:20:11.461 RUL: EVENT#SM#1_8_0 performs "var2 sm180:25613443"
19:20:11.462 SRC: Rule
19:20:11.464 CMD: Grp 0, Cmd 'VAR', Idx 2, Len 14, Pld -99, Data 'sm180:25613443'
19:20:11.470 MQT: stat/tasmota_E52AA3/RESULT = {"Var2":"sm180:25613443"}
19:20:11.479 RUL: EVENT#SM#SALDO performs "var3 saldo:1437"
19:20:11.480 SRC: Rule
19:20:11.481 CMD: Grp 0, Cmd 'VAR', Idx 3, Len 10, Pld -99, Data 'saldo:1437'
19:20:11.486 MQT: stat/tasmota_E52AA3/RESULT = {"Var3":"saldo:1437"}
19:20:11.599 CFG: Saved to flash at F4, Count 69, Bytes 4096
19:20:12.464 RUL: EVENT#SM#1_7_0 performs "var1 sm170:1432"
19:20:12.465 SRC: Rule
19:20:12.467 CMD: Grp 0, Cmd 'VAR', Idx 1, Len 10, Pld -99, Data 'sm170:1432'
19:20:12.472 MQT: stat/tasmota_E52AA3/RESULT = {"Var1":"sm170:1432"}
19:20:12.481 RUL: EVENT#SM#1_8_0 performs "var2 sm180:25613444"
19:20:12.482 SRC: Rule
19:20:12.484 CMD: Grp 0, Cmd 'VAR', Idx 2, Len 14, Pld -99, Data 'sm180:25613444'
19:20:12.489 MQT: stat/tasmota_E52AA3/RESULT = {"Var2":"sm180:25613444"}
19:20:12.496 RUL: EVENT#SM#SALDO performs "var3 saldo:1432"
19:20:12.497 SRC: Rule

exactly what i need, thanks for your help and suggestions! I appreciate this very much!
Now i can do further work on my project.

br hk2k2