Discovery with theengs decoder #1106

1technophile · 2021-12-04T22:33:33Z

Description:

Recover the auto discovery function for BLE device through the use of Theengs Decoder
Use std::string for efficiency on checking devices
Close #1102

Checklist:

The pull request is done against the latest development branch
Only one feature/fix was added per PR and the code change compiles without warnings
I accept the DCO.

1technophile · 2021-12-04T23:22:01Z

@NorthernMan54 would it be possible to check if this PR enables still to retrieve DT24 data please ?

NorthernMan54 · 2021-12-04T23:28:20Z

@1technophile Yes, I can do a regression test. Just need approx 72 hours ( currently not close to my DT24 device )

NorthernMan54 · 2021-12-10T21:57:44Z

@1technophile - Status Update - Loaded this version up last night onto one of my test devices last night, and ran into this exception being triggered just after boot up. So now double clicking into the issue, and will update. My first check will be to see if this was trigger by pull request or something prior to it. I had not updated to a newer code base in a few moths, so wanted to check that first.

N: ************** Setup OpenMQTTGateway end **************
W: MQTT connection...
N: Connected to broker
N: Send on /SYStoMQTT msg {"uptime":11,"version":"ddd-bt-rtl-v0.9.8-32-g8446f65[decoder-dev]","freemem":148396,"mqttport":"1883","mqttsecure":false,"freestack":6220,"rssi":-50,"SSID":"67 Bonacres","ip":"192.168.1.181","mac":"24:0A:C4:EC:20:DC","lowpowermode":0,"btqblck":0,"btqsum":0,"btqsnd":0,"btqavg":0,"interval":55555,"scanbcnct":10,"scnct":0,"modules":["BT"]}
N: Scan begin
N: Device detected: 68:9E:19:DA:21:E3
Guru Meditation Error: Core  0 panic'ed (Unhandled debug exception)
Debug exception reason: Stack canary watchpoint triggered (ble) 
Core 0 register dump:
PC      : 0x400d7942  PS      : 0x00060836  A0      : 0x800d79e4  A1      : 0x3ffde790  
A2      : 0x3ffdf0f0  A3      : 0x3ffde85c  A4      : 0x0000000a  A5      : 0x3f40acd0  
A6      : 0xa5a5a500  A7      : 0x3ffe5374  A8      : 0x800d793e  A9      : 0x3ffde750  
A10     : 0x3ffdf0f0  A11     : 0x00000000  A12     : 0x00000000  A13     : 0x3ffde760  
A14     : 0x00000000  A15     : 0xff000000  SAR     : 0x00000010  EXCCAUSE: 0x00000001  
EXCVADDR: 0x00000000  LBEG    : 0x4000c2e0  LEND    : 0x4000c2f6  LCOUNT  : 0x00000000  

ELF file SHA256: 0000000000000000

Backtrace: 0x400d7942:0x3ffde790 0x400d79e1:0x3ffde850 0x400faa67:0x3ffde880 0x400d3b79:0x3ffdf310 0x400d683e:0x3ffdf350 0x400d9a99:0x3ffdf390 0x400e7189:0x3ffdf570 0x400ea4ba:0x3ffdf5b0 0x400ea7e9:0x3ffdf620 0x400ef2ca:0x3ffdf640 0x400ef0b2:0x3ffdf680 0x400ef4e5:0x3ffdf6a0 0x400ee085:0x3ffdf6c0 0x400f5b0a:0x3ffdf6e0 0x400e54a3:0x3ffdf700 0x40090d52:0x3ffdf720
  #0  0x400d7942:0x3ffde790 in ArduinoJson6183_D1::DeserializationError ArduinoJson6183_D1::deserialize<ArduinoJson6183_D1::JsonDeserializer, char const*, ArduinoJson6183_D1::AllowAllFilter>(ArduinoJson6183_D1::JsonDocument&, char const*&, ArduinoJson6183_D1::NestingLimit, ArduinoJson6183_D1::AllowAllFilter) at main/ZgatewayBT.ino:1041
  #1  0x400d79e1:0x3ffde850 in ArduinoJson6183_D1::DeserializationError ArduinoJson6183_D1::deserializeJson<char const>(ArduinoJson6183_D1::JsonDocument&, char const*, ArduinoJson6183_D1::NestingLimit) at main/ZgatewayBT.ino:1041
  #2  0x400faa67:0x3ffde880 in TheengsDecoder::decodeBLEJson(ArduinoJson6183_D1::ObjectRef&) at .pio/libdeps/ddd-bt-rtl/ArduinoJson/src/ArduinoJson/Variant/VariantData.hpp:116
  #3  0x400d3b79:0x3ffdf310 in process_bledata(ArduinoJson6183_D1::ObjectRef&) at main/ZgatewayBT.ino:1041
  #4  0x400d683e:0x3ffdf350 in PublishDeviceData(ArduinoJson6183_D1::ObjectRef&, bool) at main/ZgatewayBT.ino:1041
  #5  0x400d9a99:0x3ffdf390 in MyAdvertisedDeviceCallbacks::onResult(NimBLEAdvertisedDevice*) at main/ZgatewayBT.ino:1041
  #6  0x400e7189:0x3ffdf570 in NimBLEScan::handleGapEvent(ble_gap_event*, void*) at .pio/libdeps/ddd-bt-rtl/NimBLE-Arduino/src/NimBLEScan.cpp:129
  #7  0x400ea4ba:0x3ffdf5b0 in ble_gap_disc_report at .pio/libdeps/ddd-bt-rtl/NimBLE-Arduino/src/nimble/host/src/ble_gap.c:5908
  #8  0x400ea7e9:0x3ffdf620 in ble_gap_rx_adv_report at .pio/libdeps/ddd-bt-rtl/NimBLE-Arduino/src/nimble/host/src/ble_gap.c:5908
  #9  0x400ef2ca:0x3ffdf640 in ble_hs_hci_evt_le_adv_rpt at .pio/libdeps/ddd-bt-rtl/NimBLE-Arduino/src/nimble/host/src/ble_hs_hci_evt.c:501
  #10 0x400ef0b2:0x3ffdf680 in ble_hs_hci_evt_le_meta at .pio/libdeps/ddd-bt-rtl/NimBLE-Arduino/src/nimble/host/src/ble_hs_hci_evt.c:301
  #11 0x400ef4e5:0x3ffdf6a0 in ble_hs_hci_evt_process at .pio/libdeps/ddd-bt-rtl/NimBLE-Arduino/src/nimble/host/src/ble_hs_hci_evt.c:859
  #12 0x400ee085:0x3ffdf6c0 in ble_hs_event_rx_hci_ev at .pio/libdeps/ddd-bt-rtl/NimBLE-Arduino/src/nimble/nimble_npl_os.h:109
  #13 0x400f5b0a:0x3ffdf6e0 in ble_npl_event_run at .pio/libdeps/ddd-bt-rtl/NimBLE-Arduino/src/nimble/nimble_npl_os.h:121
      (inlined by) nimble_port_run at .pio/libdeps/ddd-bt-rtl/NimBLE-Arduino/src/porting/nimble/src/nimble_port.c:81
  #14 0x400e54a3:0x3ffdf700 in NimBLEDevice::host_task(void*) at .pio/libdeps/ddd-bt-rtl/NimBLE-Arduino/src/NimBLEDevice.cpp:632
  #15 0x40090d52:0x3ffdf720 in vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c:355 (discriminator 1)

Rebooting...
����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������N: 
************* WELCOME TO OpenMQTTGateway **************
N: OpenMQTTGateway Version: ddd-bt-rtl-v0.9.8-32-g8446f65[decoder-dev]
N: OTA Hostname: OMG_240AC4EC20DC.local
[E][WiFiMulti.cpp:187] run(): [WIFI] Connecting Failed (0).
N: WiFi ok with manual config credentials
N: BLE scans interval: 55555
N: BLE scans number before connect: 10
N: Publishing only BLE sensors: false
N: minrssi: 100
N: Low Power Mode: 0
N: OpenMQTTGateway modules: ["BT"]
N: ************** Setup OpenMQTTGateway end **************
W: MQTT connection...
N: Connected to broker
N: Send on /SYStoMQTT msg {"up

1technophile · 2021-12-10T22:05:48Z

The canary exception, I had it once, would it be possible to activate the trace log level to have more information please? @NorthernMan54

h2zero · 2021-12-11T00:05:50Z

Might need to increase the NimBLE host task stack size a bit. NimBLE-Arduino Master branch might also resolve this and also has added calls to check the stack use.

NorthernMan54 · 2021-12-11T03:59:13Z

@h2zero That was it, I upped the stack size from the default 4096 to 8096 and things stabilized. Now for some regression testing.

I had added this to my env

'-DCONFIG_BT_NIMBLE_TASK_STACK_SIZE=8096'

h2zero · 2021-12-11T04:06:08Z

Thought so, in order to narrow down the stack appropriately we should check the task stack high water mark.

This can be done by calling uxTaskGetStackHighWaterMark(xTaskGetHandle("ble")); somewhere in the loop code.

NorthernMan54 · 2021-12-11T04:22:37Z

In regards to the DT24, I'm not seeing it connect and retrieve the data

I'm seeing this in the long when it scan's for BT devices

N: Scan begin
T: Creating BLE buffer
N: Device detected: 20:05:07:16:13:61
T: getDeviceByMac 20:05:07:16:13:61
T: No device found 
N: Send on /BTtoMQTT/200507161361 msg T: {"id":"20:05:07:16:13:61","name":"DT24-BLE","manufacturerdata":"733688a0200507151361","rssi":-73}
Creating BLE buffer
....
N: Found 14 devices, scan number 1 end
N: BLE Connect begin
N: BLE Connect end

But no connection and data parsing, more digging in the AM

h2zero · 2021-12-11T04:48:51Z

I think I see that issue, looks like on line 858 we are using the name "DT24-BLE" and on line 793 we check for a name of just "DT24".

I'm not sure which one is correct but it would explain the lack of detection.

1technophile · 2021-12-11T15:23:27Z

I think I see that issue, looks like on line 858 we are using the name "DT24-BLE" and on line 793 we check for a name of just "DT24".

Thanks for pointing this, corrected

NorthernMan54 · 2021-12-11T16:58:47Z

And now it is working again, tks

{"model":"DT24","id":"20:05:07:16:13:61","volt":15.8,"current":0,"power":0,"energy":0,"price":2.01,"tempc":20,"tempf":68}

@h2zero Was looking into the stack issue a little bit, and attempted to add this as a new BT element in SYStoMQTT

uxTaskGetStackHighWaterMark(xTaskGetHandle("ble"));

But got stuck as xTaskGetHandle was 'not declared in this scope', but then I noticed we are already sharing the free stack size as part of the SYStoMQTT message, so am going to let it run for a in graph mode aka freestack

After 370 seconds of uptime, I'm sitting at a free stack of 2004

That is with the stack set to '-DCONFIG_BT_NIMBLE_TASK_STACK_SIZE=8096'

h2zero · 2021-12-11T17:07:18Z

But got stuck as xTaskGetHandle was 'not declared in this scope'

This function might be disabled in Arduino core 👎

The free stack reported in SYStoMQTT is a different stack from the NimBLE host task stack.

Doesn't really matter though as we would need a way to increase the stack in arduino as well, which would require users to set the value in the nimconfig file of the library. Might be a better solution to create a new task to handle the device detection from the callback.

1technophile · 2021-12-11T17:26:55Z

And now it is working again, tks

Thanks for the feedback

1technophile · 2021-12-11T17:28:20Z

That is with the stack set to '-DCONFIG_BT_NIMBLE_TASK_STACK_SIZE=8096'

I will add this in a separate PR

1technophile · 2021-12-16T12:19:22Z

Should close #1105 also

Discovery with theengs decoder

c34393a

1technophile added this to the v0.9.9 milestone Dec 4, 2021

Add devices to doc

979ccbf

Correct missing suffix for DT24

addc59d

1technophile merged commit 7dc4916 into development Dec 11, 2021

1technophile deleted the decoder-discovery branch December 11, 2021 17:54

1technophile linked an issue Dec 16, 2021 that may be closed by this pull request

actuatorONOFF no longer working with Home Assistant #1105

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Discovery with theengs decoder #1106

Discovery with theengs decoder #1106

1technophile commented Dec 4, 2021 •

edited

Loading

1technophile commented Dec 4, 2021

NorthernMan54 commented Dec 4, 2021

NorthernMan54 commented Dec 10, 2021

1technophile commented Dec 10, 2021

h2zero commented Dec 11, 2021

NorthernMan54 commented Dec 11, 2021

h2zero commented Dec 11, 2021 •

edited

Loading

NorthernMan54 commented Dec 11, 2021

h2zero commented Dec 11, 2021 •

edited

Loading

1technophile commented Dec 11, 2021

NorthernMan54 commented Dec 11, 2021

h2zero commented Dec 11, 2021

1technophile commented Dec 11, 2021

1technophile commented Dec 11, 2021

1technophile commented Dec 16, 2021

Discovery with theengs decoder #1106

Discovery with theengs decoder #1106

Conversation

1technophile commented Dec 4, 2021 • edited Loading

Description:

Checklist:

1technophile commented Dec 4, 2021

NorthernMan54 commented Dec 4, 2021

NorthernMan54 commented Dec 10, 2021

1technophile commented Dec 10, 2021

h2zero commented Dec 11, 2021

NorthernMan54 commented Dec 11, 2021

h2zero commented Dec 11, 2021 • edited Loading

NorthernMan54 commented Dec 11, 2021

h2zero commented Dec 11, 2021 • edited Loading

1technophile commented Dec 11, 2021

NorthernMan54 commented Dec 11, 2021

h2zero commented Dec 11, 2021

1technophile commented Dec 11, 2021

1technophile commented Dec 11, 2021

1technophile commented Dec 16, 2021

1technophile commented Dec 4, 2021 •

edited

Loading

h2zero commented Dec 11, 2021 •

edited

Loading

h2zero commented Dec 11, 2021 •

edited

Loading