New advertising flags: about 8 % shorter battery life?

[JsBergbau]

Hi @pvvx
thanks for you very energy efficient versions. I've found this very interesting picture https://pvvx.github.io/ATC_MiThermometer/CustPower.html
As you can see most energy is consumend during advertising.
New advertising with version 2.9 format is 35 bytes long instead of 32 Bytes for ATC1441 Frame. So 3 Bytes more transmitted which means 35 / 32 = 1,09375, so about 9 % longer trasmit time. Now take 1 % off because of baseline current that would mean 8 % shorter battery life. Am I wrong? Is there a possibility to enable old format again?

Update: Also with this new format device is displayed when you search for bluetooth devices in Android. You can tap and then it tells that there is an App required. This makes the device more visible to average users. When they see like 10 devices or so then they will try to find out what it is about. So I'd really prefer enabling old advertising behaviour where device is hidden in standard Android Bluetooth scan.

[pvvx]

There are more detailed diagrams.
For a complete event with data updates from the sensor and indicator

The typical cycle is roughly the same as the original version.
Main loop in Advertising mode
But the RF TX level in dB is increased for the custom firmware (+3.01dB).

If sensor Normal mode:
If we take a window of 18 ms, TX-RF +3 db, then the average consumption is:
Advertising, no other action ........ 1.85 mA
Advertising + sensor reading and update display ........ 2.85 mA

TX sending time is about 300 us.
300 * 3/32 -> + 28 us * 8 mA * 3
3 * 8mA * 0.028ms / 18 ms = 0.0372 mA
Average consumption (ad only, windows 18 ms) -> 1.85 + 0.0372 mA
100 * 0.0372 / 1.85 = 2 % (max for 'Advertising, no other action') #79 (comment)

[pvvx]

Update: Also with this new format device is displayed when you search for bluetooth devices in Android. You can tap and then it tells that there is an App required. This makes the device more visible to average users. When they see like 10 devices or so then they will try to find out what it is about. So I'd really prefer enabling old advertising behaviour where device is hidden in standard Android Bluetooth scan.

The main reason for the change is to force script "writers" to different "home system" ones to parse packets according to standard Bluetooth LE rules.
Otherwise, users of HA and other systems have problems. The "community" of HA, OpenHab and etc, since 2010, the introduction of Bluetooth LE cannot be integrated into the system to support devices with BLE.
This is more important for users than a temporary 1% increase in consumption by one of the devices.

PS: For example, take a look at these doodles:
https://github.com/JsBergbau/MiTemperature2/blob/master/LYWSD03MMC.py#L445-L450 :) :)

[pvvx]

Ver 3.0 - Added toggle support for advertising package structures for third-party software written by unqualified authors.

[JsBergbau]

Thanks for adding this switch. This also helps to hide the sensors from average users.

BTW: When calling other authors unqualified because they rely on published documentation than you could call another authors also unqualified that advertise data with the same UUID only changing length of the advertisments breaking compatibility with already published data format specification.

[pvvx]

BTW: When calling other authors unqualified because they rely on published documentation than you could call another authors also unqualified that advertise data with the same UUID only changing length of the advertisments breaking compatibility with already published data format specification.

I agree - I am not a programmer and I have nothing to claim. Just to make it work right.
The same UUID was created on purpose as most of the scripts I found did not respect the size and gave erroneous values.
Moreover, there are no standards for the formation of data within AD-data.
UUID numbers are registered at https://www.bluetooth.com/ and the procedure is not free.

breaking compatibility

How is compatibility broken? The size of the structure is different.

Carmakers making big and small cars violate road compatibility :)
Doesn't fit into the gate? :)

[pvvx]

@JsBergbau - I agree with you and remove the atc1441 format in new version (?)

[JsBergbau]

Sorry that is a missunderstanding. ATC format is fine. It is most powerefficient because it is shortest. Please keep it.

[atc1441]

If its just about the size a very short version can be implemented.

Only 3byte MAC
One byte for 8bits of flag, for which values are send
2 byte temp
One byte humi
One byte battery
Plus also only advertise the data on changed values so no counter needed

In the end that may not help anyone at all and just creates the need to support one more protocol

[JsBergbau]

Plus also only advertise the data on changed values so no counter needed

Good idea. On the other hand with devices which have bad reception quality you don't know if data didn't change or it was because of bad reception. Thats why I changed to ATC format only. So every advertisment ATC format is transmitted giving 3times higher chance of receiving packet.

You do a great job and I really appreciate it. My script was only created because at that time there didn't exist one (or at least I didn't find one easy usable script) and over time it has grown.

Of course you're right with packet length and different format. I just didn't expect that. There was this UUID I checked and I thought that is enough to prevent accidentally decoding wrong data.

I also like that you've implemented a PIN code proctetion and you and also send MI Like packets and encrypt your data so no one knows for example when you shower. Perhaps someday when I have more freetime I'll include all these features to make them easily usable and uintil that personally I find the ATC1441 format the best tradeoff between information and packet length.

[atc1441]

Most of these feature did @pvvx :)

But thanks still for your words !

[pvvx]

Of course you're right with packet length and different format. I just didn't expect that. There was this UUID I checked and I thought that is enough to prevent accidentally decoding wrong data.

The mijia format has a dynamic packet size and data is positioned based on internal flags.

If there is no correct parsing of packets, there is no possibility of extension or transition to Bluetooth 5 Extended Advertisements or Periodic Advertisements.
https://docs.silabs.com/bluetooth/latest/general/adv-and-scanning/periodic-adv-bt5
Telink SDK supports advanced advertising. Your program will not work correctly if you do not make changes. It will not be possible to add an AD section with flag pointers to the "Extended Advertisements".
The format of advertising packages has not changed from 2010 to the present day, and so far there is no point in changing it.

To reduce the size, the duplicate MAC must be removed from the AD data. MAC are already conveyed in the header.

[JsBergbau]

Thanks for the hint. Changed program to parse the packet only if it is 13 Bytes in size.

To reduce the size, the duplicate MAC must be removed from the AD data. MAC are already conveyed in the header.

What do you think about ATC1441_V2 Format? It is the same but without (duplicate) MAC. Instead of starting with 10 16 1A 18 it would start with 0A 16 18 1A (0A = 10 Bytes Lenth instead of 16) and then directly begins with temperature.

Or we design a even shorter format.
We could also omit battery level in % as we can calculate it from battery level in mV. But don't know if it would be even worth it doing that effort for one byte. Whereas saving 6 Bytes, reduces payload by 46 % and complete packetlength is reduced by about 19 % which is really significant from my point of view.

[pvvx]

Significantly greater savings can be achieved by using the BT 5.2 standard.
The consumption limit is limited by the leakage in the sleep mode of the control microcircuits LCD, TLSR8251, SHTV3 at the level of 5..8 μA

"Expanded advertising" is broadcast on one channel, not 3. This is already a 3-fold gain.
Also, according to BT5.2 there is an automatic adjustment of the RF-TX power level.
Similarly, a decrease in consumption is possible when switching to MESH - does not require constant transmission of advertising packages.
There is already a preliminary version with ZigBee. But there are unknowns in it with a third-party software product.

PS: Integrated approach is required, not counting the bytes in a packet. In all cases, development is limited to supporting a third-party software product. The development of open-source BLE is just beginning. The open source "community" has done nothing for 10 years. I alone cannot reverse this situation.

[JsBergbau]

"Expanded advertising" is broadcast on one channel, not 3. This is already a 3-fold gain.

So does the sensor currently transmit (at default interval) every 2,5 on all 3 advertisment channels? Even with that it sometimes still takes some some seconds until all receivers got the packet with a higher sequence number and report it. I thought that is because of eventually listening channel and advertising channel were the same and thus the packet was sucessfully receceived.

Also, according to BT5.2 there is an automatic adjustment of the RF-TX power level.

That implies that the receiver reports back the received signal strength, right?

So if I got you right it is not worth creating a ATC1441_V2 format with 8 bytes so save energy? 8 Bytes less because I had another idea to save 2 additional bytes.
1 byte for battery percentage level and one byte by reporting the voltage in discrete levels. For example you could define max volatage 3,4V (in my database are about 30 sensors and maximum voltage was 3.34V). 3,4 / 256 = 0,01328125
So you would have a resolution of 0,01328125 V which should be exact enough. And this 256 discrete steps are also very friendly for influxdb compression. Of course you also could begin lets say at 1,5 V as minium voltage than you would gain a higher resolution. However the lowerst recorded voltage for my sensors is 0.44 V so I would begin with 0 V. This makes calculation easier.

[pvvx]

Reception of advertisements on JDY-10 (TLSR8266) or CC2540 USB Dongle.
https://github.com/pvvx/UBIA/tree/master/Sniffer/OpenWRT_SnifferAd
Reaches several hundred ad packages per second...
https://www.youtube.com/watch?v=MqFcY5Hovpw

Raspberry Pi (2 model B), Ethernet internet connection, Mosquitto MQTT client, Messages per second 0.98 👎

#include <sys/types.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <unistd.h>
#include <sys/wait.h>

static int create_process(void) {
    pid_t pid;
    int status;
    pid = fork(); // or vfork()
    if (-1 == pid) {
        return errno;
    }
    if (pid == 0) {
        /* child */
        exit(EXIT_SUCCESS);
    }
    waitpid(pid, &status, 0);
    return EXIT_SUCCESS;
}

int main(void) {
    int i;
    for (i = 0; i < 100000; i ++) {
        create_process();
    }
    return EXIT_SUCCESS;
}

$ cat /etc/os-release

NAME="Linux Mint"
VERSION="19.2 (Tina)"

$ cat /proc/cpuinfo | grep -i model
model name : Intel(R) Core(TM) i7-7820HK CPU @ 2.90GHz

$ gcc -O3 ./fork.c -o fork

$ time ./fork

real 0m6.704s
user 0m4.400s
sys 0m1.952s

6.704s / 100000 = 0.00006704 -> 67 us 👎
15 kbytes/s if you transfer one byte from process to process.

So does the sensor currently transmit (at default interval) every 2,5 on all 3 advertisment channels?

Yes.