ESPNOW Round trip takes 6 milliseconds (IDFGH-889)

[leonyuhanov]

Environment

• Development Kit: NODE32S
• Kit version 1.1
• Module or chip used: ESP32-WROOM-32
• Build System: MAKE
• Operating System: Windows
• Power Supply: Tried all

Problem Description

I have 2 ESP32 nodes a Sender and a Receiver.
The Sender sends 250 bytes, and starts a microsecond timer awaiting a reply from the receiver
My round trip times are between 5,400 - 7,500 microseconds
Size of the packet doesn't modify the time
This seems rather slow compared to the ESP8266 (even in the Arduino env) can get under 1 millisecond round trips like this.

Expected Behavior

Under 1000 microsecond round trip

Actual Behavior

My round trip times are between 5,400 - 7,500 microseconds

Code

Transmiter: https://github.com/leonyuhanov/ESP32_ESPIDF_ESPNOW/tree/master/espnowv1
Receiver: https://github.com/leonyuhanov/ESP32_ESPIDF_ESPNOW/tree/master/espnowv2

[leonyuhanov]

Issue resolved by setting a specific data rate:

1. In menuconfig DISABLE BOTH Components => Wi-Fi => "WiFi AMPDU TX" & "WiFi AMPDU RX"
2. Include "esp_private/wifi.h"

Add this line to your code to set the fastest data rate:
esp_wifi_internal_set_fix_rate(ESPNOW_WIFI_IF, 1, WIFI_PHY_RATE_MCS7_SGI);

Im now getting under 700 Microsecond round trip times!

[chegewara]

Im not using ESP NOW, but i think this could be implemented in library somehow. @projectgus

[leonyuhanov]

Im not using ESP NOW, but i think this could be implemented in library somehow. @projectgus
You can use this on any interface the function is :

esp_wifi_internal_set_fix_rate(wifi_interface_t ifx, bool en, wifi_phy_rate_t rate);

[chegewara]

@leonyuhanov this header is not suppose to be used by the users:
"esp_private/wifi.h"

[leonyuhanov]

@leonyuhanov this header is not suppose to be used by the users:
"esp_private/wifi.h"

Understood. What do you suggest? This topic has been sitting open for over a month with no input...

[projectgus]

@leonyuhanov If the workaround that you're using works now then I suggest continuing to use it. Possibly the internal API will change in the future and this workaround will stop working, possibly it won't.

I'm going to reopen the issue. It's assigned to a WiFi team engineer for them to take a look at. We want ESP-NOW performance on ESP32 to be similar or better than ESP8266. It may take a while before an official solution is available, though.

[leonyuhanov]

@projectgus Thankyou!!!

[zhangyanjiaoesp]

@leonyuhanov Sorry for reply late. You can call esp_wifi_internal_set_fix_rate() to set the specific rate. And we will investigate the difference between esp32 and 8266.

[zhangyanjiaoesp]

@leonyuhanov
Both ESP32 and ESP8266 use 1Mbps to send data, if the data len = 250Bytes, then the RTT time will be (250*8/1M)*2 = 4ms. How did you test to get the under 1 millisecond round trips time?

[leonyuhanov]

I sent a packet from 1 node with the time stamp, then waited for it to come back from node 2, then did the math. FYI yiu can change the data rate as mentioned in the coments above

[zhangyanjiaoesp]

Yes, I can use esp_wifi_internal_set_fix_rate() to set a higher data rate.
I 'm curious to know why the 8266's round trips time is under 1 millisecond before the modification.

[leonyuhanov]

the 8266 doesnt have these features in the backend on the espidf sdk, they r new to the esp32. the esp8266 was set to full speed

[zhangyanjiaoesp]

I'm sorry, I think you didn't understand my question.
In your first post, you have the following description:

This seems rather slow compared to the ESP8266 (even in the Arduino env) can get under 1 millisecond round trips like this.

My question is how you get this result?
And what's the meaning for full speed ?

[leonyuhanov]

The esp8266 arduino sdk has no bgn chanel bandwith limit or any way to change them, thus it runs at full abgn speed (150mbit if ure ap supports it) the esp32 skd gives you the abiloty to change speed from 1mb all the way up to full speed. read uo the sdk on the espressif web page

[zhangyanjiaoesp]

I got it, thank you for your explanation. So you use 150Mbps to test the 8266 at first.

[leonyuhanov]

Correct although i cant confirm it was acyualy 150mbs im just assuming that was the top speed based on bgn specs. FYI feel free to look through my examples of this for Arduino here https://github.com/leonyuhanov/ESP-NOW-TX-RX and espidf here https://github.com/leonyuhanov/ESP32_ESPIDF_ESPNOW

[zhangyanjiaoesp]

Thanks very much for your examples.

[nemonote01]

Hi @leonyuhanov

Please replace libpp.a and try again. send me the log.

https://github.com/esp8266/Arduino/tree/d24a358817f0c11f7087b3c8acbf24629b15998f/tools/sdk/lib

Thanks!

libpp.zip

[leonyuhanov]

@Junhao-Espressif Will give it ago this week & get back to you

[leonyuhanov]

@Junhao-Espressif Can you please confirm something for me:
Am I replacing the libpp.a file in the Ardunio SDK or the ESPIDF?

[leonyuhanov]

FYI i replaced it in the Ardunio SDK and got this (i put the error log in a file)

https://github.com/leonyuhanov/ESP32_ESPIDF_ESPNOW/blob/master/ErrorLog

[nemonote01]

Hi @leonyuhanov

[leonyuhanov]

OK done as advised. failed to compile with these errors
https://github.com/leonyuhanov/ESP32_ESPIDF_ESPNOW/blob/master/ErrorLog

[nemonote01]

https://github.com/esp8266/Arduino

[nemonote01]

Please do not compile ESP32 example with ESP8266 SDK

[leonyuhanov]

@Junhao-Espressif this issue has nothing to do with the 8266? Its specifically an issue with the ESP32 as mentioned originally. I have no speed constraints with the esp8266

[leonyuhanov]

@Junhao-Espressif As I understand it, the register 0x3ff20c00 contains a MICROSECOND timer. Seems to be pretty on point with my timing system. No idea why reading it slows down the round trip so much though. Without it im averaging 1000 micros. Can you elaborate on what your trying to ascertain?

[leonyuhanov]

@Junhao-Espressif
Did another test
This is with your reg call
Avg Round trip time is 3995 Micros

ABOUT TO SEND		0x2591a84
	Sent [77]
DATA RECEIVED		0x259272e
	Received [77]	Took	[3237]micros
		MinRTT	3047	MaxRTT	12529	AVG RTT	3995

Without the reg call avg RTT is 1227micros

Sent [77]
	Received [77]	Took	[868]micros
		MinRTT	826	MaxRTT	3218	AVG RTT	1227

[nemonote01]

Hi @leonyuhanov

Could you test again:
uint32_t a1 = (*(volatile uint32_t )(0x3ff20c00));
//add send and receive code
uint32_t a2 = ((volatile uint32_t *)(0x3ff20c00));
printf("time: %d\n", a2-a1);

[leonyuhanov]

@Junhao-Espressif You can do the math yourself mate :)
Its blatantly obvious the times im getting are identical to the micros() calls
EG:
Register Before SEND = 0x2591a84
Register After RCPT = 0x259272e
0x259272e - 0x2591a84 = 0xCAA (DEC:3242)
My micros maths gives use 3237 which is 5ms off

Can you please elaborate on what this is trying to achieve for your troubleshooting?

[nemonote01]

Hi @leonyuhanov

I want to prove that ESP8266 and ESP32 take the same time.

ESP8266 and ESP32 both send and receive messages at a rate of 1Mbps

The package sent by espnow contains the following content;
PLCP header (16bytes) + MAC header (24bytes) + Payload (250bytes) = 290 bytes

290bytes * 8 / 1Mbps = 2.32ms.

Sending and receiving data requires at least 2.32 * 2 = 4.64ms

[nemonote01]

The data length is 100bytes in ESP8266？

const byte dataLength=100;

[leonyuhanov]

@Junhao-Espressif I understand,

1. Do you not trust the micros() call over reading the register directly?
2. If the esp8266 and the esp32 DO both use the same data rate (by default), then why does the esp32 packet take on average 6ms when the esp8266 takes on average 1.2Ms
3. The data SIZE a user sets to TX is not relevant because the ESPNOW frame has a SET size no matter what you TX, your just limited to 200B

[leonyuhanov]

OK just for arguments sake i have updated the code as you have suggested:
https://github.com/leonyuhanov/ESP-NOW-TX-RX/tree/master/DualModeMaster
https://github.com/leonyuhanov/ESP-NOW-TX-RX/tree/master/DualModeSlave

• Sending 200byte payload
• Using your register calls instead of micros()

        Sent [76]
	Received [76]	Took	[1134]micros
		MinRTT	1100	MaxRTT	4129	AVG RTT	1567
	Sent [77]
	Received [77]	Took	[3278]micros
		MinRTT	1100	MaxRTT	4129	AVG RTT	1589
	Sent [78]
	Received [78]	Took	[1098]micros
		MinRTT	1098	MaxRTT	4129	AVG RTT	1583

Rund trip times are still averaging at 1.5Mili seconds not 6 or over. Infact the largest im getting is 4.1milis

[zhangyanjiaoesp]

@leonyuhanov
Thanks very much for taking so many tests. For the difference between ESP32 and ESP8266, I think we need to test locally and find the root cause.

For the ESP32, as mentioned earlier, you can call esp_wifi_internal_set_fix_rate() to inprove the rate.

[leonyuhanov]

@zhangyanjiaoesp No worries. The esp8266 also has some similar rate fix functions but i cant get any of them to do anything for the espnow configuration. But this isnt relevant to this issue

[zhangyanjiaoesp]

Since this issue has been solved, I think you can close it. Thank you once again.

[leonyuhanov]

@zhangyanjiaoesp according to @projectgus this isnt a resolution. He reopened it. I think you need to make this part of the lib

[zhangyanjiaoesp]

@leonyuhanov @projectgus
We have discussed about the esp_wifi_internal_set_fix_rate() internally, this is not a workaround, and it is the correct solution.

[leonyuhanov]

@zhangyanjiaoesp ok good news

[neel]

Issue resolved by setting a specific data rate:

1. In menuconfig DISABLE BOTH Components => Wi-Fi => "WiFi AMPDU TX" & "WiFi AMPDU RX"
2. Include "esp_private/wifi.h"

Add this line to your code to set the fastest data rate: esp_wifi_internal_set_fix_rate(ESPNOW_WIFI_IF, 1, WIFI_PHY_RATE_MCS7_SGI);

Im now getting under 700 Microsecond round trip times!

Where is ESPNOW_WIFI_IF declared ? I only see ESP_IF_WIFI_AP in esp_interface.h

[fyras1]

Issue resolved by setting a specific data rate:

1. In menuconfig DISABLE BOTH Components => Wi-Fi => "WiFi AMPDU TX" & "WiFi AMPDU RX"
2. Include "esp_private/wifi.h"

Add this line to your code to set the fastest data rate: esp_wifi_internal_set_fix_rate(ESPNOW_WIFI_IF, 1, WIFI_PHY_RATE_MCS7_SGI);

Im now getting under 700 Microsecond round trip times!

Hi
I came across the same problem but fixing the wifi rate doesn't change anything. I even cloned your code , disabled AMPDU TX&RX and then ran it and still getting 9-10 ms on my esp32s2. is there anything else im not paying attention to?

[oldmud0]

Where is ESPNOW_WIFI_IF declared ? I only see ESP_IF_WIFI_AP in esp_interface.h

For anyone who is still looking for an answer to this, ESPNOW_WIFI_IF is ESP_IF_WIFI_STA or ESP_IF_WIFI_AP according to espnow_example.h. Though, this function will only accept WIFI_IF_STA or WIFI_IF_AP for the wifi_interface_t parameter, so you should use one of those two.

[newtocodi]

I have been editing my code, but it seems not working. So, I want to determine how to measure latency/delay between the RX/TX and display it on the TX serial monitor.
Please, I am new to coding. I have uploaded my code for advice.
`
#include <esp_now.h>
#include <WiFi.h>
#include <ADXL345_WE.h>
#include <SPI.h>

#define CS_PIN 5 // Chip Select Pin
bool spi = true; // flag that SPI shall be used

/unsigned long wasMillis = millis(), elapsedMillis = 0;
float loopCount = 0;/

#define CHANNEL 1

// Replace with your receiver MAC address
uint8_t broadcastAddress[] = { 0x13, 0x93, 0xC5, 0xB6, 0x69, 0x92 };

ADXL345_WE myAcc = ADXL345_WE(CS_PIN, spi);
// Create a struct to hold accelerometer data
typedef struct struct_message {
int id;
float x;
float y;
float z;

} struct_message;

struct_message myData;
esp_now_peer_info_t peerInfo;

//callback when data is sent
void OnDataSent(const uint8_t *mac_addr, esp_now_send_status_t status) {
Serial.print("\r\nLast Packet Send Status:\t");
Serial.println(status == ESP_NOW_SEND_SUCCESS ? "Delivery Success" : "Delivery Fail");
}

void setup() {
Serial.begin(2000000);
//myAcc.setSPIClockSpeed(5000000);
Serial.println("ADXL345_Sketch - Basic Data");
if (!myAcc.init()) {
Serial.println("ADXL345 not connected!");
while (1)
;
}
WiFi.mode(WIFI_STA);
if (esp_now_init() != ESP_OK) {
Serial.println("Error");
return;
}
esp_now_register_send_cb(OnDataSent);
memcpy(peerInfo.peer_addr, broadcastAddress, 6);
peerInfo.channel = 1;
peerInfo.encrypt = false;
if (esp_now_add_peer(&peerInfo) != ESP_OK) {
Serial.println("Failed to add peer");
return;
}
myAcc.setDataRate(ADXL345_DATA_RATE_3200);
myAcc.setRange(ADXL345_RANGE_2G);

}

void loop() {

xyzFloat raw = myAcc.getRawValues();
xyzFloat g = myAcc.getGValues();

// Wait for 1/3200 seconds (sampling period)
//delayMicroseconds(313);
/*elapsedMillis = millis() - wasMillis; // Calc elapsed time
loopCount += 1; // Count loop passage
if (elapsedMillis = 1000) { // If one second has elapsed
Serial.print("Average time per sample = ");
Serial.print(loopCount / elapsedMillis, 2);
Serial.print("ms for ");
Serial.print(loopCount);
wasMillis = millis(); // Reset for next group
loopCount = 0.0;
} */

myData.id = 1;
myData.x = g.x;
myData.y = g.y;
myData.z = g.z;
esp_err_t result = esp_now_send(broadcastAddress, (uint8_t *)&myData, sizeof(myData));
//Serial.print(micros());
// Serial.print(",");
/Serial.print(g.x);
Serial.print(",");
Serial.print(g.y);
Serial.print(",");
Serial.println(g.z);/
}
`

[newtocodi]

This is the receiver code
`#include <espnow.h>
#include <ESP8266WiFi.h>

#define CHANNEL 1
/unsigned long wasMillis = millis(), elapsedMillis = 0;
float loopCount = 0; ///

// Structure example to receive data
// Must match the sender structure
typedef struct struct_message {
int id;
float x;
float y;
float z;

} struct_message;

// Create a struct_message called myData
struct_message myData;

// Create a structure to hold the readings from each board
struct_message rightGlove;
struct_message leftGlove;

// Create an array with all the structures
struct_message boardsStruct[3] = { rightGlove, leftGlove };

// callback function that will be executed when data is received
void OnDataRecv(uint8_t *mac_addr, uint8_t *incomingData, uint8_t len) {
char macStr[18];
//Serial.print("Packet received from: ");
snprintf(macStr, sizeof(macStr), "%02x:%02x:%02x:%02x:%02x:%02x",
mac_addr[0], mac_addr[1], mac_addr[2], mac_addr[3], mac_addr[4], mac_addr[5]);
//Serial.println(macStr);
memcpy(&myData, incomingData, sizeof(myData));
//Serial.printf("Board ID %u: %u bytes\n", myData.id, len);
// Update the structures with the new incoming data
boardsStruct[myData.id - 1].x = myData.x;
boardsStruct[myData.id - 1].y = myData.y;
boardsStruct[myData.id - 1].z = myData.z;
/elapsedMillis = millis() - wasMillis; // Calc elapsed time
loopCount += 1; // Count loop passage
if (elapsedMillis >= 1000) { // If one second has elapsed
Serial.print("Average time per sample = ");
Serial.print(loopCount / elapsedMillis, 2);
Serial.print("ms for ");
Serial.print(loopCount);
wasMillis = millis(); // Reset for next group
loopCount = 0.0;
}/

//Serial.print("acc_x:");
Serial.print(micros());
Serial.print(",");

Serial.print(boardsStruct[myData.id - 1].x);
Serial.print(",");
Serial.print(boardsStruct[myData.id - 1].y);
Serial.print(",");
Serial.println(boardsStruct[myData.id - 1].z);
}

void setup() {
//Initialize Serial Monitor
Serial.begin(2000000);

//Set device as a Wi-Fi Station
WiFi.mode(WIFI_STA);

//Init ESP-NOW
if (esp_now_init() != ERR_OK) {
Serial.println("Error initializing ESP-NOW");
return;
}

// Once ESPNow is successfully Init, we will register for recv CB to
// get recv packer info
esp_now_register_recv_cb(OnDataRecv);
}

void loop() {
// Wait for 1/3200 seconds (sampling period)
//delayMicroseconds(10);

}`

[JBKingdon]

Since this is one of the top search results for this issue, I thought it might be worth adding a note.

The relevant APIs have changed since this issue was raised, and using esp_wifi_internal_set_fix_rate is no longer effective. Something along the following lines can be used instead:

  esp_now_rate_config_t peerConfig;

   // WIFI_PHY_MODE_LR,   /**< PHY mode for Low Rate */
   // WIFI_PHY_MODE_11B,  /**< PHY mode for 11b */
   // WIFI_PHY_MODE_11G,  /**< PHY mode for 11g */
   // WIFI_PHY_MODE_HT20, /**< PHY mode for Bandwidth HT20 */
   // WIFI_PHY_MODE_HT40, /**< PHY mode for Bandwidth HT40 */
   // WIFI_PHY_MODE_HE20, /**< PHY mode for Bandwidth HE20 */

   peerConfig.phymode = WIFI_PHY_MODE_HT20;    // needs a coordinated changed with secondChannel to use HT40. HE20 is refused

   peerConfig.rate = WIFI_PHY_RATE_MCS4_LGI;   // rates are MCS0 through MCS7. Long or short guard interval LGI vs SGI
   peerConfig.ersu = false;
   ESP_ERROR_CHECK( esp_now_set_peer_rate_config(broadcastAddress, &peerConfig) ); // set the address of the specific peer you want to configure if using unicast

[programmeddeath1]

@JBKingdon I get the error -

run_now.ino:24:1: error: 'esp_now_rate_config_t' does not name a type; did you mean 'esp_eth_config_t'?
 esp_now_rate_config_t rateConfig;

Is there any additional library that needs to be imported other than the following two -

#include <esp_now.h>
#include <WiFi.h>

I am using arduino ide/platformio to burn to esp32

[JBKingdon]

esp_now_rate_config_t is defined in esp_now.h (at line 100 in the version I have, 3.50102.240122 of framework-espidf)

[nanshenwei]

Hi @leonyuhanov

I have tried with my example. But it can not send and receive one pkt in 1ms.

I'm sorry to ask a question unrelated to this issue, but I really need: Is this a wireless packet capture tool? What tool is this?

[leonyuhanov]

@nanshenwei i dont know what that screenshot is from, i dis not post it :)
This iasue was closed yonks ago, and the code i ahve in my repo is old and likely isnt updated to work with the latest espidf.

[nanshenwei]

@nanshenwei i dont know what that screenshot is from, i dis not post it :) This iasue was closed yonks ago, and the code i ahve in my repo is old and likely isnt updated to work with the latest espidf.

Thank you for such a prompt reply. I apologize for disturbing you, as I didn't notice that your ID was mentioned in the referenced reply. I think I should have mentioned @nemonote01, because the screenshot came from his reply.

Additionally, I am using ESPNOW to transmit data at 1ms intervals, sending a 200-byte payload data to a broadcast MAC address every 1ms, then flipping an LED. I've noticed that the LED does not flip as expected (I was expecting an oscilloscope to show about a 500Hz square wave), the waveform frequency appears to be between 400-500Hz momentarily.(sometimes much lower than 500Hz)

So, I want to check where the problem might be (perhaps my wireless RF environment is too crowded). I'm also unsure under what conditions the ESPNOW sends the callback function after transmission completion.

I have already turned "WiFi AMPDU TX" & "WiFi AMPDU RX" off, and successfully executed esp_wifi_config_espnow_rate(WIFI_IF_STA, WIFI_PHY_RATE_MCS7_SGI);

[zhangyanjiaoesp]

@nanshenwei Please create a new ticket to report your issue. The screenshot seems to be a wireless packet capture from Omnipeek, you can also capture packets using the Wireshark.

[nanshenwei]

@nanshenwei Please create a new ticket to report your issue. The screenshot seems to be a wireless packet capture from Omnipeek, you can also capture packets using the Wireshark.

Thanks so much

[programmeddeath1]

@nanshenwei please check this issue I had raised in esp_now - espressif/esp-now#115.

I tried to publish at a rate of 10ms, but esp_now cannot publish at a rate of less than 20ms. What happens it at frequencies of more than 50Hz, the delay's keep on going up. You cannot publish at a rate less than 20ms. based on the software limit i believe.

Please do tell me if you find a better solution.