SyntaxError and sometimes a reset when changing presets in 2D.

[MOVE1904]

What happened?

I made an illuminated sign for the back under my jersey for my favorite club.

There are 540 LEDs arranged irregularly on the sign. I configured WLED so that I use this in a 2D Matrix.
When I choose a presets I often got a syntax error or a reset of the ESP32, even with the simplest configuration.

0923.1.mp4

To Reproduce Bug

Use the attached cfg and preset file.

preset_cfg.zip

Select the “Start” preset several times.

Expected Behavior

I expected that there would be no reset or error when changing the preset.

Install Method

Self-Compiled

What version of WLED?

WLED v0.14.0-b5 "Hoshi" (Build 2309100)

Which microcontroller/board are you seeing the problem on?

ESP32, ESP32-S3

Relevant log/trace output

---- Opened the serial port COM4 ----
WS FileRead: /index.htm
Not-Found HTTP call:
URI: /skin.css
WS FileRead: /skin.css
JSON buffer locked. (17)
JSON buffer size: 7890 for request: 6
JSON content length: 5285
JSON buffer released. (17)
JSON buffer locked. (17)
JSON buffer size: 4801 for request: 8
JSON content length: 2184
JSON buffer released. (17)
JSON buffer locked. (17)
JSON buffer size: 11798 for request: 3
JSON content length: 6665
JSON buffer released. (17)
Not-Found HTTP call:
URI: /presets.json
WS FileRead: /presets.json
WS client connected.
JSON buffer locked. (12)
JSON buffer size: 10967 for WS request (6665).
heap 181664
Sending WS data to a single client.
JSON buffer released. (12)
JSON buffer locked. (14)
JSON buffer released. (14)
JSON buffer locked. (17)
JSON buffer size: 11799 for request: 3
JSON content length: 6673
JSON buffer released. (17)
JSON buffer locked. (12)
JSON buffer size: 10968 for WS request (6673).
heap 154840
Sending WS data to multiple clients.
JSON buffer released. (12)
Loop took 45ms.
Usermods took 0ms.
Strip took 7ms.
Local time: 13:19
Local time: 13:20
JSON buffer locked. (14)
JSON buffer released. (14)
JSON buffer locked. (17)
JSON buffer size: 11799 for request: 3
JSON content length: 6673
JSON buffer released. (17)
JSON buffer locked. (12)
JSON buffer size: 10968 for WS request (6673).
heap 173916
Sending WS data to multiple clients.
JSON buffer released. (12)
Loop took 68ms.
Usermods took 0ms.
Strip took 8ms.
---DEBUG INFO---
Runtime: 30000
Unix time: 1695475204,000
Free heap: 176456
Wifi state: 3
State time: 0
NTP last sync: 999000000
Client IP: XXX.XXX.XXX.XXX
Loops/sec: 1727
Loop time[ms]: 0/721
UM time[ms]: 0/9
Strip time[ms]: 0/160
Segments: 16 -> 1088B
Modes: 4*187=748B
Data: 4*187=748B
Map: 2*1024=2048B
Buffer: 3*1024=3072B
---- Closed the serial port COM4 ----

Anything else?

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[MOVE1904]

preset_cfg_2.zip

ledmap1.zip

Here is a bigger configuration with more frequent errors. Switch between preset 23 (PixelsFire) and preset 14 (Start).

[blazoncek]

For JSON errors you need browser's developer tools and see console log.

[blazoncek]

Also preset file in the movie does not correspond to attached presets.

[MOVE1904]

Also preset file in the movie does not correspond to attached presets.

Did You use the preset file from my first post?

preset_cfg.zip

[MOVE1904]

For JSON errors you need browser's developer tools and see console log.

like this?

[blazoncek]

Now check each GET JSON response to see content. One will have error.

[MOVE1904]

No JSON. Only 1 CSS maybe?

[MOVE1904]

O.K. I got it. CSS is enable custom CSS in User Settings. So I diabled it.

So where can I find the GET JSON with an error?

[blazoncek]

Any file with JSON type. I.e. effects, fxdata, etc.

[MOVE1904]

This looks strange

[blazoncek]

Yes. That's corrupt JSON.
Why that happens is still a mystery.

[blazoncek]

If you are not using ESP8266 then check if you flashed your ESP with non-standard flash or CPU speed (lowering the values to standard specs).

If you are using ESP8266 I've seen such behaviour after we upgraded core to 3.1.2 but is transient in nature. Some responses may be corrupted. I am guessing some out-of-order new/free behaviour of AsyncWebServer or its components.

[MOVE1904]

I'm flashing the ESP32 with the standard option of VS Code. Upload port is set to "auto".
I try it with 6 different ESP32-WROOM-32 an 2 different ESP32-S3-WROOM-1.
Always the same errors. Most of them with position 5533.

Here ESP32-S3

And here ESP32-WROOM-32

[blazoncek]

Looks like the issue is connected to the number of segments or the size of segment data.

[blazoncek]

Additional info:

This is UI only problem as the HTTP response contains corrupt data (HTTP headers inserted into HTTP response payload).

The issue (most likely) arises from the following code in json.cpp serveJson():

  AsyncJsonResponse *response = new AsyncJsonResponse(&doc, subJson==JSON_PATH_FXDATA || subJson==JSON_PATH_EFFECTS); // will clear and convert JsonDocument into JsonArray if necessary
  JsonVariant lDoc = response->getRoot();
...
  response->setLength();
  request->send(response);

Looks very much similar to what I've observed on ESP8266. Reducing the number of segments to 14 alleviates the problem. WS responses are not affected.

[MOVE1904]

Looks like the issue is connected to the number of segments or the size of segment data.

I deleted 3 segments. Now I can not reproduce the error. It doesn't matter which 3 segments I delete, so I think the maximum number of segments when using ledmap is 13 !?!

[blazoncek]

It is not ledmap related. At least I think so.

[MOVE1904]

It is not ledmap related. At least I think so.

You're right. I mean 2D matrix. My preset does not use a ledmap file.

[blazoncek]

Similar to #3312

[softhack007]

@blazoncek this is the first "smoking gun" with a full debug build - looks like stack overflow. I'll continue to investigate.

***ERROR*** A stack overflow in task loopTask has been detected.
abort() was called at PC 0x4008a890 on core 1

ELF file SHA256: 0000000000000000

Backtrace: 0x4008a4d4:0x3ffafe00 0x4008a879:0x3ffafe20 0x4008a890:0x3ffafe40 0x4008ce17:0x3ffafe60 0x4008ebc0:0x3ffafe80 0x4008eb76:0x3ffc0ee8 |<-CORRUPTED
  #0  0x4008a4d4:0x3ffafe00 in invoke_abort at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c:699
  #1  0x4008a879:0x3ffafe20 in abort at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c:699
  #2  0x4008a890:0x3ffafe40 in vApplicationStackOverflowHook at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c:699
  #3  0x4008ce17:0x3ffafe60 in vTaskSwitchContext at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/freertos/tasks.c:3507
  #4  0x4008ebc0:0x3ffafe80 in _frxt_dispatch at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/freertos/portasm.S:406
  #5  0x4008eb76:0x3ffc0ee8 in _frxt_int_exit at /home/cschwinne/esp32-arduino-lib-builder/esp-idf/components/freertos/portasm.S:206

Rebooting...

[blazoncek]

Debug and stack use are enemies.
Funny there isn't any WLED code involved. Makes me wonder where that happens.

[softhack007]

Debug and stack use are enemies. Funny there isn't any WLED code involved. Makes me wonder where that happens.

Seems you are right - it only happens with build_type = debug; crazy workaround is to double arduino stacksize with -DARDUINO_LOOP_STACK_SIZE=16896.

So back to zero. At least I can reproduce the UI JSON error.

[blazoncek]

At least I can reproduce the UI JSON error.

I can do that, but not consistently.

[softhack007]

@blazoncek can you cross-check something?

I've changed this line in ws.cpp
https://github.com/Aircoookie/WLED/blob/26e766ee1976491fbce698b24ce3a65d3850a855/wled00/ws.cpp#L111

to

  size_t len = measureJson(doc)+1;

With this change I do not get the UI error any more. But not sure if this is a real fix, or its just hiding the symptoms. What do you think?

the docs for measureJson say:

Computes the length of the minified JSON document that serializeJson() produces, excluding the null-terminator.

So if we want to serialise to a string, one more char is needed for the terminating '\0'.

[softhack007]

These patches for asyncWebserver might help

• (esp32) Fix #884, protect list concurrent access with mutex me-no-dev/ESPAsyncWebServer#887
• Fix #888, AsyncEventSourceMessage::send() me-no-dev/ESPAsyncWebServer#889
• Fix uninitialized frame number me-no-dev/ESPAsyncWebServer#829
• Fix of not initialized structure. me-no-dev/ESPAsyncWebServer#668
• Overload handling: low heap and high number of open connections me-no-dev/ESPAsyncWebServer#1116

[blazoncek]

Would it be possible that these two (#3295) are related?

[blazoncek]

I invite you to test 0_15 branch and possibly suspend branch, both employ some additional stuff to try to mitigate the cause.
If I do not hear back within a month (either confirming that issue is gone or persists) I am closing as unresolved.

[MOVE1904]

I still get the error "Connection to light failed!

0_15 branch:

suspend branch:

Additionally I got the error "Error 8: Effect RAM depleted!"

0_15 branch:

But overall the errors have become fewer.

[blazoncek]

The only explanation (suggested by your last screenshot) is that you run out of heap. Reduce your segment complexity or use ESP with PSRAM.
It may also help if you show browser's console log.

[MOVE1904]

Here is a simpler configuration of my Christmas lights:

WLED_cfg_preset.zip

[blazoncek]

One more thing.
Reviewing configuration (again), I am baffled with total LED count of 1024 while the sum of all lengths yields only 540. How is that so?
And if you are using matrix (as evident from config) and custom ledmap (ledmap1.json) where is your default ledmap?
Do you switch ledmaps often?

[blazoncek]

And what is the last response from WLED (network tab, click on the last message).

[MOVE1904]

One more thing. Reviewing configuration (again), I am baffled with total LED count of 1024 while the sum of all lengths yields only 540. How is that so? And if you are using matrix (as evident from config) and custom ledmap (ledmap1.json) where is your default ledmap? Do you switch ledmaps often?

In order to not use so many segments and still be able to display a graphic, I created an LED-MAP. Since I only use 2 colors, I first have all white and then all blue LEDs in the LED-MAP.

And what is the last response from WLED (network tab, click on the last message).

Sorry, I'm not that good with Chrome. I hope that's correct. It is with the simpler configuration.

[MOVE1904]

In general, I have to say that I see these errors more and more often in my projects. It doesn't matter whether I use an ESP32 or D1Mini and whether I use LED MAPs or not.
It doesn't matter whether I use a ready to use image or whether I compiled it myself.
These errors even appear in the Android app.

[blazoncek]

In order to not use so many segments and still be able to display a graphic, I created an LED-MAP. Since I only use 2 colors, I first have all white and then all blue LEDs in the LED-MAP.

That does not explain total having 1024. There is no way you could achieve that using UI and the configuration you posted.

I give up. You'll need to debug your issue on your own and if you indeed find a bug please report back.

[MOVE1904]

I give up, too.
Maybe it's another type of the Fritz!Box error.
I just think it's a shame that you only pay attention to the strange arrangement of my LED lighting, but don't address the fact that the error now appears even in the simplest configurations.
I only implemented the idea with the LED-Map to reduce the crashes. At least the paylists are stable with this solution.