help needed: stream video over WiFI

[HarikrishnanBalagopal]

Overview

Hi, is there a way to stream video over WiFi and display it using this library?
I have a 128x128 (4 64x64 panel) setup. 2 chains 2 panels each.
I can't allocate a 128x128x3 byte array because the ESP32 doesn't have enough memory.
So I have a 64x64x3 byte array that I fill and then use that to read the current frame over the network.
Then I display that image in one of the 4 panels and read the next panel of the current frame.
Frames are drawn using https://github.com/mrfaptastic/ESP32-HUB75-MatrixPanel-DMA/blob/722358ad2d990d0aa36600da95e4b36b740ff5f7/src/ESP32-VirtualMatrixPanel-I2S-DMA.h#L463

Needless to say the frame rate is bad. Around 3-5 frames per second in the best case.

Help needed

• Some example code if anyone has managed to stream video over the network and get decent FPS.
• Some way to access the internal arrays of the library so I can read the data directly into it. This avoids me having to allocate a separate array and run out of memory. The server/client providing the frames usually run on higher end hardware (Desktop PC) and can produce complicated bit patterns if required.

Things I have tried

• Raw TCP socket reads.
• ESP32 as a client pulling frames from a server running on my PC.
• ESP32 as a server with my PC pushing frame to it.

Things I have not tried

• Hacking the library to get access to internal data structures
• Compression. Using a JPEG decompression library like in this video to stream frames https://youtu.be/G6MROvlLeKE

[mrcodetastic]

From the sounds of it you can successfully stream pixel data, but the question is that you can't get the performance you need?

3-5fps seems about right to me. These LED panels have little resemblance to a LCD panel like in the video you linked, as LED panels need more CPU to update the DMA buffer because they're so "dumb".

You could try turning down the library colour depth. Send 16 bit 565 colour instead of 24 bits. Use UDP not TCP.

Probably really need to write a scaled back custom library. And, the ESP32 (the original one) is pretty old as well and not as powerful as the S3.

Or, use a raspberry pi instead.

[drvkmr]

I have been able to stream e131 (sacn) at 60fps on a 64x64 matrix. I don't have the entire code organised but can share a few snippets which can help you. From the 'sending' side, you need a way of sending sacn data (it's similar to artnet). Many softwares can help you do that, and you can write simple softwares yourself if needed as well.

Gist to the specific code which gets and displays data.

E131 library I am using (modified from original).

Finally one thing which took me a long time to realise is that you need a decent router for WiFi streaming. I have a Three 5g router which does a terrible job at this, but a cheap £20 somehow does exactly what I need.

Feel free to ask if you are struggling.

[harissutanrafiq]

I have been able to stream e131 (sacn) at 60fps on a 64x64 matrix. I don't have the entire code organised but can share a few snippets which can help you. From the 'sending' side, you need a way of sending sacn data (it's similar to artnet). Many softwares can help you do that, and you can write simple softwares yourself if needed as well.

Gist to the specific code which gets and displays data.

E131 library I am using (modified from original).

Finally one thing which took me a long time to realise is that you need a decent router for WiFi streaming. I have a Three 5g router which does a terrible job at this, but a cheap £20 somehow does exactly what I need.

Feel free to ask if you are struggling.

Wow 60fps, do you using ethernet module or Wifi ?

[harissutanrafiq]

i have modified E131 library, using async ethernet and its amazing @HarikrishnanBalagopal and thanks for amazing this dma matrix panel lib @mrcodetastic

13 universe 64x32 pixel

WhatsApp.Video.2024-07-10.at.09.48.35.mp4

[aokailin]

我已经使用此库做成功了一个80*40分辨率的WIFI流传输，并通过SIGNALRGB控制它！

[mrcodetastic]

您愿意分享代码吗？

[aokailin]

我的代码很乱，你愿意看的话。它还集成了MDNS、web服务，wifi配网等，仿WLED协议和SIGNALRGB通讯，但是我不知道怎么上传到这个网站，我也才开始登陆这个网站

[aokailin]

您愿意分享代码吗？

我已经上传代码到我的库中，你可以随时下载了

[aokailin]

概述

嗨，有没有办法通过 WiFi 流式传输视频并使用此库显示它？我有一个 128x128（4 个 64x64 面板）设置。2 条链条，每条链 2 个面板。我无法分配 128x128x3 字节的数组，因为 ESP32 没有足够的内存。因此，我填充了一个 64x64x3 字节的数组，然后使用它通过网络读取当前帧。然后，我在 4 个面板之一中显示该图像，并读取当前帧的下一个面板。框架是使用

https://github.com/mrfaptastic/ESP32-HUB75-MatrixPanel-DMA/blob/722358ad2d990d0aa36600da95e4b36b740ff5f7/src/ESP32-VirtualMatrixPanel-I2S-DMA.h#L463

不用说，帧速率很差。在最好的情况下，大约每秒 3-5 帧。

需要帮助

• 如果有人设法通过网络流式传输视频并获得体面的 FPS，则提供一些示例代码。
• 某种方式来访问库的内部数组，以便我可以将数据直接读入其中。这避免了我不得不分配一个单独的数组并耗尽内存。提供帧的服务器/客户端通常在高端硬件（台式电脑）上运行，如果需要，可以生成复杂的位模式。

我尝试过的事情

• 原始 TCP 套接字读取。
• ESP32 作为客户端，从我的 PC 上运行的服务器拉取帧。
• ESP32 作为服务器，我的 PC 将框架推到它上面。

我没有尝试过的事情

• 入侵库以访问内部数据结构
• 压缩。使用 JPEG 解压缩库（如本视频中所示）流式传输帧 https://youtu.be/G6MROvlLeKE

https://github.com/aokailin/HUB75-signalRGB-WIFI

[Facetti]

see https://github.com/Facetti/E1.31-sACN-ESP32-matrix-display