TPM2 Playback from LittleFS and SD

[constant-flow]

Changes

• new esp32-eth-sd env
• tpm2 playback from LittleFS and SD
• play recording as live-mode
• made tpm2 uploadable via /edit interface

Related

• Issue #627

[blazoncek]

This is cool. And clean. 👍

[blazoncek]

I will do my best to get this merged. Not in 0.13 though.

[ElToberino]

I hope it will be possible to merge this PR soon.
I like the support of ArtNet/sACN very much because it enables driving large matrixes easily via programs like Jinx! (Without the need of configuring the matrixes via WLED.)
Adding the support of tpm2-playing from SD would increase the possibilties of WLED into a new dimension - with an "infinte" number of custom standalone effects/Shows recorded with Jinx! or similar programs.
Especially used in combination with the timers/schedules this could be really be killer feature and would make WLED even more unique.

[ElToberino]

Meanwhile I compiled V0.13-b6 with these changes.
Everything works fine - it's really a great feature. I wonder if it would make sense to make the variable "recordingRepeats" (in wled00/tpm2record.cpp) configurable via JSON. I know it's possible just to use the playlist feature but I think the confugurable value would be a simpler and straighter solution.

[blazoncek]

Hi @constant-flow

I was finally able to delve into the code, examine it and prepared a few suggestions.

First of all, idea is great and has a lot of potential - as a usermod.

If you can rewrite it as a usermod (you may check existing usermods: multi relay, temperature, PWM fan, etc) it has a greater chance of being merged as it becomes optional for users. It will also allow for greater flexibility as the usermod framework allows usermods to tap into core WLED without changing original source.
Even though this may seem difficult at first glance it is rather simple (no need to change wled.cpp or json.cpp, etc). Once you put it into basic shape I can help you fine-tune (via draft PR).

Second, I liked the idea of employing SD card.

Unfortunately it turns out that there are 2 ways to connect SD card to ESP: MMC & SPI. Enhancing (usermod) with the ability to use either would be best as it seems that higher level API for both is the same. The actual connection may be selected using usermod's configuration page.

I am sorry this has taken so long to review the code but I had other priorities unfortunately.

I hope you will agree with the idea of rewriting your enhancement as a usermod.

Best!

[blazoncek]

No comment?
Shall we then just close the PR?

[constant-flow]

Currently on vacation 🌴 ☺️
Let me check later if i find some time.

Thanks for your suggestions though I'm still not so sure how easy an integration as a usermod would be.

[blazoncek]

No problem.
It is actually quite easy to use usermod framework.

BTW take a look at my fork, audioreactive-prototype (or 2D-support) branch for the inspiration.

[constant-flow]

I had a look into the docs again and finally understood how Usermods can alter the capabilities of the API-Commands.
After a quick test I can verify that the usermod-route doesn't break a major feature (the only thing so far is when selecting a recording in the /edit route, I was able to peek into the animation ... maybe we can add this to usermods feature-list too ... but thats a nice to have)

Also now it relies on the REALTIME_MODE_GENERIC instead of a more descriptive one.

My plan now is to clean it up a bit more and abstract it from the recording format, to enable not only TPM2 but also any other format future devs want to playback like fseq, so they can easily support their format with this usermod.

{"playback":
   {
      "file":"/record.tpm2",
      "format":"tpm2"
   }
}

Concerning the two ways to access the SD. I think I could integrate that via the usermod settings (if necessary). Do you have a link to projects using the MMC and one using SPI, so I can have a look whats the general difference in code to access files on them?

Update: Link to first version of usermod (room for improvement still), still missing the SPI way to interact with the SD card

[blazoncek]

Hi. Checked your usermod prototype linked above.
Please utilise addToConfig() and readFromConfig() to store/load SD card type and GPIO selection as well as any other configurable option.
Do the initialization in setup() (consider also that GPIO pins or type may change during runtime - do not require reboot if possible).
There is plenty of helpful information in existing usermods provided above.
Unfortunately I do not have any information on how to implement MMC and/or SPI interface other than a few Google links.

[constant-flow]

Please check update here about new progress

Anything missing for a PR?

[constant-flow]

@ElToberino thx for checking and reporting errors.

1.) In the usermod descrpition you wrote that default fps rate is 25. But when I played a tpm2-file without defining a fps rate, the animation (recorded with Jinx! @ 25 fps) was way too fast. Manually setting the fps rate to 25 via preset worked and the animation was played with the proper speed.

Looks like I forgot a default value from what you report. Will have a look. @harueg the TPM2 doesn't specify a framerate via the header. It's 25 FPS by convention from jinx! I think the TPM2 "standard" 🇩🇪 doesn't even define a default framerate

2.) Skipping / Leaving the real time mode didn't work. In opposite to the web interface which behaved properly and left the realtime mode, the leds still continued playing the animation - flickering and overlayed by the now chosen WLED effect.

I will check on that, I hope I'll find some time to investigate. Last time the interface was still showing it being in realtime mode even though it stopped it via exitRealtime() but i didn't test it with concurrent animations.

[constant-flow]

The more I look into this messed up PR the more I feel I should close this PR and open 2 new PRs

1. playback-recordings (TPM2)
2. SD card support

I hope i find the time to do it soon. Maybe then, things will be merged into main as usermod

[harueg]

Here is the plan for the protocol, I don't think everything has been implemented yet.
https://www.ledstyles.de/content/index.php?attachment/28200-tpm2-specs-v1-0-2013-ger-pdf/
A frame rate isn't mentioned.
And thank you !

[ElToberino]

@constant-flow:
I took a deeper look at the code and found a solution for the two issues in usermod_playback_recordings.h:

1. change code in line 56
   #define PLAYBACK_FRAME_DELAY 40
   // set default fps rate: calculation: 1000/PLAYBACK_FRAME_DELAY = fps -> 1000/40 = 25fps

2. comment out lines 273-276. This is no longer needed and will be done later.
   add following code in line 411:
   if (realtimeOverride == REALTIME_OVERRIDE_ALWAYS) realtimeOverride = REALTIME_OVERRIDE_ONCE;
   // workaround - not 100% satisfying, but necessary with current real time mode web interface
   if (realtimeOverride == REALTIME_OVERRIDE_ONCE) { exitRealtime(); playbackFile.close(); file_clearLastPlayback(); }
   // stopping file playback and leaving real time mode on web interface request

With these modifications everything works as intendend.
I also think it would be better to start a new PR. Your tpm2 feature is really essential and should be merged asap.

@harueg:
I don't think the tpm2 protocol will ever be changed or extended. That is not required and would break many applications. It's widely used on many players and devices - most of them are programmed for a fps rate of 25 because of the Jinx! output rate.

[constant-flow]

@ElToberino thx for investigating and solving this issue 🙇 . I merged the current main into my branch (v14) and added your fixes. It's still on my branch, from what I tested the fixes work, maybe check it.

I'll try PR both mods individually soon, didn't have enough time today.

[constant-flow]

So i created two PRs to close this one

PR1 : Playback of .tpm2
PR2 : SD card support: MMC or configurable SPI