SOF_GUI: Implement GUI and TUI for SOF demonstration #9406

alexb3103 · 2024-08-25T05:01:33Z

Implements a gui and tui that can be used to easily demonstrate SOF on target HW. See README and README-dev for more information on functionality and purpose.

See #9223 for the initial description of the project and overall scope. Most features from that description were met here.

One important goal before this PR is ready to be merged is for this to be tested on a variety of platforms. At this point, the GUI is tested on most NXP platforms, but little other testing has been done yet. Please feel free to comment with any issues that are had on any platform, and I'll work on resolving them ASAP.

A second focus is on understandability of the docs, please let me know if any sections are unclear. A large goal of this project is to allow for new components to be easily tested and demonstrated on target HW, so it needs to be very easy for any SOF developer to pick up.

sofci · 2024-08-25T05:04:16Z

Can one of the admins verify this patch?

reply test this please to run this test once

iuliana-prodan · 2024-08-26T04:53:38Z

test this please

hongshui3000 · 2024-08-26T09:16:35Z

Which systems can run this GUI, Linux or Windows? Or both?

singalsu · 2024-08-26T09:51:45Z

tools/gui/gui/eq_configs/eq_iir_bandpass.txt

@@ -0,0 +1 @@
+4607827,0,116,50331648,0,0,0,0,116,2,1,0,0,0,0,0,0,2,2,0,0,0,0,3316150158,2048164275,513807534,3267352229,513807534,0,16384,3867454526,1191025347,38870735,77741469,38870735,4294967292,24197,


These are duplicates from tools/ctl/ip3. It would be good to align and have a single set of blobs for sof-ctl somewhere. And somehow have selection for ipc3 vs. ipc4 blobs usage. I'm also thinking to change the .txt blob headers format to be the same as .bin used by UCM2, so if duplication can be avoided there's less work to maintain.

So, I prefer .txt over .bin especially in git where we can see the diffs. UCM2 chose a slightly different headers format for blobs and since sof-ctl is only a small utility I'm planning to change it to align with UCM2 but still keep also the .txt blobs. Then the .bin and .txt blob content would be the same.

Do you have any suggestions for where this should be? For now, I'll remove them from this directory and document where they are/how to add them to the gui.

Commit here: 01c19dd

Could there be a configure widget in the GUI where the path could be set? It would also make this compatible with both IPC versions.

@alexb3103 @singalsu it should be, maybe, a text box where one can write the path to their eq_configs.

singalsu · 2024-08-26T09:56:34Z

tools/gui/gui/sof_controller_engine.py

+        print(f"Failed to set volume: {e}")
+
+def handle_eq(eq_file_name: str):
+    ctl_command = f"./sof-ctl -D{device_string} -n {eq_numid} -s {eq_file_name}"


the -c with control name as text as shown by amixer controls is more robust than number value. E.g. -c name='DMIC0 Capture IIR Eq' . The numbers change if topology is modified.

I definitely agree with that, however one functionality of the controller engine is that it automatically detects the numid for these basic components. Check out the initialize_device() method in the controller engine, specifically lines 37-43 and 45-51.

Because of this autodetection, I think that using the explicit name leads to a higher chance of error overall.

OK, I didn't know there's such feature. It sounds like the numids are then fine.

kv2019i

Very cool @alexb3103 , thanks for the submission! Before more detailed comments, one high-level nit I have is with the name "SOF GUI". This is a bit too generic as SOF can be used in various places and purposes. I wonder if we could come up with a bit more descriptive name, perhaps "SOF Control GUI" (control-gui), or maybe something around SOF components (component-gui)?

Another is how to deal with the wave files. Could you generate these as part of the build process as these seem pretty standard files? Not completely against this, but it's just we haven't put raw audio data files to the git so far in SOF.

marc-hb · 2024-08-26T15:33:10Z

Another is how to deal with the wave files. Could you generate these as part of the build process as these seem pretty standard files? Not completely against this, but it's just we haven't put raw audio data files to the git so far in SOF.

.wav is a very old and common format so hopefully not much of an attack vector anymore but it's still pretty bad from a security perspective to have unauditable binaries in a source repo.

Also, git is really bad with binaries (hence LFS and others) and 11M per file is pretty big.

lgirdwood · 2024-08-26T19:15:39Z

@alexb3103 good stuff ! btw, care to share some screen shots ?

alexb3103 · 2024-08-30T00:21:21Z

Which systems can run this GUI, Linux or Windows? Or both?

Hi, thanks for the comment.

As of now, this GUI is made for Linux and is not tested for Windows. It is definitely possible to extend support in the future if there is a lot of desire for this.

alexb3103 · 2024-08-30T00:28:41Z

Very cool @alexb3103 , thanks for the submission! Before more detailed comments, one high-level nit I have is with the name "SOF GUI". This is a bit too generic as SOF can be used in various places and purposes. I wonder if we could come up with a bit more descriptive name, perhaps "SOF Control GUI" (control-gui), or maybe something around SOF components (component-gui)?

Another is how to deal with the wave files. Could you generate these as part of the build process as these seem pretty standard files? Not completely against this, but it's just we haven't put raw audio data files to the git so far in SOF.

Thoughts on "sof-demo-gui"? The overall goal here is to provide an easy way to demonstrate sof as a whole or a specific component on target hardware, so making the name too specific will not encapsulate the entire purpose.

In terms of the .wav files, I removed them in this commit 01c19dd and added documentation for how to add when needed.

marc-hb · 2024-08-30T00:34:14Z

In terms of the .wav files, I removed them in this commit 01c19dd

I just noticed you add them in the first commit and remove them in the second commit. But this project and repo do not use Github that usual way. It does not use the "squash" button either. Instead, you must rewrite your commits and force-push = the "traditional", non-Github way to use git.

For more about this "traditional" way to rewrite git commits:

Process: investigate GitHub code review replacements zephyrproject-rtos/zephyr#39194
https://fossil-scm.org/home/doc/trunk/www/rebaseharm.md <- we do exactly this because this page is wrong :-D
https://gitlab.com/gitlab-org/gitlab/-/issues/24096

alexb3103 · 2024-08-30T00:37:35Z

In terms of the .wav files, I removed them in this commit 01c19dd

I just noticed you add them in the first commit and remove them in the second commit. But this project and repo do not use Github that usual way. It does not use the "squash" button either. Instead, you must rewrite your commits and force-push = the "traditional", non-Github way to use git.
For more about this "traditional" way to rewrite git commits:

Process: investigate GitHub code review replacements zephyrproject-rtos/zephyr#39194

https://fossil-scm.org/home/doc/trunk/www/rebaseharm.md <- we do exactly this because this page is wrong :-D

https://gitlab.com/gitlab-org/gitlab/-/issues/24096

Hi, thanks for the info. I am used to working this way as that is how it is done in my professional work. However, we wait to rebase until approval so that I can provide atomic commit messages to make review much simpler. Happy to fix it now if you would prefer though.

alexb3103 · 2024-08-30T00:41:31Z

@alexb3103 good stuff ! btw, care to share some screen shots ?

Here's a screenshot, I will be preparing a demonstration video in a couple days as well.

lyakh · 2024-09-02T08:39:13Z

tools/gui/README-dev.md

@@ -0,0 +1,40 @@
+## Developing sof-gui and tui
+
+The architecture of the sof UIs is simple, and designed to make implementing new features extremely easy.


capital "SOF" - as is it looks just like a normal word with a typo

lyakh · 2024-09-02T08:39:40Z

tools/gui/README-dev.md

+
+The architecture of the sof UIs is simple, and designed to make implementing new features extremely easy.
+
+Controller engine "sof_controller_engine.py", that handles all interaction with linux and sof.


"Linux" "SOF"

lyakh · 2024-09-02T08:43:05Z

tools/gui/README-dev.md

+
+Add graphics and other quality of life functions to the GUI.
+
+Create a version of sof-ctl that provides direct python bindings to communicate with SOF components, rather than needing a linux command.


"Python" "Linux"

lyakh · 2024-09-02T08:45:01Z

tools/gui/README.md

+User input logic and display handling
+
+### sof-controller-engine
+Controller to abstract the gui and tui control.


let's use consistent capitalisation "GUI" "TUI"

kv2019i

I'm good with "sof-demo-gui". With the other minor things addressed, I think this is ready to be merged.

tools/topology/topology1/sof/pipe-gui-components-capture.m4

kv2019i · 2024-09-06T11:38:53Z

Release reminder - one week to v2.11-rc1.

dbaluta · 2024-09-06T12:04:50Z

@iuliana-prodan if you agree I think @AnneOnciulescu can take over this PR and fix the comments until next week?

iuliana-prodan · 2024-09-06T12:17:50Z

@iuliana-prodan if you agree I think @AnneOnciulescu can take over this PR and fix the comments until next week?

@alexb3103 do you have time to fix the comments/suggestions by the end of next week?
Otherwise, @AnneOnciulescu can help.
What do you think?

alexb3103 · 2024-09-15T02:24:42Z

Hi all,

Just fixed the remaining comments unless I missed something. I also rebased down to the single commit.

Let me know what remaining changes are necessary, thanks very much for the feedback so far.

tools/demo-gui/demo-gui/sof_controller_engine.py

iuliana-prodan

For a POC this is really good.
Thanks @alexb3103

We can add new features/ improvements on top of this.

iuliana-prodan · 2024-09-26T11:48:06Z

@alexb3103 good stuff ! btw, care to share some screen shots ?

Here's a screenshot, I will be preparing a demonstration video in a couple days as well.

@alexb3103 Any demo?
It will be nice to have one

alexb3103 · 2024-09-26T22:08:05Z

@alexb3103 good stuff ! btw, care to share some screen shots ?

Here's a screenshot, I will be preparing a demonstration video in a couple days as well.

@alexb3103 Any demo? It will be nice to have one

Unfortunately I have the board/monitor packed so I'm not able to make it. Also, I think it would be best for whoever takes over the project to make the demo, as that would give experience with it. I'm happy to assist in any way necessary towards that.

lrudyX · 2024-09-27T08:01:16Z

CI fails at:

08:37:25,380 INFO  - [ 71%] Built target topology1_sof-imx8-eq-fir-wm8960
08:37:25,397 INFO  - [ 71%] Generating production/sof-imx8mp-eq-fir-wm8962.conf
08:37:25,419 INFO  - [ 71%] Generating production/sof-imx8mp-eq-fir-wm8962.tplg
08:37:25,429 INFO  - [ 71%] Built target topology1_sof-imx8mp-eq-fir-wm8962
08:37:25,451 INFO  - [ 72%] Generating production/sof-imx8mp-wm8962-mixer.conf
08:37:25,473 INFO  - [ 72%] Generating production/sof-imx8mp-wm8962-mixer.tplg
08:37:25,484 INFO  - [ 72%] Built target topology1_sof-imx8mp-wm8962-mixer
08:37:25,502 INFO  - [ 73%] Generating production/sof-acp.conf
08:37:25,517 INFO  - [ 73%] Generating production/sof-acp.tplg
08:37:25,527 INFO  - [ 73%] Built target topology1_sof-acp
08:37:25,550 INFO  - [ 73%] Generating production/sof-imx8mp-gui-components-wm8960.conf
08:37:25,570 INFO  - [ 73%] Generating production/sof-imx8mp-gui-components-wm8960.tplg
08:37:25,573 INFO  - ALSA lib dapm.c:158:(tplg_build_widget) cannot find '1 Master GUI Playback Volume' referenced by widget 'PGA1.0'
08:37:25,573 INFO  - ALSA lib parser.c:283:(tplg_build) failed to check topology integrity
08:37:25,573 INFO  - failed to compile context production/sof-imx8mp-gui-components-wm8960.conf: Invalid argument
08:37:25,574 INFO  - make[2]: *** [topology/topology1/CMakeFiles/topology1_sof-imx8mp-gui-components-wm8960.dir/build.make:73: topology/topology1/production/sof-imx8mp-gui-components-wm8960.tplg] Error 1
08:37:25,574 INFO  - make[1]: *** [CMakeFiles/Makefile2:1722: topology/topology1/CMakeFiles/topology1_sof-imx8mp-gui-components-wm8960.dir/all] Error 2
08:37:25,575 INFO  - make: *** [Makefile:136: all] Error 2
08:37:25,579 INFO  - Executing post-execute action...

dbaluta · 2024-09-27T14:54:13Z

SOFCI TEST

dbaluta · 2024-09-27T14:55:19Z

08:37:25,570 INFO - [ 73%] Generating production/sof-imx8mp-gui-components-wm8960.tplg
08:37:25,573 INFO - ALSA lib dapm.c:158:(tplg_build_widget) cannot find '1 Master GUI Playback Volume' referenced by widget 'PGA1.0'
08:37:25,573 INFO - ALSA lib parser.c:283:(tplg_build) failed to check topology integrity
08:37:25,573 INFO - failed to compile context production/sof-imx8mp-gui-components-wm8960.conf: Invalid ar

Looks like this is an error from older code. I wonder if I can retrigger the CI.

marc-hb · 2024-09-27T18:04:48Z

I wonder if I can retrigger the CI.

The only way to retrigger everything is:

git commit -C HEAD --amend # change date hence SHA1
git push -f ...

There is one shortcut for Jenkins but only for Jenkins. It will re-test but not rebuild anything if the SHA1 has not changed.

Implements a gui and tui that can be used to easily demonstrate SOF on target HW. See README and README-dev for more information on functionality and purpose. Signed-off-by: Alexander Brown <alex.brown.3103@gmail.com>

lgirdwood · 2024-10-01T12:30:27Z

Not tested yet by CI, failures are known and unrelated to this PR.

alexb3103 requested review from marc-hb, ranj063, lgirdwood, plbossart, mmaka1, lbetlej, dbaluta and kv2019i as code owners August 25, 2024 05:01

singalsu reviewed Aug 26, 2024

View reviewed changes

kv2019i reviewed Aug 26, 2024

View reviewed changes

marc-hb removed their request for review August 30, 2024 00:35

lyakh reviewed Sep 2, 2024

View reviewed changes

lgirdwood added this to the v2.11 milestone Sep 2, 2024

kv2019i reviewed Sep 3, 2024

View reviewed changes

iuliana-prodan reviewed Sep 3, 2024

View reviewed changes

tools/topology/topology1/sof/pipe-gui-components-capture.m4 Show resolved Hide resolved

alexb3103 force-pushed the sof-gui branch from abd0128 to 3ae1637 Compare September 15, 2024 02:23

dbaluta reviewed Sep 16, 2024

View reviewed changes

tools/demo-gui/demo-gui/sof_controller_engine.py Show resolved Hide resolved

kv2019i modified the milestones: v2.11, v2.12 Sep 20, 2024

iuliana-prodan approved these changes Sep 26, 2024

View reviewed changes

alexb3103 force-pushed the sof-gui branch 2 times, most recently from 3596d96 to 3241a58 Compare September 26, 2024 22:12

Implement gui and tui for sof demonstration

dc03c21

Implements a gui and tui that can be used to easily demonstrate SOF on target HW. See README and README-dev for more information on functionality and purpose. Signed-off-by: Alexander Brown <alex.brown.3103@gmail.com>

alexb3103 force-pushed the sof-gui branch from 3241a58 to dc03c21 Compare September 27, 2024 21:52

lgirdwood approved these changes Oct 1, 2024

View reviewed changes

lgirdwood merged commit 23d3d55 into thesofproject:main Oct 1, 2024
41 of 43 checks passed

		@@ -0,0 +1 @@
		4607827,0,116,50331648,0,0,0,0,116,2,1,0,0,0,0,0,0,2,2,0,0,0,0,3316150158,2048164275,513807534,3267352229,513807534,0,16384,3867454526,1191025347,38870735,77741469,38870735,4294967292,24197,

		@@ -0,0 +1,40 @@
		## Developing sof-gui and tui

		The architecture of the sof UIs is simple, and designed to make implementing new features extremely easy.


		The architecture of the sof UIs is simple, and designed to make implementing new features extremely easy.

		Controller engine "sof_controller_engine.py", that handles all interaction with linux and sof.


		Add graphics and other quality of life functions to the GUI.

		Create a version of sof-ctl that provides direct python bindings to communicate with SOF components, rather than needing a linux command.

SOF_GUI: Implement GUI and TUI for SOF demonstration #9406

SOF_GUI: Implement GUI and TUI for SOF demonstration #9406

Conversation

alexb3103 commented Aug 25, 2024

sofci commented Aug 25, 2024

iuliana-prodan commented Aug 26, 2024 • edited Loading

hongshui3000 commented Aug 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kv2019i left a comment

Choose a reason for hiding this comment

marc-hb commented Aug 26, 2024

lgirdwood commented Aug 26, 2024

alexb3103 commented Aug 30, 2024

alexb3103 commented Aug 30, 2024

marc-hb commented Aug 30, 2024

alexb3103 commented Aug 30, 2024

alexb3103 commented Aug 30, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kv2019i left a comment

Choose a reason for hiding this comment

kv2019i commented Sep 6, 2024

dbaluta commented Sep 6, 2024

iuliana-prodan commented Sep 6, 2024

alexb3103 commented Sep 15, 2024

iuliana-prodan left a comment

Choose a reason for hiding this comment

iuliana-prodan commented Sep 26, 2024

alexb3103 commented Sep 26, 2024

lrudyX commented Sep 27, 2024 • edited Loading

dbaluta commented Sep 27, 2024

dbaluta commented Sep 27, 2024 • edited Loading

marc-hb commented Sep 27, 2024 • edited Loading

lgirdwood commented Oct 1, 2024

iuliana-prodan commented Aug 26, 2024 •

edited

Loading

lrudyX commented Sep 27, 2024 •

edited

Loading

dbaluta commented Sep 27, 2024 •

edited

Loading

marc-hb commented Sep 27, 2024 •

edited

Loading