Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve UI performance with QOpenGLWidget. #1974

Closed
wants to merge 37 commits into from

Conversation

rryan
Copy link
Member

@rryan rryan commented Jan 2, 2019

Opening a new PR to make discussion and viewing the diff easier.

Successor to #1863, greatly improves lp:1810099.

116fb61 contains a prototype of my proposed new approach to handling waveform rendering.

Summary of the changes and some background:

  • Switch to from QGLWidget to QOpenGLWidget (based on @sgielen's work in 650114d)
  • QOpenGLWidget always renders to an offscreen framebuffer, and the QBackingStore composites these framebuffers together when a region is marked dirty (i.e. when we call QWidget::update on any widget). There are two signals QOpenGLWidget gives us:
    • aboutToCompose: When the backing store is about to composite this widget, blitting its offscreen framebuffer to the Mixxx framebuffer.
    • frameSwapped: When the backing store has rendered the widget's offscreen framebuffer to the Mixxx framebuffer.
    • Qt Weekly on QOpenGLWidget
  • No more VSyncThread for tracking the swap time. The VsyncThread's only purpose is to send render events to the main thread at a desired FPS (will re-name the class if we agree on this implementation). The render event simply calls QWidget::update on all QOpenGLWidgets and sets a flag that they should render themselves on the next aboutToCompose signal.
    • NB: VsyncThread's swap estimates aren't accurate on macOS, because we set QGLFormat::swapInterval to 1, so every call to swapBuffers was waiting for a new vsync. This means that we waited for a vsync for every QGLWidget in the app (4 decks, 4 spinnies -> 8 vsync intervals at 60 Hz, 133 ms per second spent swapping 😱 ).
  • Each QOpenGLWidget tracks its own delta between aboutToCompose and frameSwapped, allowing us to estimate the time until swap on a per-widget basis. VisualPlayPosition is used as usual to interpolate the playposition based on the estimated time to the next swap and estimated time until the current playposition comes out the DAC.
  • The estimator used for the time until next swap is a moving average with decay 0.5 of the time between aboutToCompose and frameSwapped signals. I've yet to measure the accuracy of this estimator.
  • QOpenGLWidgets are receiving aboutToCompose and frameSwapped signals all the time (e.g. in response to VU meters, knobs, WOverview, etc. re-rendering). We only render on the aboutToCompose signal directly after VsyncThread triggers a render. QOpenGLWidget preserves the contents of its framebuffer from the last render, so we do not need to re-render until we choose to.

Backported this change to 2.2 in #1994:
I also discovered something very weird. Using QStylePainter in WSpinny causes all QOpenGLWidgets (not just WSpinnys) to take 3 ms to render instead of 300 us -- a 10x slowdown. I've removed QStylePainter -- I think where we relied on QSS for WSpinny we should find another way because that's a heavy price to pay. Not sure if it's macOS-specific, or QOpenGLWidget-specific.

With the changes in this PR, performance on macOS is significantly improved. With 4 decks, 4 spinnies, 60 FPS waveforms and 1.4ms latency, fullscreen on a 15" retina (2880 x 1800) display:

  • This PR: 65% CPU usage, a tiny amount of interface lag
  • 2.2.0: 90% CPU usage, really laggy and hard to use
  • 2.1.6: 97% CPU usage, strangely variable lag (comes and goes), not as bad as 2.2.0 though

Setting latency to 90ms, the waveforms are still smooth, which suggests that the VisualPlayPosition interpolation based on the swap interval is working.

@rryan rryan requested a review from daschuer January 2, 2019 16:44
@Be-ing
Copy link
Contributor

Be-ing commented Jan 2, 2019

Test results on Linux with a quad core Core i7 8550U and Intel UHD Graphics 620 using Deere with 4 playing decks and spinnies, Mixxx in full screen mode:

master GL (what I usually use)
~83% CPU for Mixxx
~15% CPU for Xwayland
average reported framerate: ~40 FPS
minimal jerking

master RGB (CPU)
100% CPU for Mixxx
~30% CPU for Xwayland
average reported framerate: ~40 FPS
frequent waveform jerking

master GLSL
~40% CPU for Mixxx
~20% CPU for Xwayland
average reported framerate: ~20 FPS
frequent waveform jerking

This branch GLSL (because that's all that's working currently)
~50% CPU for Mixxx
0.7% CPU for Xwayland
average reported framerate: ~20 FPS
frequent waveform jerking

Comparing the GLSL renderers between this branch and master, it seems that CPU increases for Mixxx are offset by decreases in Xwayland 🤷‍♂️ In all cases the GUI is responsive.

Running with --safeMode, either this branch or master:
~25% CPU for Mixxx
~15% CPU for Xwayland

Potentially relevant GPU info from glxinfo:

Extended renderer info (GLX_MESA_query_renderer):
    Vendor: Intel Open Source Technology Center (0x8086)
    Device: Mesa DRI Intel(R) UHD Graphics 620 (Kabylake GT2)  (0x5917)
    Version: 18.2.6
    Accelerated: yes
    Video memory: 3072MB
    Unified memory: yes
    Preferred profile: core (0x1)
    Max core profile version: 4.5
    Max compat profile version: 3.0
    Max GLES1 profile version: 1.1
    Max GLES[23] profile version: 3.2
OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) UHD Graphics 620 (Kabylake GT2) 
OpenGL core profile version string: 4.5 (Core Profile) Mesa 18.2.6

Something curious is going on with the OpenGL version reported. glxinfo reports my GPU's max supported version is 4.5. With master, Mixxx says OpenGL 4.3 in the waveform preferences. With this branch and Xwayland, I get 2.1. With this branch with -platform wayland is only 2.0 and the GLSL renderer is not an available option.

With this branch, distorted spinnies are drawn over a correctly sized background:
screenshot from 2019-01-02 14-01-41
Spinnies look fine in master.

@Be-ing
Copy link
Contributor

Be-ing commented Jan 2, 2019

Mixxx CPU usage in safe mode with all VuMeter tags commented out in Deere goes down to 8% with Xwayland at 2% (compared to 25% and 15% with VuMeters). I wonder if we could improve performance by making WVuMeter a QOpenGLWidget?

@rryan
Copy link
Member Author

rryan commented Jan 2, 2019

This branch GLSL (because that's all that's working currently)

Thank you for testing! I wasn't really aware anyone used the non-GLSL renderers :P. Just force pushed a new revision that implements rendering for all the QOpenGLWidget renderers.

Something to try, since it's a change from master on Linux: set swapInterval back to 0 here.

@rryan
Copy link
Member Author

rryan commented Jan 2, 2019

Mixxx CPU usage with all VuMeter tags commented out in Deere goes down to 8% with Xwayland at 2% (compared to 25% and 15% with VuMeters). I wonder if we could improve performance by making WVuMeter a QOpenGLWidget?

Yea, we should try this. VU meter rendering isn't too slow itself, but it seems to trigger a re-render of the entire skin. I get much smoother performance when disabling WNumberPos and WVUMeter creation in legacyskinparser.cpp.

BTW, you can set QT_DEBUG_FPS=1 and Qt will print out its own measurement of the FPS it is flushing the backing store at every 5 seconds.

@Be-ing
Copy link
Contributor

Be-ing commented Jan 2, 2019

The latest commit does not build:

scons: *** [lin64_build/src/waveform/widgets/baseqopenglwidget.o] Source `src/waveform/widgets/baseqopenglwidget.cpp' not found, needed by target `lin64_build/src/waveform/widgets/baseqopenglwidget.o'.
In file included from src/waveform/waveformwidgetfactory.cpp:19:
src/waveform/widgets/glrgbwaveformwidget.h:4:10: fatal error: waveform/widgets/baseqopenglwidget.h: No such file or directory
 #include "waveform/widgets/baseqopenglwidget.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
scons: *** [lin64_build/src/waveform/waveformwidgetfactory.o] Error 1
scons: building terminated because of errors.

@rryan
Copy link
Member Author

rryan commented Jan 2, 2019

Another thing to play around with:
https://github.com/rryan/mixxx/blob/e7a3901e649afc66503e51985ef32abb4e008cc8/src/waveform/visualplayposition.cpp#L65

  • disable playpos interpolation (set offsetMicros to 0)
  • disable swap interval estimation (set microsUntilSwap to 0)
  • disable offsetMicros clamping to within 2 callbacks

@rryan
Copy link
Member Author

rryan commented Jan 2, 2019

The latest commit does not build:

Doh, forgot to add files.

@rryan
Copy link
Member Author

rryan commented Jan 2, 2019

With this branch, distorted spinnies are drawn over a correctly sized background:

Oh cool, I can reproduce with Deere but not Tango.

@Be-ing
Copy link
Contributor

Be-ing commented Jan 2, 2019

With the latest commit using the RGB GL renderer on Linux, CPU use jumps to 100% for Mixxx but Xwayland CPU use is ~3%. Strangely the RGB GLSL renderer is not available but the Filtered GLSL renderer is. The GUI remains responsive and I probably wouldn't notice the extreme CPU use if I wasn't watching htop.

@Be-ing
Copy link
Contributor

Be-ing commented Jan 2, 2019

Something to try, since it's a change from master on Linux: set swapInterval back to 0 here.
disable playpos interpolation (set offsetMicros to 0)
disable swap interval estimation (set microsUntilSwap to 0)
disable offsetMicros clamping to within 2 callbacks

None of those make a noticeable difference for me using the GL renderer. In every case CPU usage is around 100% for Mixxx, the framerate reported in the Waveform preferences is ~50-55 FPS, and there is subtle jerking (as there always has been).

@Be-ing
Copy link
Contributor

Be-ing commented Jan 3, 2019

VU meter rendering isn't too slow itself, but it seems to trigger a re-render of the entire skin. I get much smoother performance when disabling WNumberPos and WVUMeter creation in legacyskinparser.cpp.

CPU use drops down to ~60% (from 100%) with RGB GL renderer when commenting both those out.

@Be-ing
Copy link
Contributor

Be-ing commented Jan 3, 2019

Is mouse input for knobs and sliders okay now if you comment out the WidgetRenderTimer hacks?

@Be-ing
Copy link
Contributor

Be-ing commented Jan 3, 2019

With this branch, distorted spinnies are drawn over a correctly sized background:

Oh cool, I can reproduce with Deere but not Tango.

I tested other skins and yes this only happens for me with Deere. Deere uses some ugly hacks in its spinny template. I do not know why the distorted spinnies are not square considering all the sizes in that template are fixed size squares. 😕

@rryan
Copy link
Member Author

rryan commented Jan 3, 2019

Is mouse input for knobs and sliders okay now if you comment out the WidgetRenderTimer hacks?

No :(, still very laggy.

@rryan
Copy link
Member Author

rryan commented Jan 3, 2019

None of those make a noticeable difference for me using the GL renderer. In every case CPU usage is around 100% for Mixxx, the framerate reported in the Waveform preferences is ~50-55 FPS, and there is subtle jerking (as there always has been).

What audio buffer size do you use and what's the "system reported latency" in the preferences?

@Be-ing
Copy link
Contributor

Be-ing commented Jan 3, 2019

Interesting. IIRC @sgielen reported that mouse input wasn't laggy with knobs/faders when using QOpenGLWidget and building with macOS SDK 10.14. You're building with macOS SDK 10.13, correct?

@Be-ing
Copy link
Contributor

Be-ing commented Jan 3, 2019

What audio buffer size do you use and what's the "system reported latency" in the preferences?

23.2 ms with ALSA. "System reported latency" is 23.22 ms. Going down to 1.45 ms doesn't change the subtle jerking of the waveforms or make the GUI less responsive. Look at htop, I still see a Mixxx thread taking 100% CPU but with a lower latency I now see another Mixxx thread taking ~17% CPU, which I presume is the audio thread.

@rryan
Copy link
Member Author

rryan commented Jan 3, 2019

Interesting. IIRC @sgielen reported that mouse input wasn't laggy with knobs/faders when using QOpenGLWidget and building with macOS SDK 10.14. You're building with macOS SDK 10.13, correct?

Yep, 10.13 SDK. I'll give 10.14 a shot.

@rryan
Copy link
Member Author

rryan commented Jan 3, 2019

23.2 ms with ALSA. "System reported latency" is 23.22 ms. Going down to 1.45 ms doesn't change the subtle jerking of the waveforms or make the GUI less responsive. Look at htop, I still see a Mixxx thread taking 100% CPU but with a lower latency I now see another Mixxx thread taking ~17% CPU, which I presume is the audio thread.

Gotcha. The VisualPlayposition code uses the audio buffer size, which is potentially incorrect (because if you ask for 10ms and portaudio can only get you 20ms, then it calls the engine with a 10ms buffer twice in quick succession). But in your case the audio buffer size is equal to the latency, so that isn't it.

Does 90 ms latency also have subtle jerking?

@Be-ing
Copy link
Contributor

Be-ing commented Jan 3, 2019

Does 90 ms latency also have subtle jerking?

Yes, and it also occurs with JACK.

@rryan
Copy link
Member Author

rryan commented Jan 3, 2019

Hm, sounds like I need to do some testing with X and Wayland.

I tested other skins and yes this only happens for me with Deere. Deere uses some ugly hacks in its spinny template. I do not know why the distorted spinnies are not square considering all the sizes in that template are fixed size squares. 😕

Turns out QOpenGLWidget::resizeEvent wasn't getting run (and it has logic that's important). Fixed in dda5096.

Something curious is going on with the OpenGL version reported. glxinfo reports my GPU's max supported version is 4.5. With master, Mixxx says OpenGL 4.3 in the waveform preferences. With this branch and Xwayland, I get 2.1. With this branch with -platform wayland is only 2.0 and the GLSL renderer is not an available option.

Ah yea, what it's currently showing is the OpenGL profile Mixxx requests, which is a 2.1 compatibility profile. That's not showing the maximum supported OpenGL version anymore.

@Be-ing
Copy link
Contributor

Be-ing commented Jan 3, 2019

Ah yea, what it's currently showing is the OpenGL profile Mixxx requests, which is a 2.1 compatibility profile. That's not showing the maximum supported OpenGL version anymore.

I tried setting the requested OpenGL version in WaveformWidgetFactory::setDefaultVersion higher. I tried 4.5, 4.3, and 3.0, and every time that made the waveforms only render the beatgrid.

@ywwg
Copy link
Member

ywwg commented Jan 3, 2019

I'm on a new Dell XPS 13 (9370) running Ubuntu 18.10 with Wayland, and almost identical output of glxinfo except Mesa 18.2.2 instead of 18.2.6. I don't have any of the performance issues @Be-ing is reporting. I can run 4 decks at a time non-keylocked and use about 40% cpu. There are occasional frame drops but nothing major. LateNight skin, 10ms latency, GLSL RGB

This is with GNOME 3 desktop environment.

@Be-ing
Copy link
Contributor

Be-ing commented Apr 17, 2020

I tried wayland mode and the whole UI disappears when I try to make it full screen.

I encountered that bug too.

One thing I've noticed is that the waveforms in master look jerkier than the frame drop meter would imply. I think this reinforces RJ's theory that smooth waveforms are less about our detecting vsync than just writing frames fast enough for the compositor to draw them when needed

I think that would be consistent with my observation that waveforms are much smoother with a small window than expanding to the full width of my screen.

@daschuer daschuer removed the blocker label Apr 21, 2020
@Be-ing Be-ing added the blocker label May 7, 2020
@Be-ing
Copy link
Contributor

Be-ing commented May 7, 2020

I re-added the "beta blocker" tag to indicate we must have this for 2.4.

@daschuer
Copy link
Member

daschuer commented May 7, 2020

I have work in progress

@Holzhaus
Copy link
Member

Holzhaus commented Jun 9, 2020

@daschuer What's the status on this?

@daschuer
Copy link
Member

daschuer commented Jun 9, 2020

I have made a tiny progress since last comment. To many other loose ends.

@Holzhaus
Copy link
Member

Holzhaus commented Sep 3, 2020

Any news on this?

@daschuer
Copy link
Member

daschuer commented Sep 3, 2020

No, my work has stalled due to GSoC.
I plan to continue soon.

@Be-ing Be-ing changed the base branch from master to main October 23, 2020 23:28
Be-ing added a commit to Be-ing/buildserver that referenced this pull request Nov 14, 2020
More up-to-date versions of Qt have major performance problems
with QGLWidget. This will not be fixed for Mixxx 2.3:
mixxxdj/mixxx#1974
Be-ing added a commit to Be-ing/buildserver that referenced this pull request Nov 14, 2020
More up-to-date versions of Qt have major performance problems
with QGLWidget. This will not be fixed for Mixxx 2.3:
mixxxdj/mixxx#1974
@github-actions
Copy link

This PR is marked as stale because it has been open 90 days with no activity.

@github-actions github-actions bot added the stale Stale issues that haven't been updated for a long time. label Jan 23, 2021
@Holzhaus Holzhaus removed the blocker label Jun 1, 2021
@Holzhaus
Copy link
Member

Holzhaus commented Jun 1, 2021

Removed the "blocker" label because we're currently doing a release without this :D

@github-actions github-actions bot removed the stale Stale issues that haven't been updated for a long time. label Jun 2, 2021
@Be-ing
Copy link
Contributor

Be-ing commented Jul 1, 2021

Thank you very much for working on this difficult issue @rryan. Your work helped us understand the extent of our technical debt and the requirements for moving Mixxx forward. We have decided to reimplement the whole GUI in QML instead of continue with this.

@Be-ing Be-ing closed this Jul 1, 2021
@rryan
Copy link
Member Author

rryan commented Jul 2, 2021 via email

@ronso0
Copy link
Member

ronso0 commented Jul 3, 2021

There will be a few more major releases until the QML implementation is finished.
So if I understand correctly this would help improving performance until then.
Just trying to understand, can the remaining issues here be resolved or is this wasting time, also considering this (maybe?) needs to be adapted this to Qt6 later on?
@daschuer

@Holzhaus
Copy link
Member

Holzhaus commented Jul 3, 2021

AFAIK QOpenGLWidget is supported in Qt6, just QGLWidget is dropped. I agree with @ronso0 that it may be worthwhile to use this for the transition, unless it's an extreme amount of work to integrate.

@Be-ing
Copy link
Contributor

Be-ing commented Jul 3, 2021

It would be an extreme amount of work to integrate. There are fundamental architectural issues in this branch which we do not know how to solve.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.