Graphical glitches when client opengl is disabled

[ndonkersloot]

Describe the bug
When client opengl isn't used there are graphical glitches in Firefox.

To Reproduce
Steps to reproduce the behavior:

1. server command: xpra start :10
2. client command: issue when starting with xpra attach ssh://user@x.x.x.x/10 fixed when starting with xpra attach ssh://user@x.x.x.x/10 --opengl=on
3. specific action to trigger the bug: Open Firefox and scroll or hover on scroll bar.

Screen recording of the issue is found here: (download link has expired)

System Information (please complete the following information):

• Server OS: Fedora 36
• Client OS: Fedora 36
• Xpra Server Version 4.4
• Xpra Client Version 4.4

Additional context
When looking in the compress debug logs i notice there are different rgb_formats available when using opengl.
Maybe the issue occurs when converting RGB instead of YUV?

without opengl:

589 compress:   8.3ms for 1920x1130 pixels at    0,0    for wid=5     using      h264 with ratio   0.0%  ( 8482KB to     0KB), sequence    64, client_options={'csc': 'BGRX', 'frame': 15, 'pts': 3726, 'quality': 91, 'encoder': 'nvenc'}, options={'quality': 91, 'speed': 5, 'rgb_formats': ('BGRA', 'BGRX', 'RGBA', 'RGBX', 'BGR', 'RGB', 'r210', 'BGR565'), 'lz4': True, 'content-type': 'video', 'window-size': (1920, 1131), 'cuda-device-context': cuda_device_context(0 - False), 'b-frames': True}
598 compress:   0.1ms for 1920x1    pixels at    0,1130 for wid=5     using     rgb24 with ratio   1.1%  (    7KB to     0KB), sequence    65, client_options={'rgb_format': 'BGRX', 'lz4': 7, 'encoder': 'argb', 'flush': 1}, options={'quality': 91, 'speed': 5, 'rgb_formats': ('BGRA', 'BGRX', 'RGBA', 'RGBX', 'BGR', 'RGB', 'r210', 'BGR565'), 'lz4': True, 'content-type': 'video', 'window-size': (1920, 1131), 'cuda-device-context': cuda_device_context(0 - False)}
605 compress:   6.9ms for 1920x1130 pixels at    0,0    for wid=5     using      h264 with ratio   0.0%  ( 8482KB to     0KB), sequence    66, client_options={'csc': 'BGRX', 'frame': 16, 'pts': 3744, 'quality': 91, 'encoder': 'nvenc'}, options={'quality': 91, 'speed': 5, 'rgb_formats': ('BGRA', 'BGRX', 'RGBA', 'RGBX', 'BGR', 'RGB', 'r210', 'BGR565'), 'lz4': True, 'content-type': 'video', 'window-size': (1920, 1131), 'cuda-device-context': cuda_device_context(0 - False), 'b-frames': True}
614 compress:   0.1ms for 1920x1    pixels at    0,1130 for wid=5     using     rgb24 with ratio   1.1%  (    7KB to     0KB), sequence    67, client_options={'rgb_format': 'BGRX', 'lz4': 7, 'encoder': 'argb', 'flush': 1}, options={'quality': 91, 'speed': 5, 'rgb_formats': ('BGRA', 'BGRX', 'RGBA', 'RGBX', 'BGR', 'RGB', 'r210', 'BGR565'), 'lz4': True, 'content-type': 'video', 'window-size': (1920, 1131), 'cuda-device-context': cuda_device_context(0 - False)}
623 compress:   8.5ms for 1920x1130 pixels at    0,0    for wid=5     using      h264 with ratio   0.0%  ( 8482KB to     0KB), sequence    68, client_options={'csc': 'BGRX', 'frame': 17, 'pts': 3760, 'quality': 91, 'encoder': 'nvenc'}, options={'quality': 91, 'speed': 5, 'rgb_formats': ('BGRA', 'BGRX', 'RGBA', 'RGBX', 'BGR', 'RGB', 'r210', 'BGR565'), 'lz4': True, 'content-type': 'video', 'window-size': (1920, 1131), 'cuda-device-context': cuda_device_context(0 - False), 'b-frames': True}

with opengl:

230 compress:   0.1ms for 1920x1    pixels at    0,1130 for wid=5     using     rgb24 with ratio   1.1%  (    7KB to     0KB), sequence   105, client_options={'rgb_format': 'BGRX', 'lz4': 4, 'encoder': 'argb', 'flush': 1}, options={'optimize': False, 'auto_refresh': True, 'quality': 100, 'speed': 50, 'rgb_formats': ('YUV420P', 'YUV422P', 'YUV444P', 'GBRP', 'BGRA', 'BGRX', 'RGBA', 'RGBX', 'RGB', 'BGR'), 'lz4': True, 'content-type': 'video', 'window-size': (1920, 1131), 'cuda-device-context': cuda_device_context(0 - False)}
265 compress:  35.5ms for 1920x1131 pixels at    0,0    for wid=5     using      webp with ratio   6.0%  ( 8482KB to   505KB), sequence   106, client_options={'rgb_format': 'BGRX', 'quality': 100, 'encoder': 'webp'}, options={'optimize': False, 'auto_refresh': True, 'quality': 100, 'speed': 50, 'rgb_formats': ('YUV420P', 'YUV422P', 'YUV444P', 'GBRP', 'BGRA', 'BGRX', 'RGBA', 'RGBX', 'RGB', 'BGR'), 'lz4': True, 'content-type': 'video', 'window-size': (1920, 1131), 'cuda-device-context': cuda_device_context(0 - False)}
490 compress:   0.1ms for 1920x1    pixels at    0,1130 for wid=5     using     rgb24 with ratio   1.1%  (    7KB to     0KB), sequence   107, client_options={'rgb_format': 'BGRX', 'lz4': 7, 'encoder': 'argb', 'flush': 1}, options={'quality': 78, 'speed': 15, 'rgb_formats': ('YUV420P', 'YUV422P', 'YUV444P', 'GBRP', 'BGRA', 'BGRX', 'RGBA', 'RGBX', 'RGB', 'BGR'), 'lz4': True, 'content-type': 'video', 'window-size': (1920, 1131), 'cuda-device-context': cuda_device_context(0 - False)}
497 compress:   7.0ms for 1920x1130 pixels at    0,0    for wid=5     using      h264 with ratio   0.0%  ( 8482KB to     0KB), sequence   108, client_options={'csc': 'BGRX', 'frame': 17, 'pts': 9642, 'quality': 78, 'encoder': 'nvenc'}, options={'quality': 78, 'speed': 15, 'rgb_formats': ('YUV420P', 'YUV422P', 'YUV444P', 'GBRP', 'BGRA', 'BGRX', 'RGBA', 'RGBX', 'RGB', 'BGR'), 'lz4': True, 'content-type': 'video', 'window-size': (1920, 1131), 'cuda-device-context': cuda_device_context(0 - False), 'b-frames': True}
650 compress:   0.1ms for 1920x1    pixels at    0,1130 for wid=5     using     rgb24 with ratio   1.0%  (    7KB to     0KB), sequence   109, client_options={'rgb_format': 'BGRX', 'lz4': 4, 'encoder': 'argb', 'flush': 1}, options={'optimize': False, 'auto_refresh': True, 'quality': 100, 'speed': 50, 'rgb_formats': ('YUV420P', 'YUV422P', 'YUV444P', 'GBRP', 'BGRA', 'BGRX', 'RGBA', 'RGBX', 'RGB', 'BGR'), 'lz4': True, 'content-type': 'video', 'window-size': (1920, 1131), 'cuda-device-context': cuda_device_context(0 - False)}

[totaam]

i notice there are different rgb_formats available when using opengl.
without:
'quality': 91 ... 'csc': 'BGRX' ... 'rgb_formats': 'BGRA', 'BGRX', 'RGBA', 'RGBX', 'BGR', 'RGB', 'r210', 'BGR565'
with:
'quality': 78 ... 'csc': 'BGRX' ... 'rgb_formats': ('YUV420P', 'YUV422P', 'YUV444P', 'GBRP', 'BGRA', 'BGRX', 'RGBA', 'RGBX', 'RGB', 'BGR')

The quality is high (>=70) so in both cases the server is using full colourspace input mode: 'csc': 'BGRX'

More details on rgb_formats: the non-opengl backend supports 15-bit (BGR565) and 30-bit (r210) modes for all windows.
These modes are also implemented in the OpenGL backend but they cannot be used unless the window also uses this same pixel format - and this one does not. (windows almost always run in 24 or 32 bit modes)
Then the opengl backend also supports YUV modes natively - generally, this is what you want to use for video content: it compresses better and uses less memory to represent each frame. The OpenGL backend will paint YUV data to the window very efficiently using shaders.
(you would need a lower quality value to use that - when using nvenc, this is less of an issue because the encoder is so fast - unless you run at really high resolutions, then you may want to use subsampled input for better performance)

Other points of interest:

• your window is 1920x1131, odd heights can be problematic because video encoders normally work with even input heights, so we send the last row of pixels using rgb24 - which means both updates need to be applied for each frame - this usually works fine, but you can disable the edge with this env var to verify: XPRA_VIDEO_SKIP_EDGE=1. Perhaps this should be re-visited and if nvenc (and others?) can handle odd heights properly with BGRX input then we can just change its input mask:

  xpra/xpra/codecs/nvidia/nvenc/encoder.pyx

  Lines 1369 to 1370 in bc05bef

  	WIDTH_MASK = 0xFFFE
  	HEIGHT_MASK = 0xFFFE

• the webp update sticks out like a sore thumb at 505KB! rgb with lz4 and h264 compressed updates show up as 0KB (in the future this will be rounded up: ab9472f) - this can make a massive impact on usability. It is an auto_refresh update, which means that the screen was not updating fast enough and the server decided to send a lossless screen update - 250ms later, another h264 screen update comes through, perhaps increasing the default auto refresh delay would have avoided this expensive lossless update..

[totaam]

The encoder mask issue already had a ticket (now mostly done): #3359.

See also #3761 and #3837

[totaam]

I haven't found the culprit yet, as I got distracted and fixed a number of transparency inconsistencies between different encoders and paint paths: 6e5e50d, 3e61ef1, 7ed4f4f, c2ef9c0, 644e061, 20ddd05

---

I am testing with Firefox and using the control channel to trigger screen updates with a specific encoding, ie with png:

xpra control :100 request-update png all "*"

tweaking the quality - for the encoders that honour this setting:

xpra control :100 quality 50

[totaam]

Very likely to be the same as #4201
The issue has been present for a while, just somehow made worse by a recent update.