Cannot run pyvista examples on npps0

[plexoos]

The error message is:

libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
2024-05-16 18:35:20.590 (   3.412s) [    7F9ADE20F240]vtkXOpenGLRenderWindow.:651    ERR| vtkXOpenGLRenderWindow (0x334a150): Cannot create GLX context.  Aborting.
ERROR:root:Cannot create GLX context.  Aborting.
Aborted (core dumped)

To reproduce on npps0 run:

python -m venv venv
source venv/bin/activate
pip install pyvista
python -c "import pyvista as pv; pv.examples.download_puppy().plot()"

The libGL errors disappear if the following environment variable is set:

export LIBGL_ALWAYS_INDIRECT=1

[rahmans1]

I can confirm that I can reproduce the issue.

[rahmans1]

@plexoos I think that I found the solution to the issue. Need to use the vtk os-mesa variant. See pyvista/pyvista#4024

python -m venv venv
source venv/bin/activate 
pip install pyvista
pip uninstall vtk -y
pip install --extra-index-url https://wheels.vtk.org vtk-osmesa

After this, the command runs without error.

python -c "import pyvista as pv; pv.examples.download_puppy().plot()"

Can you please verify?

[rahmans1]

Can confirm that I can run this example and save the image output to pdf

python -c "import pyvista as pv; from pyvista import examples; pl = pv.Plotter(); _ = pl.add_mesh(examples.load_airplane(), smooth_shading=True); _ =pl.add_background_image(examples.mapfile); pl.save_graphic('img.pdf') " 

[plexoos]

Thank you Sakib! I followed your instructions and the command indeed ran without errors on npps0. On my laptop, however, when I run the same pyvista example I get an interactive window displaying some 3D capabilities. I wonder if there is something else is missing from our setup on npps0...

[buddhasystem]

@dmitri: Is that the 3D window you mentioned?

[plexoos]

If it is interactive and the object can be rotated with your mouse pointer then yes. Here is the command I use python -c "import pyvista as pv; pv.examples.download_puppy().plot()"

[buddhasystem]

@plexoos It actually worked, i.e. I can rotate the puppy. I just installed pyvista into a fresh virtual env on npps0, and am running a ssh session remotely from my workstation at home, with X11 forwarding.

[plexoos]

Great! It means the problem is on my side. Could you freeze your virtual environment and share it with me so I can reproduce? Also, if there is anything else you think I need to adjust in my environment please let me know.

[buddhasystem]

@plexoos I really did nothing special, just "python -m venv blah",with the standard Python 3.12 as installed on our node, then "pip install pyvista". If there is anything suspect it's the pipeline for graphics from npps0 to your screen (not a strong theory but still). I'm on mobaXterm. Are you sure you have the X11 forwarding in place?

[plexoos]

AFAICT, my X11 forwarding works and I can open graphical windows (xclock, xeyes) on my laptop from npps0. However, when I do the test as in the original post I get nothing and the command just exits. @rahmans1 does it open a window for you on your remote terminal?

@buddhasystem there must be something different in your environment on npps0 as both Sakib and I see the errors as reported but in your case it simply works with the default vtk

[buddhasystem]

Well, since I did this with the virtual env, and I don't have extras, it really boils down to something external to the host, npps0. But there are more mysteries -- when running examples on my local WSL2 machine, I see black screen in the pyvista GUI. When I kill the app the true picture appears briefly. I tried my GL trick I use with Geant, it didn't help. Unfortunately, there is something arcane in how pyvista works wrt drivers.

[plexoos]

Well, if you claim that the remote setup on npps0 is identical for us then what difference on our local machines we should look into? It sounds like you are on a windows machine whereas I am on a mac. Do you have a mac to test this by any chance? Doesn't the rendering happen on the remote machine anyway? I'd still love Sakib to confirm whether the test opens a remote window for him. BTW, the test runs locally on my i9 mac without any issues out of the box.

[buddhasystem]

The issue may be quite complex i.e. qt and vtk version related: pyvista/pyvistaqt#54

[buddhasystem]

@plexoos ...and I also tested on a standalone Debian machine with an older NVIdia GPU where things "just work". So in the end, we are looking at some combinatorial situation with vtk, qt, pyvista and perhaps actual drivers. Do we like this platform?

[plexoos]

So in the end, we are looking at some combinatorial situation with vtk, qt, pyvista and perhaps actual drivers

My assumption is that we use the same libraries on the same machine with the same drivers... Isn't it the case?

[rahmans1]

Can confirm that it doesn't open an interactive window when I remote login from Mac. Has to be something to do with port-forwarding and not intrinsic to this case. For example, even this does not open a window for me.

from matplotlib import pyplot as plt
plt.plot([1,2],[2,4])
[<matplotlib.lines.Line2D object at 0x7fe664af4c20>]
plt.show()

[plexoos]

I also cannot open x windows when using the default python on npps0:

$ python -c "import matplotlib.pyplot as plt; plt.plot([1,2,3]); plt.show()"
<string>:1: UserWarning: FigureCanvasAgg is non-interactive, and thus cannot be shown

However, the official python image works just fine

docker run --rm --volume="$HOME/.Xauthority:/root/.Xauthority" -e DISPLAY=$DISPLAY --network=host python:3.12.0 sh -c 'pip install matplotlib; python -c "import matplotlib.pyplot as plt; plt.plot([1,2,3]); plt.show()"'

From that I conclude that there is something wrong with how python was installed on npps0

[buddhasystem]

@plexoos No, there is nothing wrong with Python on npps0. It's the qt bit that was missing, and I suspect some incongruity with the Qt explains the pyvista problem. Case in point, this matplotlib example of yours works affter "pip install PyQt5".

[buddhasystem]

Just to add that the airplane example works 100% for me on npps0:
python -c "import pyvista as pv; pv.examples.load_airplane().plot()"

[plexoos]

...and yet, this forces us to install indirect dependencies like pyqt. What else do we need to install to make the pyvista example work?

[plexoos]

Just to add that the airplane example works 100% for me on npps0: python -c "import pyvista as pv; pv.examples.load_airplane().plot()"

please provide complete instructions so we can also verify that

[plexoos]

Nope, same error for me

$ python -c "import pyvista as pv; pv.examples.load_airplane().plot()"
libGL error: No matching fbConfigs or visuals found
libGL error: failed to load driver: swrast
2024-05-22 11:56:06.802 (   1.738s) [    7F21B6827240]vtkXOpenGLRenderWindow.:651    ERR| vtkXOpenGLRenderWindow (0x34ebef0): Cannot create GLX context.  Aborting.
ERROR:root:Cannot create GLX context.  Aborting.
Aborted (core dumped)
(venv) [dmitri@npps0:~] 
$ pip freeze
certifi==2024.2.2
charset-normalizer==3.3.2
contourpy==1.2.1
cycler==0.12.1
fonttools==4.51.0
idna==3.7
kiwisolver==1.4.5
matplotlib==3.9.0
numpy==1.26.4
packaging==24.0
pillow==10.3.0
platformdirs==4.2.2
pooch==1.8.1
pyparsing==3.1.2
python-dateutil==2.9.0.post0
pyvista==0.43.8
requests==2.32.2
scooby==0.10.0
six==1.16.0
urllib3==2.2.1
vtk==9.3.0

[plexoos]

Looks like one need to enable GLX on the MacOs X-window side

defaults write org.xquartz.X11 enable_iglx -bool true

With this the pyvista test command at least tries to open a local window but it is black and disappears right away. The new error message is:

$ python -c "import pyvista as pv; pv.examples.download_puppy().plot()"
2024-05-22 14:00:26.238 (   1.346s) [    7F6559354240]vtkOpenGLRenderWindow.c:710    ERR| vtkXOpenGLRenderWindow (0x355fae0): Unable to find a valid OpenGL 3.2 or later implementation. Please update your video card driver to the latest version. If you are using Mesa please make sure you have version 11.2 or later and make sure your driver in Mesa supports OpenGL 3.2 such as llvmpipe or openswr. If you are on windows and using Microsoft remote desktop note that it only supports OpenGL 3.2 with nvidia quadro cards. You can use other remoting software such as nomachine to avoid this issue.
ERROR:root:Unable to find a valid OpenGL 3.2 or later implementation. Please update your video card driver to the latest version. If you are using Mesa please make sure you have version 11.2 or later and make sure your driver in Mesa supports OpenGL 3.2 such as llvmpipe or openswr. If you are on windows and using Microsoft remote desktop note that it only supports OpenGL 3.2 with nvidia quadro cards. You can use other remoting software such as nomachine to avoid this issue.
Segmentation fault (core dumped)

[rahmans1]

@plexoos Can we try using a different platform plugin than what it defaults to. Check out this issue.
pyvista/pyvista#2924

[plexoos]

Sure, I am open to any solution. Have you tried to use xvfb?

[rahmans1]

@plexoos I don't have permission to install xvfb on npps0.

[plexoos]

hm... looks like you are in the wheel group. Does sudo work for you? @buddhasystem

[buddhasystem]

@plexoos I would really like to understand what we are pursuing here. If it's just validation of the Opticks tests, would an export to PDF work as a reasonable approximation? Unless this "pyvista" is a part of our final product, there is no sense in trying to make this finicky package work for all cases. Alternatively, I can lend you an extra desktop for your office where you can visualize with pyvista -- if it's about going through the opticks example, it should do the trick.

[buddhasystem]

This is becoming a time sink.

[plexoos]

If you have better suggestions how to visualize the results from an opticks tests, please let us know. pyvista was suggested by the author, see https://groups.io/g/opticks/message/526

[buddhasystem]

@plexoos I made a suggestion which is workable, use any computer other than Mac to visualize it. This is the most expedient solution.

[plexoos]

I am closing this issue as we have found a workaround for Mac users to view graphical windows (with 3D OpenGL graphics) by utilizing remote desktops such as tigervnc. It appears the Windows users may have better luck opening such windows directly from a remote GPU host.