WIP nixos/opengl: move to hardware.drivers #158079

jonringer · 2022-02-04T05:11:35Z

Motivation for this change

Implementation part of NixOS/rfcs#121

closes: #141803

Things needing to be done:

services.xserver.videoDrivers -> hardware.gpu.drivers
hardware.opengl -> hardware.drivers
Update nixos manual
Update other nixos modules
Update hardcoded references to /run/opengl-driver (postponed til after 22.11)
Rename addOpenGLRunpath -> addHardwareRunpath [staging] addHardwareRunpath: init, alias addOpenGLRunpath #196174
Add release notes

jonringer · 2022-02-04T07:12:46Z

took off draft so i could see non-ofborg gates

jonringer · 2022-02-05T20:45:47Z

The old nixos manual reference should probably be made into literals. Was just trying to get the manual to build again

nixos-discourse · 2022-02-08T22:37:55Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/include-nixgl-in-nix-flake-app/17588/2

nixos-discourse · 2022-03-10T00:27:04Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-use-nvidia-v100-a100-gpus/17754/4

Kiskae · 2022-03-26T20:01:32Z

pkgs/os-specific/linux/amdgpu-pro/default.nix

@@ -172,15 +172,15 @@ in stdenv.mkDerivation rec {
    # this could also be done with LIBGL_DRIVERS_PATH, but it would need to be
    # set in the user session and for Xorg
    then ''
-      expr1='s:/opt/amdgpu/lib/x86_64-linux-gnu/dri\0:/run/opengl-driver/lib/dri\0\0\0\0\0\0\0\0\0\0\0:g'
-      expr2='s:/usr/lib/x86_64-linux-gnu/dri[\0\:]:/run/opengl-driver/lib/dri\0\0\0\0:g'
+      expr1='s:/opt/amdgpu/lib/x86_64-linux-gnu/dri\0:/run/hardware-drivers/lib/dri\0\0\0\0\0\0\0\0\0\0\0:g'


I'm pretty sure these changes will clobber the binary, the original code contains padding to match the length of the string and the new path is longer

The legacy link will be retained. So it might be an exception

nixos-discourse · 2022-07-05T20:20:21Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/cuda-in-nixos-on-gcp-for-a-tesla-k80/20145/9

jonringer · 2022-10-15T19:03:05Z

Updated canonical driver path to be /run/current-system/drivers for the systemd tmpfile link but didn't update the nixpkgs references to /run/opengl-driver/ for smoother user transition, as mentioned in the RFC discussion.

jonringer · 2022-10-15T21:08:08Z

This PR still needs some more testing, I will be trying to use it with my current system configuration today.

jonringer · 2022-10-15T21:12:07Z

ran into some issues with extraPackages being removed

…oDrivers/hardware.gpu.drivers/g

jonringer · 2022-10-15T21:17:24Z

was able to get my current system configuration building. Would like for others to test

roberth · 2022-10-16T11:18:51Z

nixos/modules/hardware/drivers.nix

+    (mkRenamedOptionModule [ "hardware" "opengl" "enable" ] [ "hardware" "drivers" "enable" ])
+    (mkRenamedOptionModule [ "hardware" "opengl" "package" ] [ "hardware" "drivers" "packages" ])
+    (mkRenamedOptionModule [ "hardware" "opengl" "extraPackage" ] [ "hardware" "drivers" "packages" ])
+    (mkRenamedOptionModule [ "hardware" "opengl" "driSupport32Bit" ] [ "hardware" "drivers" "enable32bit" ])
+    (mkRenamedOptionModule [ "services" "xserver" "vaapiDrivers" ] [ "hardware" "drivers" "packages" ])
+    (mkRemovedOptionModule [ "hardware" "opengl" "s3tcSupport" ] ''
+      S3TC support is now always enabled in Mesa.
+    '')
+    (mkRemovedOptionModule [ "hardware" "opengl" "driSupport" ] ''
+      dri support is now always enabled in Mesa.
+    '')


Please only start warning when the new options are available on all supported releases. To sketch the issue, otherwise, you're putting "3rd party" repos in a spot where they have to behave differently for each version just to avoid user complaints about warnings. This "3rd party" includes nixos-hardware.

what's the desired way forward? I thought this was the canonical way to do it.

I only added the first 4 opengl lines, the vaapiDrivers and s3tc lines existed already.

nixpkgs/nixos/modules/hardware/opengl.nix

Lines 28 to 31 in a941ed5

(mkRenamedOptionModule [ "services" "xserver" "vaapiDrivers" ] [ "hardware" "opengl" "extraPackages" ])

(mkRemovedOptionModule [ "hardware" "opengl" "s3tcSupport" ] ''

S3TC support is now always enabled in Mesa.

'')

The driSupport I removed because all the consuming logic just was never used and implied.

You could use mkRenamedOptionModuleWith { sinceRelease = 2211; from = ...; to = ...; }. This will start warning on master when 2205 is EOL.

didn't realise this existed. It should be used more often, as a lot times I just reference similar modules for correctness

LunNova · 2022-10-27T17:25:00Z

If we're taking the effort to rename these might make sense to make them easier to override at the same time? A single list means having to use mkForce when you set it if you want to remove one item from it to replace it.

hardware.drivers.packages = {
  mesa = pkgs.mesa.drivers;
  amdvlk = pkgs.amdvlk;
}

This way you could cleanly use hardware.drivers.packages.mesa = lib.mkForce myOverriddenMesa.

Otherwise this PR removing the separation between the main package and extra packages makes it a little harder to properly override your mesa version without potentially accidentally removing other entries from the list.

purepani · 2023-08-10T19:58:41Z

Is there any help needed to move this PR forward?

Atemu · 2023-11-22T17:38:16Z

nixos/modules/services/x11/xserver.nix

@@ -155,6 +155,9 @@ in
        "Use services.xserver.fontPath instead of useXFS")
      (mkRemovedOptionModule [ "services" "xserver" "useGlamor" ]
        "Option services.xserver.useGlamor was removed because it is unnecessary. Drivers that uses Glamor will use it automatically.")
+      (mkRemovedOptionModule ["services" "xserver" "videoDriver" ]
+        "Use hardware.gpu.drivers instead")
+      (mkRenamedOptionModule [ "services" "xserver" "videoDrivers" ] [ "hardware" "gpu" "drivers" ])


I think this option should either stay under xorg or be removed entirely. What's in this list are the xorg DDX drivers, not OGL/VK/whatever drivers. Deriving the latter from the former has always been a bad pattern IMHO.

Instead, we should have an option where you can enable specific drivers directly and ideally also pass driver packages as overrides.

Something like

hardware.drivers.<driver> = { enable = true; package = ...; driverSpecificOption = ...; };

for mesa, amdvlk, nvidia, amdgpu-pro and whatever other significant driver packages exist.

I also think we might want to have a per-API interface. APIs would include obvious ones like OGL and VK but also OpenCL, VAAPI, oneAPI, CUDA, rocm and various other acceleration APIs for TPUs and the like; anything that facilitates talking to specialised hardware and requires a central userspace component basically.

Each API would have the option to enable/disable (a headless media server might not need OGL/VK but might want VAAPI for instance), declaring available drivers (many APIs allow multiple drivers to co-exist) and perhaps the option to set a preferred/primary driver where applicable.

Something like:

hardware.acceleration.<api> = { enable = ...; packages = { ... }; preferredPackage = config.hardware.acceleration.packages.<driver>; apiSpecificOption = ...; };

The <driver>.enable option would place its package into every API the driver implements, allowing the user to override a certain driver's package "globally".

When the desktop is enabled (that setting should also be moved out of xorg), we would enable the mesa driver and the OGL and VAAPI APIs.
If a user requires the nvidia driver, they would set hardware.drivers.nvidia.enable = true which would in-turn also enable the kernel module. Should they need an older driver, they'd set hardware.drivers.nvidia.package = nvidia_xyz;.
Enabling steam (or some other gaming related thing) would enable the VK driver.

What do you think of this design?

Your design looks like it resolves my concern over hardware.drivers.packages making mesa hard to override.

What's the benefit of splitting up enabling opengl and vulkan? Bundling everything in mesa together is somewhat expected so the default shouldn't be to split that up or it will add a new confusing way to have games break.

Similarly, mesa should probably default to on if any graphics drivers are in use to match current behavior and behavior of other distros.
Enabling only nvidia libs should be possible but it's unusual and wouldn't be a good thing to have happen by default or accident.

What's the benefit of splitting up enabling opengl and vulkan?

The reasoning is that these can be en-/disabled separately is that these are entirely separate acceleration APIs. A regular desktop user does not require Vulkan drivers, oneAPI or CUDA.

It also simplifies enabling less usual APIs as all you'd need to do is set hardware.acceleration.obscureAPI.enable = true and if there is a "mesa situation" where its the most sensible driver to use, we might opt to enable that driver when the API gets enabled.

mesa should probably default to on if any graphics drivers are in use to match current behavior and behavior of other distros.

This is what I meant with enabling mesa when the desktop is enabled.

A regular desktop user does not require Vulkan drivers

I think this will change soon - wayland compositors are in various stages (eg kwin https://invent.kde.org/plasma/kwin/-/issues/169 ) of investigating or supporting vulkan and I would be surprised if this doesn't eventually become the default.

As long as the defaults are good / the behavior for someone who installs NixOS and wants to have their graphics card work as expected is good then I support your proposal for making very granular options available.

mweinelt · 2023-11-23T15:15:29Z

@jonringer Fancy rebasing this for 24.05? The way things are set up right now are really misleading and need fixing!

Atemu · 2023-11-23T16:35:16Z

If you don't want to continue this, I think I'll move forward with attempting to implement my proposed design this weekend.

jonringer · 2024-03-20T03:17:47Z

@Atemu seems to have a better long term solution to handle hardware acceleration, whereas I was just concerned with naming alignment. #158079 (comment)

jonringer marked this pull request as draft February 4, 2022 05:11

github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` labels Feb 4, 2022

jonringer force-pushed the rfc-121 branch from 182c4e2 to e85625a Compare February 4, 2022 06:24

ofborg bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Feb 4, 2022

github-actions bot added the 6.topic: python label Feb 4, 2022

jonringer force-pushed the rfc-121 branch from a03fd77 to 38cc982 Compare February 4, 2022 07:12

jonringer marked this pull request as ready for review February 4, 2022 07:12

jonringer requested a review from FRidh as a code owner February 4, 2022 07:12

jonringer force-pushed the rfc-121 branch from 38cc982 to 7570528 Compare February 4, 2022 07:15

ofborg bot added 10.rebuild-darwin: 501+ 10.rebuild-darwin: 1001-2500 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Feb 4, 2022

jonringer force-pushed the rfc-121 branch from 7570528 to 27f5642 Compare February 4, 2022 07:43

github-actions bot added 8.has: changelog 8.has: documentation This PR adds or changes documentation labels Feb 4, 2022

jonringer force-pushed the rfc-121 branch from 27f5642 to 4fca2d2 Compare February 4, 2022 07:53

ofborg bot added 8.has: clean-up 8.has: package (new) This PR adds a new package labels Feb 5, 2022

jansol mentioned this pull request Feb 21, 2022

unigine-heaven: init at 4.0 #161221

Closed

1 task

samuela mentioned this pull request Mar 13, 2022

cudatoolkit: expose compute-sanitizer, nsys utilities #153542

Merged

13 tasks

Kiskae reviewed Mar 26, 2022

View reviewed changes

colemickens mentioned this pull request Mar 30, 2022

Start decoupling X and graphics support with videoDrivers #153808

Closed

13 tasks

ofborg bot added 10.rebuild-linux: 1-10 and removed 10.rebuild-darwin: 501+ 10.rebuild-darwin: 1001-2500 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ labels Oct 15, 2022

jonringer force-pushed the rfc-121 branch from 8ce8597 to 39585fe Compare October 15, 2022 19:02

jonringer marked this pull request as draft October 15, 2022 21:11

Jonathan Ringer added 3 commits October 15, 2022 14:16

nixos/opengl: move to hardware.drivers

b1d7bda

nixos/xserver: move gpu logic into it's own module

de4a492

nixos/*: s/hardware.opengl/hardware.drivers/g s/services.xserver.vide…

0597622

…oDrivers/hardware.gpu.drivers/g

jonringer force-pushed the rfc-121 branch from 39585fe to 0597622 Compare October 15, 2022 21:16

jonringer marked this pull request as ready for review October 15, 2022 21:16

roberth requested changes Oct 16, 2022

View reviewed changes

dryya mentioned this pull request Mar 5, 2023

services.xserver doesn't support multiple GPU setups #108018

Open

SomeoneSerge added the 6.topic: cuda Parallel computing platform and API label Mar 23, 2023

Kiskae mentioned this pull request Sep 15, 2023

libcuda.so: driver mismatch on nixos-rebuild switch #255070

Open

Atemu reviewed Nov 22, 2023

View reviewed changes

Tungsten842 mentioned this pull request Dec 27, 2023

nixos: opengl -> impure-drivers.opengl #276558

Closed

10 tasks

wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 19, 2024

jonringer closed this Mar 20, 2024

Tungsten842 mentioned this pull request Jun 16, 2024

treewide: big opengl cleanups #320228

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP nixos/opengl: move to hardware.drivers #158079

WIP nixos/opengl: move to hardware.drivers #158079

jonringer commented Feb 4, 2022 •

edited

Loading

jonringer commented Feb 4, 2022

jonringer commented Feb 5, 2022

nixos-discourse commented Feb 8, 2022

nixos-discourse commented Mar 10, 2022

Kiskae Mar 26, 2022

jonringer Mar 27, 2022

nixos-discourse commented Jul 5, 2022

jonringer commented Oct 15, 2022 •

edited

Loading

jonringer commented Oct 15, 2022

jonringer commented Oct 15, 2022

jonringer commented Oct 15, 2022

roberth Oct 16, 2022

jonringer Oct 16, 2022 •

edited

Loading

roberth Oct 18, 2022

jonringer Oct 18, 2022

LunNova commented Oct 27, 2022

purepani commented Aug 10, 2023

Atemu Nov 22, 2023 •

edited

Loading

LunNova Nov 22, 2023

Atemu Nov 23, 2023 •

edited

Loading

LunNova Nov 23, 2023 •

edited

Loading

mweinelt commented Nov 23, 2023

Atemu commented Nov 23, 2023

jonringer commented Mar 20, 2024

	(mkRenamedOptionModule [ "services" "xserver" "vaapiDrivers" ] [ "hardware" "opengl" "extraPackages" ])
	(mkRemovedOptionModule [ "hardware" "opengl" "s3tcSupport" ] ''
	S3TC support is now always enabled in Mesa.
	'')

WIP nixos/opengl: move to hardware.drivers #158079

WIP nixos/opengl: move to hardware.drivers #158079

Conversation

jonringer commented Feb 4, 2022 • edited Loading

Motivation for this change

Things needing to be done:

jonringer commented Feb 4, 2022

jonringer commented Feb 5, 2022

nixos-discourse commented Feb 8, 2022

nixos-discourse commented Mar 10, 2022

Kiskae Mar 26, 2022

Choose a reason for hiding this comment

jonringer Mar 27, 2022

Choose a reason for hiding this comment

nixos-discourse commented Jul 5, 2022

jonringer commented Oct 15, 2022 • edited Loading

jonringer commented Oct 15, 2022

jonringer commented Oct 15, 2022

jonringer commented Oct 15, 2022

roberth Oct 16, 2022

Choose a reason for hiding this comment

jonringer Oct 16, 2022 • edited Loading

Choose a reason for hiding this comment

roberth Oct 18, 2022

Choose a reason for hiding this comment

jonringer Oct 18, 2022

Choose a reason for hiding this comment

LunNova commented Oct 27, 2022

purepani commented Aug 10, 2023

Atemu Nov 22, 2023 • edited Loading

Choose a reason for hiding this comment

LunNova Nov 22, 2023

Choose a reason for hiding this comment

Atemu Nov 23, 2023 • edited Loading

Choose a reason for hiding this comment

LunNova Nov 23, 2023 • edited Loading

Choose a reason for hiding this comment

mweinelt commented Nov 23, 2023

Atemu commented Nov 23, 2023

jonringer commented Mar 20, 2024

jonringer commented Feb 4, 2022 •

edited

Loading

jonringer commented Oct 15, 2022 •

edited

Loading

jonringer Oct 16, 2022 •

edited

Loading

Atemu Nov 22, 2023 •

edited

Loading

Atemu Nov 23, 2023 •

edited

Loading

LunNova Nov 23, 2023 •

edited

Loading