Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blendfarm: init at 1.1.6; nixos/blendfarm: init #306516

Merged
merged 2 commits into from
Jul 15, 2024

Conversation

gador
Copy link
Member

@gador gador commented Apr 24, 2024

Description of changes

Init of blendfarm

blendfarm is an "open-source, cross-platform, stand-alone, Network Renderer for Blender".

It can support chunking and rendering of still-images as well as animations over the network.

Due to its architecture it needs some workarounds for the client (which is basically the master and has a GUI) and the server (which is a headless worker).

For the server part this PR implements a nixos systemd service which the client can connect to.

The workarounds are necessary due to blendfarm reading and writing to the directory it is executed from. This can have some undesired sideeffects (such as having renders and setting files in different directories). By adding a custom script blendfarm-nix this is fixed by changing the directory before executing the client.

Also blendfarm wants to download blender on all systems by itself and tries to use this downloaded vanilla version. This obviously doesn't work for nixos. Therefore blendfarm-nix links to blender and installs a version called nix-blender. The server systemd service also installs such a nix-blender version. By doing so, blendfarm can use this version on the client and server and both are agreeing on this version.

Tested locally on several nixos x86_64-linux machines and VM's.

Still to do: Build and test for darwin. ✔️

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` labels Apr 24, 2024
@gador gador force-pushed the init-blenderfarm branch 2 times, most recently from da04f4d to c5630bf Compare April 24, 2024 12:39
@ofborg ofborg bot added 8.has: package (new) This PR adds a new package 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild 10.rebuild-linux: 1-10 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 and removed 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild labels Apr 24, 2024
nixos/modules/services/misc/blenderfarm.nix Outdated Show resolved Hide resolved
nixos/modules/services/misc/blenderfarm.nix Outdated Show resolved Hide resolved
nixos/modules/services/misc/blenderfarm.nix Outdated Show resolved Hide resolved
nixos/modules/services/misc/blenderfarm.nix Outdated Show resolved Hide resolved
pkgs/by-name/bl/blenderfarm/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/bl/blenderfarm/package.nix Outdated Show resolved Hide resolved
nixos/modules/services/misc/blenderfarm.nix Show resolved Hide resolved
@pluiedev
Copy link
Contributor

pluiedev commented May 9, 2024

BTW, the title of the PR by convention should look more like `blenderfarm: init at 1.1.6; nixos/blenderfarm: init" or something like that

@gador gador changed the title Init blenderfarm blenderfarm: init at 1.1.6; nixos/blenderfarm: init May 10, 2024
@pluiedev
Copy link
Contributor

Result of nixpkgs-review pr 306516 run on x86_64-linux 1

2 packages blacklisted:
  • nixos-install-tools
  • tests.nixos-functions.nixos-test
1 package failed to build:
  • blenderfarm

Copy link
Contributor

@pluiedev pluiedev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there are some missing dependencies:

error: 1 dependencies of derivation '/nix/store/ax98w9ms9yam8x2g5639ilcraxpp1qa3-review-shell.drv' failed to build
┏━ 1 Errors: 

┃        >     libpng12.so.0 -> found: /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm
┃        >     libexif.so.12 -> found: /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm
┃        >     libfontconfig.so.1 -> found: /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm
┃        > setting RPATH to: /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm
┃        > searching for dependencies of /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm/libpixman-1.so.0
┃        > searching for dependencies of /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm/libexif.so.12
┃        > auto-patchelf: 10 dependencies could not be satisfied
┃        > warn: auto-patchelf ignoring missing libpcre.so.3 wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm/libgl…
┃        > error: auto-patchelf could not satisfy dependency libssl.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blender…
┃        > error: auto-patchelf could not satisfy dependency libgssapi_krb5.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib…
┃        > error: auto-patchelf could not satisfy dependency libicui18n.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/ble…
┃        > error: auto-patchelf could not satisfy dependency libicuuc.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blend…
┃        > warn: auto-patchelf ignoring missing liblzma.so.5 wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm/libti…
┃        > error: auto-patchelf could not satisfy dependency libssl.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blender…
┃        > error: auto-patchelf could not satisfy dependency libgssapi_krb5.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib…
┃        > error: auto-patchelf could not satisfy dependency libicui18n.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/ble…
┃        > error: auto-patchelf could not satisfy dependency libicuuc.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blend…
┃        > auto-patchelf failed to find all the required dependencies.
┃        > Add the missing dependencies to --libs or use `--ignore-missing="foo.so.1 bar.so etc.so"`.
┃        For full logs, run 'nix log /nix/store/k3xb7978r226kif4r4zi1jy8yggzr4hr-blenderfarm-1.1.6.drv'.

@gador
Copy link
Member Author

gador commented May 10, 2024

Looks like there are some missing dependencies:

error: 1 dependencies of derivation '/nix/store/ax98w9ms9yam8x2g5639ilcraxpp1qa3-review-shell.drv' failed to build
┏━ 1 Errors: 
 ⋮ 
┃        >     libpng12.so.0 -> found: /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm
┃        >     libexif.so.12 -> found: /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm
┃        >     libfontconfig.so.1 -> found: /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm
┃        > setting RPATH to: /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm
┃        > searching for dependencies of /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm/libpixman-1.so.0
┃        > searching for dependencies of /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm/libexif.so.12
┃        > auto-patchelf: 10 dependencies could not be satisfied
┃        > warn: auto-patchelf ignoring missing libpcre.so.3 wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm/libgl…
┃        > error: auto-patchelf could not satisfy dependency libssl.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blender…
┃        > error: auto-patchelf could not satisfy dependency libgssapi_krb5.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib…
┃        > error: auto-patchelf could not satisfy dependency libicui18n.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/ble…
┃        > error: auto-patchelf could not satisfy dependency libicuuc.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blend…
┃        > warn: auto-patchelf ignoring missing liblzma.so.5 wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blenderfarm/libti…
┃        > error: auto-patchelf could not satisfy dependency libssl.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blender…
┃        > error: auto-patchelf could not satisfy dependency libgssapi_krb5.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib…
┃        > error: auto-patchelf could not satisfy dependency libicui18n.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/ble…
┃        > error: auto-patchelf could not satisfy dependency libicuuc.so wanted by /nix/store/17f6ig92mhhz0wyxc8d8mz0m08jiixka-blenderfarm-1.1.6/lib/blend…
┃        > auto-patchelf failed to find all the required dependencies.
┃        > Add the missing dependencies to --libs or use `--ignore-missing="foo.so.1 bar.so etc.so"`.
┃        For full logs, run 'nix log /nix/store/k3xb7978r226kif4r4zi1jy8yggzr4hr-blenderfarm-1.1.6.drv'.

That is really interessting!
It works if this branch is not rebased on master and fails when rebased. Something must've changed in between.
Will look into it

@gador
Copy link
Member Author

gador commented May 10, 2024

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).

Result of nixpkgs-review pr 306516 run on x86_64-linux 1

1 package blacklisted:
  • nixos-install-tools
1 package built:
  • blenderfarm

@gador
Copy link
Member Author

gador commented May 10, 2024

Result of nixpkgs-review pr 306516 run on aarch64-darwin 1

1 package built:
  • blenderfarm

@pluiedev
Copy link
Contributor

Result of nixpkgs-review pr 306516 run on x86_64-linux 1

2 packages blacklisted:
  • nixos-install-tools
  • tests.nixos-functions.nixos-test
1 package built:
  • blenderfarm

@wegank wegank added the 12.approvals: 1 This PR was reviewed and approved by one reputable person label May 17, 2024
@h7x4 h7x4 added the 8.has: module (new) This PR adds a module in `nixos/` label Jun 1, 2024
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/1724

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/1760

@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/4161

@JohnRTitor
Copy link
Contributor

Result of nixpkgs-review pr 306516 run on x86_64-linux 1

2 packages blacklisted:
  • nixos-install-tools
  • tests.nixos-functions.nixos-test
1 package built:
  • blenderfarm

Copy link
Contributor

@JohnRTitor JohnRTitor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this blendfarm or blenderfarm? According to https://github.com/LogicReinc/LogicReinc.BlendFarm it is the former.

@JohnRTitor
Copy link
Contributor

Result of nixpkgs-review pr 306516 run on aarch64-linux 1

1 package blacklisted:
  • nixos-install-tools
1 package built:
  • blenderfarm

@gador
Copy link
Member Author

gador commented Jul 5, 2024

Is this blendfarm or blenderfarm? According to https://github.com/LogicReinc/LogicReinc.BlendFarm it is the former.

Darn, you're absolutely right.

I changed it. Thanks for your review!

@gador gador changed the title blenderfarm: init at 1.1.6; nixos/blenderfarm: init blendfarm: init at 1.1.6; nixos/blendfarm: init Jul 5, 2024
Copy link
Contributor

@JohnRTitor JohnRTitor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: Florian Brandes <florian.brandes@posteo.de>
Signed-off-by: Florian Brandes <florian.brandes@posteo.de>
@JohnRTitor JohnRTitor merged commit a14c5d6 into NixOS:master Jul 15, 2024
25 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (new) This PR adds a module in `nixos/` 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 11.by: package-maintainer This PR was created by the maintainer of the package it changes 12.approvals: 1 This PR was reviewed and approved by one reputable person needs_merger
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants