libredirect: fix on darwin

[stephank]

Motivation for this change

ZHF: #144627
@NixOS/nixos-release-managers

Fixes: #141811

Also fixes an issue that may be new on macOS 11 or 12 where DYLD_FORCE_FLAT_NAMESPACE was removed. In this case, all tests failed, not just test_subprocess.

I'm not sure if there is away to make the macros prettier, using maybe some magic syntax I don't know of. 🙂

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 21.11 Release Notes (or backporting 21.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

[risicle]

I think the macros are fine (or at least I've seen much uglier macros), and I haven't found any new failures on macos 10.15 & nixos x86_64.

I had to look up why the DYLD_INSERT_LIBRARIES thing was necessary at all, so it could probably do with a comment or something.

[siraben]

I ran git bisect on libredirect on darwin, below is the results. First bad commit is 5ca0265

$ git bisect log
git bisect start
# bad: [b836d1d6676b6a9ebffabf7435bd3adc13995c6a] Merge pull request #144464 from r-ryantm/auto-update/minio
git bisect bad b836d1d6676b6a9ebffabf7435bd3adc13995c6a
# good: [4a57860df4b0a69fecdf2b0469d4a98f31d8d5ca] Merge pull request #136090 from jakubgs/more-syncthing-options
git bisect good 4a57860df4b0a69fecdf2b0469d4a98f31d8d5ca
# bad: [1645417960138185b7f5db5619a901fceb07f703] Merge pull request #139871 from fabaff/bump-urllib3
git bisect bad 1645417960138185b7f5db5619a901fceb07f703
# skip: [6f3ee459deac73b6d509b6e74a0fb0df78612d57] git: disable flaky test 6421-merge-partial-clone
git bisect skip 6f3ee459deac73b6d509b6e74a0fb0df78612d57
# good: [cfb993f75586757293e884b8403c1ea7ca507dd3] inferno: 0.10.6 -> 0.10.7
git bisect good cfb993f75586757293e884b8403c1ea7ca507dd3
# bad: [4d5f14d5f3123cb4346ee70ebd06c2068f5c16d8] python3Packages.claripy: 9.0.9792 -> 9.0.9947
git bisect bad 4d5f14d5f3123cb4346ee70ebd06c2068f5c16d8
# bad: [3a7e5f6c25155e09f7fe4b9ccb13dde339007377] ipxe: allow to pass options with values
git bisect bad 3a7e5f6c25155e09f7fe4b9ccb13dde339007377
# bad: [4e89f2a6091d2422f20804c603b05395d5f218c2] Merge #137106: knot-dns: 3.1.1 -> 3.1.2
git bisect bad 4e89f2a6091d2422f20804c603b05395d5f218c2
# bad: [ebd5774d272fc52a7ca0da667014f8fec90f6cd8] Duplicated shell-command-plus - manual fixup
git bisect bad ebd5774d272fc52a7ca0da667014f8fec90f6cd8
# bad: [4a4294d74be2c6fdc2041275762a8a1dc7216a0b] vim-plugins.direnv.vim: fix directory for substitution
git bisect bad 4a4294d74be2c6fdc2041275762a8a1dc7216a0b
# good: [717cbf8e7dcd14d97125cc1176d43c5dabc5c933] python38Packages.eventlet: 0.31.1 -> 0.32.0
git bisect good 717cbf8e7dcd14d97125cc1176d43c5dabc5c933
# bad: [1871c113f1d44f8e8922586b333e11d174bd19ca] Merge pull request #124157 from nh2/libredirect-children
git bisect bad 1871c113f1d44f8e8922586b333e11d174bd19ca
# good: [0ecfdfcc60e8dfbfb1df2044aeb8968317d76a05] Merge pull request #136599 from lugray/update_shadowenv
git bisect good 0ecfdfcc60e8dfbfb1df2044aeb8968317d76a05
# good: [0197de93b6cac9b88a71d969aa4802c2a24a47d2] Merge pull request #136671 from primeos/intel-media-driver
git bisect good 0197de93b6cac9b88a71d969aa4802c2a24a47d2
# good: [e0f90f86e9e73668efd7c71fe473db4c318d6798] procs: add changelog to meta
git bisect good e0f90f86e9e73668efd7c71fe473db4c318d6798
# good: [90982af6a14f75b148f98ac6cc09c569b14c0b61] rust-analyzer: 2021-08-23 -> 2021-08-30
git bisect good 90982af6a14f75b148f98ac6cc09c569b14c0b61
# bad: [0afbd6c86a29160386e5386332b65ba707a25340] libredirect: Enable debug symbols
git bisect bad 0afbd6c86a29160386e5386332b65ba707a25340
# bad: [5ca02655057bb568f09b3aed4acee44e3f805291] libredirect: Add missing phase hooks
git bisect bad 5ca02655057bb568f09b3aed4acee44e3f805291
# first bad commit: [5ca02655057bb568f09b3aed4acee44e3f805291] libredirect: Add missing phase hooks

[siraben]

Result of nixpkgs-review pr 144653 run on x86_64-darwin 1

12 packages failed to build:

• python38Packages.oslo-concurrency
• python38Packages.oslo-log
• python38Packages.python-manilaclient
• python38Packages.subunit2sql
• python38Packages.swift
• python38Packages.tempest
• python39Packages.oslo-concurrency
• python39Packages.oslo-log
• python39Packages.python-manilaclient
• python39Packages.subunit2sql
• python39Packages.swift
• python39Packages.tempest

29 packages built:

• libredirect
• openstackclient (python39Packages.python-openstackclient)
• python38Packages.keystoneauth1
• python38Packages.openstacksdk
• python38Packages.osc-lib
• python38Packages.oslo-db
• python38Packages.oslo-serialization
• python38Packages.oslo-utils
• python38Packages.python-cinderclient
• python38Packages.python-glanceclient
• python38Packages.python-heatclient
• python38Packages.python-ironicclient
• python38Packages.python-keystoneclient
• python38Packages.python-novaclient
• python38Packages.python-openstackclient
• python38Packages.python-swiftclient
• python39Packages.keystoneauth1
• python39Packages.openstacksdk
• python39Packages.osc-lib
• python39Packages.oslo-db
• python39Packages.oslo-serialization
• python39Packages.oslo-utils
• python39Packages.python-cinderclient
• python39Packages.python-glanceclient
• python39Packages.python-heatclient
• python39Packages.python-ironicclient
• python39Packages.python-keystoneclient
• python39Packages.python-novaclient
• swiftclient (python39Packages.python-swiftclient)

[stephank]

I ran git bisect on libredirect on darwin, below is the results. First bad commit is 5ca0265

@siraben The commit before this also fails for me, but was cached. Were you testing with nix-build --check?

Both fixes in this PR are for issues caused by newer macOS versions. I expect the cached build was produced on an older macOS builder.

12 packages failed to build:

I think this is fine? Guessing these are new failures in packages now unblocked.

I had to look up why the DYLD_INSERT_LIBRARIES thing was necessary at all, so it could probably do with a comment or something.

@risicle Added a comment and rebased the PR. 👍

[misuzu]

Result of nixpkgs-review pr 144653 run on x86_64-darwin 1

40 packages failed to build:

• openstackclient (python39Packages.python-openstackclient)
• python38Packages.keystoneauth1
• python38Packages.openstacksdk
• python38Packages.osc-lib
• python38Packages.oslo-concurrency
• python38Packages.oslo-db
• python38Packages.oslo-log
• python38Packages.oslo-serialization
• python38Packages.oslo-utils
• python38Packages.python-cinderclient
• python38Packages.python-glanceclient
• python38Packages.python-heatclient
• python38Packages.python-ironicclient
• python38Packages.python-keystoneclient
• python38Packages.python-manilaclient
• python38Packages.python-novaclient
• python38Packages.python-openstackclient
• python38Packages.python-swiftclient
• python38Packages.subunit2sql
• python38Packages.swift
• python38Packages.tempest
• python39Packages.keystoneauth1
• python39Packages.openstacksdk
• python39Packages.osc-lib
• python39Packages.oslo-concurrency
• python39Packages.oslo-db
• python39Packages.oslo-log
• python39Packages.oslo-serialization
• python39Packages.oslo-utils
• python39Packages.python-cinderclient
• python39Packages.python-glanceclient
• python39Packages.python-heatclient
• python39Packages.python-ironicclient
• python39Packages.python-keystoneclient
• python39Packages.python-manilaclient
• python39Packages.python-novaclient
• swiftclient (python39Packages.python-swiftclient)
• python39Packages.subunit2sql
• python39Packages.swift
• python39Packages.tempest

1 package built:

• libredirect

[misuzu]

With sandbox = false

Result of nixpkgs-review pr 144653 run on x86_64-darwin 1

16 packages failed to build:

• openstackclient (python39Packages.python-openstackclient)
• python38Packages.oslo-concurrency
• python38Packages.oslo-log
• python38Packages.python-manilaclient
• python38Packages.subunit2sql
• python38Packages.swift
• python38Packages.tempest
• python39Packages.osc-lib
• python39Packages.oslo-concurrency
• python39Packages.oslo-log
• python39Packages.python-heatclient
• python39Packages.python-ironicclient
• python39Packages.python-manilaclient
• python39Packages.subunit2sql
• python39Packages.swift
• python39Packages.tempest

25 packages built:

• libredirect
• python38Packages.keystoneauth1
• python38Packages.openstacksdk
• python38Packages.osc-lib
• python38Packages.oslo-db
• python38Packages.oslo-serialization
• python38Packages.oslo-utils
• python38Packages.python-cinderclient
• python38Packages.python-glanceclient
• python38Packages.python-heatclient
• python38Packages.python-ironicclient
• python38Packages.python-keystoneclient
• python38Packages.python-novaclient
• python38Packages.python-openstackclient
• python38Packages.python-swiftclient
• python39Packages.keystoneauth1
• python39Packages.openstacksdk
• python39Packages.oslo-db
• python39Packages.oslo-serialization
• python39Packages.oslo-utils
• python39Packages.python-cinderclient
• python39Packages.python-glanceclient
• python39Packages.python-keystoneclient
• python39Packages.python-novaclient
• swiftclient (python39Packages.python-swiftclient)

[stephank]

I think those failing packages are also out of scope here? I tried a few builds, and found some test failures that don't seem related to libredirect, but correct me if I'm wrong.

[risicle]

Sorry I should have been more clear, I meant the injection of the extra DYLD_INSERT_LIBRARIES in rewriteSystemCall being because of the darwin "security" feature.

[stephank]

Oh duh, that makes a lot more sense. Fixed again and rebased. 😅