binutils: 2.38 -> 2.39 #185297

trofi · 2022-08-05T20:30:56Z

Changes: https://sourceware.org/pipermail/binutils/2022-August/122246.html

Description of changes

Things done

winterqt · 2022-08-05T20:53:52Z

This should be available more widely now. 🤞

@ofborg eval

trofi · 2022-08-05T22:05:54Z

libbfd

Will need a libbfd fix as well. Will look in detail tomorrow. Switching to draft meanwhile.

       > patching file bfd/Makefile.am
       > Reversed (or previously applied) patch detected!  Assume -R? [n]
       > Apply anyway? [n]
       > Skipping patch.
       > 1 out of 1 hunk ignored -- saving rejects to file bfd/Makefile.am.rej
       > patching file bfd/Makefile.in
       > Reversed (or previously applied) patch detected!  Assume -R? [n]
       > Apply anyway? [n]
       > Skipping patch.
       > 1 out of 1 hunk ignored -- saving rejects to file bfd/Makefile.in.rej
       For full logs, run 'nix log /nix/store/8h4mfxwgsh0sh0wyg79yghw1bkakxg08-libbfd-2.39.drv'.

trofi · 2022-08-06T06:10:25Z

More failures:

systemd: systemd: 251.3 -> 251.4 #186069
- systemd-boot fails to build on binutils-2.39 due to -Wl,--fatal-warnings systemd/systemd#24226
gnu-efi: gnu-efi: 3.0.14 -> 3.0.15 #185680
- https://sourceforge.net/p/gnu-efi/patches/85/

Fails to build EFI images (due to gnu-efi deficiency). Not sure yet if the failure is intended or it's a warning-as-error bug:

FAILED: src/boot/efi/linuxx64.elf.stub
/nix/store/vq7x24g13zpqglpazlx3k574l74cnhl5-gcc-wrapper-13.0.0/bin/gcc -o src/boot/efi/linuxx64.elf.stub -Wno-format-signedness -Wno-missing-field-initializers -Wno-unused-parameter -Wdate-time -Wendif-labels -Werror=format=2 -Werror=implicit-function-declaration -Werror=incompatible-pointer-types -Werror=int-conversion -Werror=overflow -Werror=override-init -Werror=return-type -Werror=shift-count-overflow -Werror=shift-overflow=2 -Werror=undef -Wfloat-equal -Wimplicit-fallthrough=5 -Winit-self -Wlogical-op -Wmissing-include-dirs -Wmissing-noreturn -Wnested-externs -Wold-style-definition -Wpointer-arith -Wredundant-decls -Wshadow -Wstrict-aliasing=2 -Wstrict-prototypes -Wsuggest-attribute=noreturn -Wunused-function -Wwrite-strings -Wno-unused-result -fno-stack-protector -fno-strict-aliasing -fpic -fwide-exec-charset=UCS2 -Wall -Wextra -Wsign-compare -nostdlib -std=gnu11 -ffreestanding -fshort-wchar -fvisibility=hidden -isystem /nix/store/0am9hs6bvh3id3g3n3fhyynznfcqnfwd-gnu-efi-3.0.14/include/efi -isystem /nix/store/0am9hs6bvh3id3g3n3fhyynznfcqnfwd-gnu-efi-3.0.14/include/efi/x86_64 -I /build/source/src/fundamental -DSD_BOOT -DGNU_EFI_USE_MS_ABI -include src/boot/efi/efi_config.h -include version.h -mno-red-zone -mno-sse -mno-mmx -fuse-ld=bfd -L /nix/store/0am9hs6bvh3id3g3n3fhyynznfcqnfwd-gnu-efi-3.0.14/lib -nostdlib -T /nix/store/0am9hs6bvh3id3g3n3fhyynznfcqnfwd-gnu-efi-3.0.14/lib/elf_x86_64_efi.lds -Wl,--build-id=sha1 -Wl,--fatal-warnings -Wl,--no-undefined -Wl,--warn-common -Wl,-Bsymbolic -z nocombreloc /nix/store/0am9hs6bvh3id3g3n3fhyynznfcqnfwd-gnu-efi-3.0.14/lib/crt0-efi-x86_64.o -pie -Wl,--no-dynamic-linker src/boot/efi/bootspec-fundamental.c.o src/boot/efi/efivars-fundamental.c.o src/boot/efi/sha256.c.o src/boot/efi/string-util-fundamental.c.o src/boot/efi/assert.c.o src/boot/efi/devicetree.c.o src/boot/efi/disk.c.o src/boot/efi/graphics.c.o src/boot/efi/measure.c.o src/boot/efi/pe.c.o src/boot/efi/secure-boot.c.o src/boot/efi/ticks.c.o src/boot/efi/util.c.o src/boot/efi/cpio.c.o src/boot/efi/initrd.c.o src/boot/efi/splash.c.o src/boot/efi/stub.c.o src/boot/efi/linux_x86.c.o -lefi -lgnuefi -lgcc
/nix/store/s2lxn7xgyf7crhny8afl4chq7111n9kv-binutils-2.39/bin/ld.bfd: warning: /nix/store/0am9hs6bvh3id3g3n3fhyynznfcqnfwd-gnu-efi-3.0.14/lib/crt0-efi-x86_64.o: missing .note.GNU-stack section implies executable stack
/nix/store/s2lxn7xgyf7crhny8afl4chq7111n9kv-binutils-2.39/bin/ld.bfd: NOTE: This behaviour is deprecated and will be removed in a future version of the linker
collect2: error: ld returned 1 exit status

UPDATE: due to -Wl,--fatal-warnings.

trofi · 2022-08-06T06:12:12Z

elfutils
- ~~elfutils test workaround: https://sourceware.org/PR29450~~ (fixed binutils's gas instead)
- binutils gas fix: https://sourceware.org/PR29451
- OR enable .o stripping in glibc: setup-hooks/separate-debug-info.sh: don't inhibit strip hook #185537

Fails build with a failure that looks like a printf arguments problem and an address range problem:

FAIL: run-low_high_pc.sh
========================

[b] main.c
[2d] main

[b] ../sysdeps/i386/start.S
[26] _start
[40] ../sysdeps/x86/abi-note.c
[b52] init.c
[b8e] static-reloc.c
[2dba] _dl_relocate_static_pie
[2dd8] ../sysdeps/i386/crti.S
[2def] _init
lowpc: 8049000, highpc: 8049000lx
../sysdeps/i386/crti.S: [2def] '_init' highpc <= lowpc
FAIL run-low_high_pc.sh (exit status: 255)

lovesegfault · 2022-08-06T06:40:08Z

Bumping binutils is always a huge pain :(

trofi · 2022-08-06T09:28:26Z

linuxPackages_latest.perf: linuxPackages_latest.perf: apply upstream patch for binutils-2.39 #186318
- https://lore.kernel.org/lkml/20220703044814.892617-2-andres@anarazel.de/
linuxPackages.perf: linuxPackages.perf: use binutils-2.38 for linux before 5.19 #188121

init_disassemble_info changed API slightly to emit coloured output: https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=60a3da00bd5407f07d64dff82a4dae98230dfaac

util/annotate.c: In function 'symbol__disassemble_bpf':
util/annotate.c:1765:9: error: too few arguments to function 'init_disassemble_info'
 1765 |         init_disassemble_info(&info, s,
      |         ^~~~~~~~~~~~~~~~~~~~~

trofi · 2022-08-11T17:53:48Z

pkgsMusl.binutils: disabled gprofng
- ~~gprofng needs javac: https://sourceware.org/PR29479~~ (disabled gprofng)
- ~~gprofng needs musl fixes: https://sourceware.org/PR29477~~ (disabled gprofng)

gprofng fails to build on musl:

/nix/store/4k23hc0n15afv1nk951vh1c4c3f0yacz-bootstrap-tools/include-libc/sys/signal.h:1:2: warning: #warning redirecting incorrect #include <sys/signal.h> to <signal.h> [-Wcpp]
 #warning redirecting incorrect #include <sys/signal.h> to <signal.h>
  ^~~~~~~
In file included from dispatcher.c:30:0:
./../src/collector_module.h:78:13: error: duplicate member ‘pwrite’
   ssize_t (*pwrite64)();
             ^

trofi · 2022-08-14T09:46:06Z

bpftool: bpftools: use binutils-2.38 for linux before 5.19 #188122

Same failure as perf, se will need to apply patch to the kernel itself, not the individual tools.

error: builder for '/nix/store/gy683d4gdhz6ids9y3ig3a8yqf6m64xn-bpftools-5.19.drv' failed with exit code 2;
       last 10 log lines:
       >   472 | extern void init_disassemble_info (struct disassemble_info *dinfo, void *stream,
       >       |             ^~~~~~~~~~~~~~~~~~~~~
       > jit_disasm.c:105:17: error: too few arguments to function 'init_disassemble_info'
       >   105 |                 init_disassemble_info(&info, stdout,
       >       |                 ^~~~~~~~~~~~~~~~~~~~~
       > /nix/store/gziacflg8p7xd3p65rafwqvfni9j03n3-libopcodes-2.39-dev/include/dis-asm.h:472:13: note: declared here
       >   472 | extern void init_disassemble_info (struct disassemble_info *dinfo, void *stream,
       >       |             ^~~~~~~~~~~~~~~~~~~~~
       > make[1]: *** [Makefile:222: jit_disasm.o] Error 1 shuffle=1660469830
       > make: *** [Makefile:110: bpftool] Error 2 shuffle=1660469830
       For full logs, run 'nix log /nix/store/gy683d4gdhz6ids9y3ig3a8yqf6m64xn-bpftools-5.19.drv'.

ghost · 2022-08-16T22:16:59Z

Hrm, pkgs/top-level/linux-kernels.nix still has linux_default = packages.linux_5_15;, so packages like cargo-flamegraph which depend on perf don't build after this.

I don't know what the right solution is. It's unfortunate that libopcode changed the API when they added pretty-printing instead of exposing a new function and having the old one invoke it with a sensible default.

In the meantime maybe we should throw, since if the patch is neither applied nor included (kernel>=5.20) there is no hope of compilation succeeding? i.e.:

https://github.com/NixOS/nixpkgs/pull/186318/files#r947293455

trofi · 2022-08-17T05:49:46Z

Hrm, pkgs/top-level/linux-kernels.nix still has linux_default = packages.linux_5_15;, so packages like cargo-flamegraph which depend on perf don't build after this.

I don't know what the right solution is. It's unfortunate that libopcode changed the API when they added pretty-printing instead of exposing a new function and having the old one invoke it with a sensible default.

In the meantime maybe we should throw, since if the patch is neither applied nor included (kernel>=5.20) there is no hope of compilation succeeding? i.e.:

https://github.com/NixOS/nixpkgs/pull/186318/files#r947293455

Note that this PR is a draft. It will require a bit of porting for older kernels (or pulling in older libbfd version for older kernels).

PR you linked is already merged into master, does it cause build failures for you on master as well? Can you specify which attributes?

ghost · 2022-08-17T19:18:44Z

With override { linux_default = packages.linux_5_19; } this PR successfully builds:

powerpc64le-linux
- full workstation except ghc and its referrers
- ghc and its referrers
mips64el-linux-gnuabi64
- cross from powerpc64le-linux
  - bootstrap-files
  - full router environment
- native
  - stdenv
  - full router environment
mips64el-linux-gnuabin32
- cross from powerpc64le-linux
  - bootstrap-files
- native
  - stdenv
  - full router environment (building)

LGTM if we can figure out what to do about the linux_default situation.

trofi · 2022-08-23T07:04:02Z

Added 2.38 release for libbfd and libopcodes to allow pinning individual tools to it.

Also proposed 2.38 holdback release against master as: Create binutils 2.38 holdback set of releases #187972.

That way we can pin tools to it regardless of status of this PR. The changes should be a no-op as derivation of libbfd / libbfd_2_38 will be identical in master.

SuperSandro2000 · 2022-08-23T21:49:44Z

pkgs/development/tools/misc/binutils/libbfd.nix

+  # We just want to build libbfd
+  postPatch = ''
+    cd bfd
+  '';


Should we use sourceRoot instead?

sourceRoot would require us knowing the extracted path (which probably isn't a good idea to rely on), and will break patches.

I think Sandro means cd $sourceRoot

trofi · 2022-08-24T21:22:17Z

More failures:

init_disassemble_info() API fallout:
PTR -> void * API cleanup fallout:
- ucx: ucx: pull upstream fix for binutils-2.39 #188258

trofi · 2022-08-25T07:38:16Z

Rebased against current master to include all the master fixes (mostly pin to held back binutils-2.38). Piled on a few more fixes for a small number of leaf packages. Should be good enough shape to request a new jobset to weed out the rest of failures.

lovesegfault · 2022-08-26T14:37:53Z

@vcunat Usually helps me get a jobset for these kinds of bumps

trofi · 2022-08-26T18:32:13Z

Also asked to create a jobset on infra matrix channel: https://app.element.io/#/room/#infra:nixos.org/$Glu5NtSnZGrc_3TdGbA89SAslq4tJGiGVnFgiaglGHw

vcunat · 2022-08-27T15:30:35Z

For reference, now created as https://hydra.nixos.org/jobset/nixpkgs/pr-185297-binutils-2.39

trofi · 2022-08-29T07:22:19Z

New failures:

dmd: dmd: disable whitespace-sensitive tests for binutils-2.39 #189087
- fails tests: https://hydra.nixos.org/build/188843476 (2 test failures)
- upstream issue: https://issues.dlang.org/show_bug.cgi?id=23317

UPDATE: it's a minor output change difference. objdump stopped inserting whitespace after ret instruction and broke expected/actual comparison:

dmd/test $ diff -u ./test_results/compilable/cdcmp_0.o.dump.sanitized ./compilable/extra-files/cdcmp.out
 Disassembly of section .text.d_dso_init:

@@ -1069,5 +1069,5 @@
   1c:  6a 01                   push   0x1
   1e:  48 8b fc                mov    rdi,rsp
   21:  e8 00 00 00 00          call   26 <.text.d_dso_init+0x26>
-  26:  c9                      leave
-  27:  c3                      ret<whitespace>
+  26:  c9                      leave
+  27:  c3                      ret

Changes: https://sourceware.org/pipermail/binutils/2022-August/122246.html

trofi · 2022-09-02T07:15:57Z

Jobset finished with: success: 53091, failures: 812. Inspecting failures against trunk jobset I found no new failures(!).

Rebased current branch past all the fixes above. Ready for review.

lovesegfault · 2022-09-02T14:57:12Z

You're a superstar trofi :)

Happy to see this land!

trofi · 2022-09-02T17:42:42Z

Heh, you are too kind :)

risicle · 2022-09-04T19:12:01Z

Since this merge, on macos 10.15 (and reportedly also darwin aarch64):

  CCLD     libctf-nobfd.la
./libtool: eval: line 1115: syntax error near unexpected token `|'
./libtool: eval: line 1115: `/nix/store/2z3fkfy273yqg2ylxyybvly0dks1nlby-bootstrap-stage0-clang-wrapper-11.1.0/bin/nm -B  .libs/libctf_nobfd_la-ctf-archive.o .libs/libctf_nobfd_la-ctf-dump.o .libs/libctf_nobfd_la-ctf-create.o .libs/libctf_nobfd_la-ctf-decl.o .libs/libctf_nobfd_la-ctf-error.o .libs/libctf_nobfd_la-ctf-hash.o .libs/libctf_nobfd_la-ctf-labels.o .libs/libctf_nobfd_la-ctf-dedup.o .libs/libctf_nobfd_la-ctf-link.o .libs/libctf_nobfd_la-ctf-lookup.o .libs/libctf_nobfd_la-ctf-open.o .libs/libctf_nobfd_la-ctf-serialize.o .libs/libctf_nobfd_la-ctf-sha1.o .libs/libctf_nobfd_la-ctf-string.o .libs/libctf_nobfd_la-ctf-subr.o .libs/libctf_nobfd_la-ctf-types.o .libs/libctf_nobfd_la-ctf-util.o   |  | /nix/store/8ysd696cpacbvhy61z57w4vg7jn2gncz-bootstrap-stage0-clang/bin/sed 's/.* //' | sort | uniq > .libs/libctf-nobfd.exp'

trofi · 2022-09-04T19:30:58Z

./libtool: eval: line 1115: syntax error near unexpected token `|'

This usually happens when a command is completely missing between pipes: ./a | $foo | $bar when $foo is undefined. binutils might have taken on some new dependency normally present on linux. Do you have a full build long handy by chance?

Possibly a $global_symbol_pipe failed: export_symbols_cmds='$NM $libobjs $convenience | $global_symbol_pipe | $SED '\''s/.* //'\'' | sort | uniq > $export_symbols'. https://sourceware.org/git/?p=binutils-gdb.git;a=commitdiff;h=caf606c90d55305967b9253447dda93d2f1835ab is likely related.

This update dropped one patch that is directly relevant. I thought it was upstreamed, but perhaps it does not quite work:

-    # Breaks nm BSD flag detection
-    ./0001-Revert-libtool.m4-fix-nm-BSD-flag-detection.patch

Trying to reproduce locally meanwhile.

risicle · 2022-09-04T20:25:55Z

https://gist.github.com/risicle/dda04fc876a118f494e41a66d26f7eea

trofi · 2022-09-04T21:29:36Z

Aha, checking the name lister (/nix/store/2z3fkfy273yqg2ylxyybvly0dks1nlby-bootstrap-stage0-clang-wrapper-11.1.0/bin/nm -B) interface... yes, but that nm does not support -B option. That's a binutils's configure bug. I'll stare at it a bit more to see where -B slips in.

Do you know where nm comes from in bootstrap-stage0 on darwin? At least on Linux llvmPackages_11.llvm does have nm -B flag support. In my environment ld -v from bootstrap tools says it's something called ld64-530. ttps://opensource.apple.com/source/ld64/ld64-530/src/ld/ld.cpp.auto.html says it's from 2011.

Could it be that it's an ancient llvm version?

trofi · 2022-09-04T21:43:35Z

Oh, it's something called dawin.cctools (found in pkgs/stdenv/darwin/make-bootstrap-tools.nix). https://opensource.apple.com/source/cctools/. Has no support for -B option in latest version either. No easy workaround it seems. binutils will need to adapt.

trofi · 2022-09-04T22:36:45Z

Filed upstream bug as https://sourceware.org/PR29547

trofi · 2022-09-05T18:21:56Z

Proposed nixpkgs-specific fix as #189894 by reverting nm-specific binutils patches again until it gets fixed upstream.

alyssais · 2022-09-12T07:21:04Z

Seems to have broken cross-compilation of libbfd.

alyssais · 2022-09-12T08:06:10Z

Ah, because this PR enabled the libbfd documentation ~~and it isn't cross friendly~~.

trofi · 2022-09-12T08:45:37Z

Let's try to sort it with #190897

trofi marked this pull request as draft August 5, 2022 22:05

trofi force-pushed the binutils-2.39 branch from 95b66e4 to ce47c7f Compare August 6, 2022 06:06

lovesegfault self-requested a review August 6, 2022 06:40

trofi force-pushed the binutils-2.39 branch 2 times, most recently from d9a821e to 8ca63bb Compare August 11, 2022 16:50

ofborg bot added 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild 10.rebuild-linux-stdenv This PR causes stdenv to rebuild labels Aug 11, 2022

ofborg bot requested a review from Ericson2314 August 11, 2022 18:17

ofborg bot added 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ labels Aug 11, 2022

trofi force-pushed the binutils-2.39 branch from 8ca63bb to 9eaa095 Compare August 12, 2022 09:41

ghost mentioned this pull request Aug 17, 2022

linuxPackages_latest.perf: apply upstream patch for binutils-2.39 #186318

Merged

13 tasks

This comment was marked as resolved.

Sign in to view

trofi mentioned this pull request Aug 23, 2022

Create binutils 2.38 holdback set of releases #187972

Merged

13 tasks

trofi force-pushed the binutils-2.39 branch from 9eaa095 to 364a924 Compare August 23, 2022 07:01

ofborg bot added the 8.has: package (new) This PR adds a new package label Aug 23, 2022

trofi force-pushed the binutils-2.39 branch from c23eaf7 to c9e7da6 Compare August 23, 2022 20:16

SuperSandro2000 reviewed Aug 23, 2022

View reviewed changes

trofi force-pushed the binutils-2.39 branch from c9e7da6 to a64411d Compare August 25, 2022 06:33

trofi force-pushed the binutils-2.39 branch from a64411d to 8a569d2 Compare August 26, 2022 06:51

binutils: 2.38 -> 2.39

6a378e9

Changes: https://sourceware.org/pipermail/binutils/2022-August/122246.html

trofi force-pushed the binutils-2.39 branch from 8a569d2 to 6a378e9 Compare September 2, 2022 06:48

trofi marked this pull request as ready for review September 2, 2022 07:15

lovesegfault merged commit c625d67 into NixOS:staging Sep 2, 2022

trofi deleted the binutils-2.39 branch September 2, 2022 17:42

toonn mentioned this pull request Sep 12, 2022

freshBootstrapTools: rename from stdenvBootstrapTools #183072

Merged

10 tasks

binutils: 2.38 -> 2.39 #185297

binutils: 2.38 -> 2.39 #185297

Conversation

trofi commented Aug 5, 2022 • edited Loading

Description of changes

Things done

winterqt commented Aug 5, 2022

trofi commented Aug 5, 2022 • edited Loading

trofi commented Aug 6, 2022 • edited Loading

trofi commented Aug 6, 2022 • edited Loading

lovesegfault commented Aug 6, 2022

trofi commented Aug 6, 2022 • edited Loading

trofi commented Aug 11, 2022 • edited Loading

trofi commented Aug 14, 2022 • edited Loading

ghost commented Aug 16, 2022

trofi commented Aug 17, 2022

This comment was marked as resolved.

This comment was marked as resolved.

ghost commented Aug 17, 2022 • edited by ghost Loading

trofi commented Aug 23, 2022 • edited Loading

SuperSandro2000 Aug 23, 2022

Choose a reason for hiding this comment

winterqt Aug 24, 2022

Choose a reason for hiding this comment

Mindavi Sep 2, 2022

Choose a reason for hiding this comment

trofi commented Aug 24, 2022 • edited Loading

trofi commented Aug 25, 2022 • edited Loading

lovesegfault commented Aug 26, 2022

trofi commented Aug 26, 2022

vcunat commented Aug 27, 2022

trofi commented Aug 29, 2022 • edited Loading

trofi commented Sep 2, 2022

lovesegfault commented Sep 2, 2022

trofi commented Sep 2, 2022

risicle commented Sep 4, 2022

trofi commented Sep 4, 2022 • edited Loading

risicle commented Sep 4, 2022

trofi commented Sep 4, 2022

trofi commented Sep 4, 2022

trofi commented Sep 4, 2022

trofi commented Sep 5, 2022

alyssais commented Sep 12, 2022

alyssais commented Sep 12, 2022 • edited Loading

trofi commented Sep 12, 2022

trofi commented Aug 5, 2022 •

edited

Loading

trofi commented Aug 5, 2022 •

edited

Loading

trofi commented Aug 6, 2022 •

edited

Loading

trofi commented Aug 6, 2022 •

edited

Loading

trofi commented Aug 6, 2022 •

edited

Loading

trofi commented Aug 11, 2022 •

edited

Loading

trofi commented Aug 14, 2022 •

edited

Loading

ghost commented Aug 17, 2022 •

edited by ghost

Loading

trofi commented Aug 23, 2022 •

edited

Loading

trofi commented Aug 24, 2022 •

edited

Loading

trofi commented Aug 25, 2022 •

edited

Loading

trofi commented Aug 29, 2022 •

edited

Loading

trofi commented Sep 4, 2022 •

edited

Loading

alyssais commented Sep 12, 2022 •

edited

Loading