__builtin_bswap32 referenced symbol not found

[gco]

The latest borg git repo refers to the __builtin_bswap32 symbol when built on Big Endian architectures without using gcc. (Solaris 10, Sun's compiler).

% borg
Traceback (most recent call last):
  File "...python/borg-env/bin/borg", line 7, in <module>
    from borg.archiver import main
  File "...python/borg-env/lib/python3.6/site-packages/borg/archiver.py", line 36, in <module>
    from . import helpers
  File ".../borg-env/lib/python3.6/site-packages/borg/helpers.py", line 44, in <module>
    from . import hashindex
ImportError: ld.so.1: python3.6: fatal: relocation error:
    file .../python/borg-env/lib/python3.6/site-packages/borg/hashindex.cpython-36m.so:
    symbol __builtin_bswap32: referenced symbol not found

The macros in src/borg/_endian.h are to blame.

[ThomasWaldmann]

Well, if it wants to use the __builtin_bswap32, at least the big endian detection worked ok.

Your problem is just that you do not have that function (because you do not use gcc?), so it would need to detect either the absence of that or the other compiler and use whatever is the replacement for that case.

[ThomasWaldmann]

Note:

• it looks like this issue applies to 1.0-maint, 1.1-maint and master branches.
• i personally have no means to develop / test the required change (i don't have solaris on big-endian, not even openindiana on little-endian right now as vagrant is broken for OI)

[gco]

It's not Solaris- or SPARC-specific. The same thing happens on any big-endian machines. You're not testing on IBM z/Architecture? :-)

The usual code looks something like what the simple-minded implementation of ntohl looks like on little-endian machines:

x = ((y>>24)&0xff) | ((y<<8)&0xff0000) | ((y>>8)&0xff00) | ((y<<24)&0xff000000);

[ThomasWaldmann]

I am running a check on (big endian) powerpc debian linux right now (with gcc). But I expect it to work and you said yourself (see top post) that it happens without gcc when using Sun's compiler. So it isn't that far fetched that the code needs to be adapted for that compiler (which is only used on Solaris AFAIK).

[gco]

The issue isn't with Sun's compiler, it's the code that relies on a very specific compiler. Use portable byte-swapping code and it'll work with any compiler.

[ThomasWaldmann]

OK, but fixing the issue still needs someone with a platform and compiler who can test the non-gcc code that has to be added.

[despair86]

hmm
SPARCWorks 12.6 implements some of those GCC intrinsic functions, I'd have to look at the docs again to verify whether or not it has __builtin_bswap32().

On the other hand, __builtin_c[tl]z() do NOT exist there, and I have to patch those inline myself. (The only reason i even use anything other than Microsoft C/C++, gcc, or Apple C/C++ is to maintain the ability to build/bootstrap software in the absence of any of these compilers: the UNIX C compilers are somewhat similar to what current PCC is now, given that the former descend of the original non-std PCC + ANSI compliance)

[despair86]

most of them are unimplemented even now, see https://docs.oracle.com/cd/E77782_01/html/E77792/gqexw.html#OSGCCgqexp and /opt/SUNWspro/lib/compilers/include/CC/gnu/builtins*.h (or, wherever you installed to.../opt/*/lib/compilers/include/CC/gnu/builtins*.h

for 12.6, only functions used directly in libstdc++ v5.4 were implemented, anything that isn't referenced directly was left out (including __builtin_bswap* sadly)

if it has no inline definitions in those files (builtins.h, builtins-def.h)--all of them are declared however, then it doesn't exist at all (yet, if ever)

[ThomasWaldmann]

@despair86 thanks for checking this.

can you add and test some working code to deal with that (see other comments)?

[despair86]

oh right
the only reason it works in openindiana is because their SFW-gate CBE (common build env) uses GCC 6.5

[ThomasWaldmann]

Is this still an issue with current borg code? 1.2-maint or master branch.

[ThomasWaldmann]

If somebody finds this is still a problem with current versions, please reopen.