Link to libatomic for 32-bit builds

[fhunleth]

libatomic functions are being put into executables for 32-bit platforms. 64-bit targets are unaffected (I checked aarch64 and riscv64). This seems to affect other projects like rust-openssl (see https://github.com/sfackler/rust-openssl/pull/1547/files). The fix hopefully is the same. Given my unfamiliarity with Rustler and RustlerPrecompiled, I'm not sure where the best place is to add the cargo:rustc-link-lib=dylib=atomic option that the rust-openssl PR had.

Specifically, this causes libraries using RustlerPrecompiled to fail to load. The error message for libexplorer-v0.5.4-nif-2.16-arm-unknown-linux-gnueabihf.so is that __atomic_store_8 can't be found.

Running readelf -d on libexplorer-v0.5.4-nif-2.16-arm-unknown-linux-gnueabihf.so shows the following:

Dynamic section at offset 0x1962b40 contains 33 entries:
  Tag        Type                         Name/Value
 0x00000001 (NEEDED)                     Shared library: [libgcc_s.so.1]
 0x00000001 (NEEDED)                     Shared library: [librt.so.1]
 0x00000001 (NEEDED)                     Shared library: [libpthread.so.0]
 0x00000001 (NEEDED)                     Shared library: [libm.so.6]
 0x00000001 (NEEDED)                     Shared library: [libdl.so.2]
 0x00000001 (NEEDED)                     Shared library: [libc.so.6]
 0x00000001 (NEEDED)                     Shared library: [ld-linux-armhf.so.3]

You can see that libatomic.so.1 isn't in the list of needed shared libraries. However, if you run readelf -s on libexplorer-v0.5.4-nif-2.16-arm-unknown-linux-gnueabihf.so and grep for atomic, you'll see:

   66: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND __atomic_store_8
    83: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND __atomic_load_8
   105: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND __atomic_compare[...]
   151: 00000000     0 NOTYPE  GLOBAL DEFAULT  UND __atomic_fetch_add_8

So libatomic is being used. Adding libatomic.so.1 to the needed shared libraries list fixes that. I manually added it with patchelf:

$ patchelf --add-needed libatomic.so.1 ./_build/rpi3_dev/lib/explorer/priv/native/libexplorer-v0.5.4-nif-2.16-arm-unknown-linux-gnueabihf.so
$ readelf -d ./_build/rpi3_dev/lib/explorer/priv/native/libexplorer-v0.5.4-nif-2.16-arm-unknown-linux-gnueabihf.so          bumblebee*

Dynamic section at offset 0x2247000 contains 34 entries:
  Tag        Type                         Name/Value
 0x00000001 (NEEDED)                     Shared library: [libatomic.so.1]
 0x00000001 (NEEDED)                     Shared library: [libgcc_s.so.1]
 0x00000001 (NEEDED)                     Shared library: [librt.so.1]
 0x00000001 (NEEDED)                     Shared library: [libpthread.so.0]
 0x00000001 (NEEDED)                     Shared library: [libm.so.6]
 0x00000001 (NEEDED)                     Shared library: [libdl.so.2]
 0x00000001 (NEEDED)                     Shared library: [libc.so.6]
 0x00000001 (NEEDED)                     Shared library: [ld-linux-armhf.so.3]

Once I did that, explorer loaded and worked fine on the Raspberry Pi 3 which was being run in 32-bit mode w/ Nerves.

All that I believe that needs to be done is to add cargo:rustc-link-lib=dylib=atomic to the right place. Checking that it worked can be done with readelf -d, so there's no need to run on a Raspberry Pi 3 or anything. I don't know of a downside of unconditionally adding it for 32-bit Rust builds.

I hope this makes sense and there's a good place to put it.

[philss]

Thanks for the details! ❤️

I'm not sure where to put that, but I guess it's in the .cargo file. I just don't know the syntax. Maybe it's in the "rustflags".

I found this option described in the docs: https://doc.rust-lang.org/rustc/command-line-arguments.html#option-l-link-lib
The equivalent in rustc would be something like -l dylib=atomic.

I can´t try it right now, but tomorrow I can test that.

[philss]

I was able to add the libatomic to the list by setting the option in the .cargo/config file.

Here is the output of the command readelf -d target/arm-unknown-linux-gnueabihf/debug/libexplorer.so after the change :

Dynamic section at offset 0x4aba34c contains 34 entries:
  Tag        Type                         Name/Value
 0x00000001 (NEEDED)                     Shared library: [libatomic.so.1]
 0x00000001 (NEEDED)                     Shared library: [libgcc_s.so.1]
 0x00000001 (NEEDED)                     Shared library: [librt.so.1]
 0x00000001 (NEEDED)                     Shared library: [libpthread.so.0]
 0x00000001 (NEEDED)                     Shared library: [libm.so.6]
 0x00000001 (NEEDED)                     Shared library: [libdl.so.2]
 0x00000001 (NEEDED)                     Shared library: [libc.so.6]
 0x00000001 (NEEDED)                     Shared library: [ld-linux-armhf.so.3]
 0x0000000c (INIT)                       0x8f238
 0x0000000d (FINI)                       0x440cb70
 0x00000019 (INIT_ARRAY)                 0x4a32cb0
 0x0000001b (INIT_ARRAYSZ)               12 (bytes)
 0x0000001a (FINI_ARRAY)                 0x4a32cbc
 0x0000001c (FINI_ARRAYSZ)               4 (bytes)
 0x00000004 (HASH)                       0x114
 0x6ffffef5 (GNU_HASH)                   0x818
 0x00000005 (STRTAB)                     0x1980
 0x00000006 (SYMTAB)                     0x9e0
 0x0000000a (STRSZ)                      3969 (bytes)
 0x0000000b (SYMENT)                     16 (bytes)
 0x00000003 (PLTGOT)                     0x4aca47c
 0x00000002 (PLTRELSZ)                   1560 (bytes)
 0x00000014 (PLTREL)                     REL
 0x00000017 (JMPREL)                     0x8ec20
 0x00000011 (REL)                        0x2c28
 0x00000012 (RELSZ)                      573432 (bytes)
 0x00000013 (RELENT)                     8 (bytes)
 0x0000001e (FLAGS)                      BIND_NOW STATIC_TLS
 0x6ffffffb (FLAGS_1)                    Flags: NOW
 0x6ffffffe (VERNEED)                    0x2af8
 0x6fffffff (VERNEEDNUM)                 8
 0x6ffffff0 (VERSYM)                     0x2902
 0x6ffffffa (RELCOUNT)                   71652
 0x00000000 (NULL)                       0x0

Here is the PR: elixir-explorer/explorer#544

I plan to release a new version with the fix.

[fhunleth]

Nice! Looks like it worked! Thanks!

[philss]

@fhunleth version 0.5.5 is available. Can you give it a try and see if it is really working?

Also, do you think that this problem affects all the NIFs using Rustler? Probably not, right?

[fhunleth]

@philss I'm running explorer v0.5.5 right now on a Raspberry Pi 3 and it works!

I wish I knew an easy way to tell when libatomic is needed without running readelf or finding out the hard way. I'll keep an eye out. tokenizers and html5ever are fine as is.

[philss]

I'm closing this for now. But I think it would worth to add a troubleshooting guide to this types of errors. I just don't know how to point the user to the "missing link". Anyway, if appears again, we can try to add this guide.