arm-unknown-linux-gnueabihf generates armv7 instructions

[ddemarco5]

I'm working on a dual platform embedded project. One is running an ARMv7 processor, and one is running on ARMv6 (RPI1 B+).
The ARMv7 platform runs without issue, however, trying to get ARMv6 code generated has caused problems.
When running the binary on the ARMv6 target, I fail with an illegal instruction.
Tracing on it reveals cbz r3, 0x7f55a264, a compare and branch on 0... an ARMv7 instruction, absent from the ARMv6 instruction set.

Here is my target in the cargo config:

linker = "arm-linux-gnueabihf-gcc"
rustflags = [
	#"-Ctarget-cpu=generic",
	"-Ctarget-feature=+armv6",
	"-Clink-args=-march=armv6"
	]

The broadcom SOC isn't an available target-cpu (oddly, all the cpus in the target list are armv7 processors), so I tried generic, I've tried enabling the armv6 feature and disabling a handful of armv7 features, but had no luck. I've passed args to the linker to ensure the target arch is indeed armv6 as far as it's concerned, but armv7 instructions are still getting through.

I've tried a couple options, but the supported targets list at https://forge.rust-lang.org/platform-support.html
arm-unknown-linux-gnueabihf | ✓ | ✓ | ✓ | ARMv6 Linux, hardfloat
is misleading.

Is there a way to disable all target-features except the ones you explicitly enable? That might be a possible workaround for the time being.

[parched]

Hi @ddemarco5, just to be sure, have you checked if this happens in rust code not C code?

Also, the target cpu you want for the RPi1 is arm1176jzf-s.

[ddemarco5]

@parched this is compiling rust code with rustc via cargo, not compiling any C code to my knowledge.

I set the target cpu to arm1176jzf-s, outcome didn't change.

[arielb1]

@ddemarco5

Could you find out from which function did the problem instruction come from, by using objdump -rld on the binary and searching for the offending instruction?

[ddemarco5]

The instruction that hits first is in "deregister_tm_clones", but it appears elsewhere as well.

Here's the output of a grep on the file for cbz. I'd be happy to upload the full file if anyone needs it.

    5260:	b103      	cbz	r3, 5264 <deregister_tm_clones+0x1c>
    5294:	b103      	cbz	r3, 5298 <register_tm_clones+0x20>
    52be:	b123      	cbz	r3, 52ca <__do_global_dtors_aux+0x1e>
   8cdf8:	b11b      	cbz	r3, 8ce02 <atexit+0x12>
   8ce14:	b103      	cbz	r3, 8ce18 <__pthread_atfork+0x8>

[parched]

@parched this is compiling rust code with rustc via cargo, not compiling any C code to my knowledge.

Yes but the standard library depends on C code, e.g., libc, and the compiler also generates calls to it's own runtime, some of which are written in C.

I set the target cpu to arm1176jzf-s, outcome didn't change.

arm-unknown-linux-gnueabihf should just work anyway so I wouldn't expect setting the target cpu to make a difference here, it just helps the compiler possibly generate more optimal code for that cpu.

[parched]

The instruction that hits first is in "deregister_tm_clones", but it appears elsewhere as well.

I think that's in the C runtime startup code so I suspect you are using a Armv7 toolchain (which most named arm-unknown-linux-gneabihf-gcc are). Where did you get this compiler from?

[parched]

Also what Linux distribution are you using on the RPi1? To my knowledge, there aren't any hard-float distros for RPi1, so maybe you want to use the arm-unknown-linux-gnueabi rust target.

[ddemarco5]

I pulled gcc-arm-linux-gnueabihf from the standard debian repos. I'm using the newest version of raspbian on the pi. I'll try without the hardfloat.

[jonas-schievink]

There are plenty of hard-float distros, Raspbian uses hard floats, as does Arch Linux ARM. I'm deploying to a similar target locally (RPi 1 Model B) and successfully use the arm-unknown-linux-gnueabihf target.

[ddemarco5]

I checked my distro and it does in fact use hardfloat, so I stopped trying to compile it for the softfloat target. I'm currently operating under the assumption that the arm-unknown-linux-gnueabi compiler I have is an armv7 version. I'm trying to get a different version.
@jonas-schievink can you paste your uname -a and /proc/cpuinfo?

[jonas-schievink]

Sure:

jonas@jonas-raspi ~> uname -a
Linux jonas-raspi 4.9.52-1-ARCH #1 SMP Sat Sep 30 01:00:26 UTC 2017 armv6l GNU/Linux
jonas@jonas-raspi ~> cat /proc/cpuinfo
processor	: 0
model name	: ARMv6-compatible processor rev 7 (v6l)
BogoMIPS	: 697.95
Features	: half thumb fastmult vfp edsp java tls 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xb76
CPU revision	: 7

Hardware	: BCM2835
Revision	: 0002
Serial		: 000000002fdbf225

Note that I'm using nightly-2017-09-15, not the current nightly (haven't checked if that one works).

[ddemarco5]

I'm building a toolchain specific to the RPI B+. If that solves the issue I'll go ahead and close the issue, as it appears to be a problem of libc being compiled for ARMv7.

[parched]

There are plenty of hard-float distros.

Ah yes you're quite right, I was just thinking of the standard debian ports armel and armhf.

@ddemarco5 Yes, I think the debian gcc-arm-linux-gnueabihf toolchain is for the armhf port. Possibly try this one https://github.com/raspberrypi/tools/tree/master/arm-bcm2708/arm-rpi-4.9.3-linux-gnueabihf.

[jonas-schievink]

My project works fine with the current nightly, but I'm using the libc that's installed on the Pi.

[ddemarco5]

Built a toolchain with libc included targeted toward ARMv6 and the Pi B+ and that fixed the issue, thanks for the help guys.

[apiraino]

@ddemarco5 sorry to comment on this old issue, but I'm facing the same problem. I can't figure out the flags to pass in the ~/.cargo/config file to successfully xcompile for the Raspi1.

Can anyone give me some directions? I'm targeting arm-unknown-linux-gnueabihf and I get segfaults when running the binary on the raspi1

I've tried compiling with rustc with the following cmd (and similar permutations):
CFLAGS="-march=armv6+fp -marm -mfloat-abi=hard -mfpu=vpf" rustc --target=arm-unknown-linux-gnueabihf -C linker=arm-linux-gnueabihf-gcc main.rs

Is this the right way to pass flags to the compiler?

p.s. I can also target arm-unknown-linux-gnueabi (minus the hf) and it works but then I have other problems (not relevant to the issue).

thanks!

[ddemarco5]

Oof. This was a while ago. I read the bug through again to try to refresh myself...

It looks like you're getting confused as I was, that target you're using says it works for the rpi1, but the default arm-linux-gnueabihf toolchain you get from pretty much any distro with apt-get, yum, etc... if built for ARMv7.

So when you go to compile your program, some ARMv7 code is included (not sure if it's in the compile or link step, I need a refresher on rust's libc).

The solution is to compile (or find) another version of the gcc toolchain targeted specifically for ARMv6.

I would recommend buildroot (https://buildroot.org/), it takes most of the pain out of the process.

Once you have that compiled and good to go, you need to go and configure cargo to use that toolchain for the triple you're using to compile for the RPI1.

You can see a little bit of an example of that here.
https://doc.rust-lang.org/cargo/reference/config.html

I hope this sets you on the right path! Again, it's been a while so some of the information here may be a bit off here any there... but I hope it gets you going in the right direction.

[apiraino]

uhm ... interesting but also a bit of a let down. Reading this answer on S.O. I thought I could tweak the flags to have the right executable for armv6.

The bad surprise is that arm-linux-gnueabihf on the Ubuntu toolchain might not match what I have on the raspi1, same triple as declared by dpkg:

pi@raspberrypi:~ $ dpkg-architecture 
DEB_BUILD_ARCH=armhf
DEB_BUILD_ARCH_ABI=eabihf
DEB_BUILD_ARCH_BITS=32
DEB_BUILD_ARCH_CPU=arm
DEB_BUILD_ARCH_ENDIAN=little
DEB_BUILD_ARCH_LIBC=gnu
DEB_BUILD_ARCH_OS=linux
DEB_BUILD_GNU_CPU=arm
DEB_BUILD_GNU_SYSTEM=linux-gnueabihf
DEB_BUILD_GNU_TYPE=arm-linux-gnueabihf
DEB_BUILD_MULTIARCH=arm-linux-gnueabihf
DEB_HOST_ARCH=armhf
DEB_HOST_ARCH_ABI=eabihf
DEB_HOST_ARCH_BITS=32
DEB_HOST_ARCH_CPU=arm
DEB_HOST_ARCH_ENDIAN=little
DEB_HOST_ARCH_LIBC=gnu
DEB_HOST_ARCH_OS=linux
DEB_HOST_GNU_CPU=arm
DEB_HOST_GNU_SYSTEM=linux-gnueabihf
DEB_HOST_GNU_TYPE=arm-linux-gnueabihf
DEB_HOST_MULTIARCH=arm-linux-gnueabihf
DEB_TARGET_ARCH=armhf
DEB_TARGET_ARCH_ABI=eabihf
DEB_TARGET_ARCH_BITS=32
DEB_TARGET_ARCH_CPU=arm
DEB_TARGET_ARCH_ENDIAN=little
DEB_TARGET_ARCH_LIBC=gnu
DEB_TARGET_ARCH_OS=linux
DEB_TARGET_GNU_CPU=arm
DEB_TARGET_GNU_SYSTEM=linux-gnueabihf
DEB_TARGET_GNU_TYPE=arm-linux-gnueabihf
DEB_TARGET_MULTIARCH=arm-linux-gnueabihf

This is all pretty confusing. Anyway, thanks for the heads-up 👍 I'll explore your suggestion, too.

[ddemarco5]

Welcome to the reason people like interpreted languages! :P It can feel like a lot to take on, and error messages are rarely helpful, but you'll start to get the hang of it. Once you see enough of this stuff, the picture starts to come together.