Alignment trap seems unable to handle ARM VLDR instruction

[felixfrank]

Describe the bug
I run into SIGBUS crashes due to unaligned memory access via the VLDR instruction in one of my applications running on a raspberry pi. The application in question uses zero-copy networking, which leads to unaligned memory. Specifically, I have a few float values which are not 4 byte aligned. The kernel seems to be fine with using the LDR instruction on unaligned memory, however when the compiler decides to rely on the VLDR instruction, SIGBUS is thrown.

To reproduce
I managed to generate a small code example which exactly reproduces what I am seeing. See the code below and compile without optimization. In the additional context you see the generated assembly code for foo::x() and foo::x_id() on my machine. In the first case, the compiler relies on LDR and VMOV, whereas the second function uses VLDR and causes a SIGBUS error. See the excerpt from the journal.

#include <cstring>
#include <iostream>

float ident(float in) { return in; }

struct foo final {
  foo(float _x) : x_(_x) {}

  float x() const { return x_; }
  float x_id() const { return ident(x_); }

  float x_;
};

int main(int argc, char const *argv[]) {
  char *buffer = new char[10];
  foo orig(.2);
  foo *copy = (foo *)(buffer + 2);
  std::memcpy(copy, &orig, sizeof(foo));
  std::cout << "x: " << copy->x() << std::endl;
  std::cout << "x_id: " << copy->x_id() << std::endl;
  delete[] buffer;
  return 0;
}

Expected behaviour
I'm not entirely sure whether this is a kernel bug and the kernel should be able to handle this unaligned memory access. Based on the ARM manual the VLDR instruction should be able to handle an unaligned memory address, so the alignment trap shouldn't even trigger? The comments in the respective kernel code do not talk about the VLDR instruction.

Actual behaviour
The process crashes with SIGBUS.

System
Copy and paste the results of the raspinfo command in to this section. Alternatively, copy and paste a pastebin link, or add answers to the following questions:

• gcc -v

Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/arm-linux-gnueabihf/8/lto-wrapper
Target: arm-linux-gnueabihf
Configured with: ../src/configure -v --with-pkgversion='Raspbian 8.3.0-6+rpi1' --with-bugurl=file:///usr/share/doc/gcc-8/README.Bugs --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++ --prefix=/usr --with-gcc-major-version-only --program-suffix=-8 --program-prefix=arm-linux-gnueabihf- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-bootstrap --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libitm --disable-libquadmath --disable-libquadmath-support --enable-plugin --with-system-zlib --with-target-system-zlib --enable-objc-gc=auto --enable-multiarch --disable-sjlj-exceptions --with-arch=armv6 --with-fpu=vfp --with-float=hard --disable-werror --enable-checking=release --build=arm-linux-gnueabihf --host=arm-linux-gnueabihf --target=arm-linux-gnueabihf
Thread model: posix
gcc version 8.3.0 (Raspbian 8.3.0-6+rpi1)

• Which model of Raspberry Pi?
  Pi3B
• Which OS and version (cat /etc/rpi-issue)?

Raspberry Pi reference 2019-07-10
Generated using pi-gen, https://github.com/RPi-Distro/pi-gen, 175dfb027ffabd4b8d5080097af0e51ed9a4a56c, stage2

• Which firmware version (vcgencmd version)?

Jul  9 2019 14:40:53
Copyright (c) 2012 Broadcom
version 6c3fe3f096a93de3b34252ad98cdccadeb534be2 (clean) (release) (start)

• Which kernel version (uname -a)?

Linux raspberrypi 4.19.57-v7+ #1244 SMP Thu Jul 4 18:45:25 BST 2019 armv7l GNU/Linux

Logs

Jul 23 17:00:45 raspberrypi kernel: Alignment trap: a.out (1119) PC=0x0001098c Instr=0xedd37a00 Address=0x01cd005a FSR 0x001
Jul 23 17:00:45 raspberrypi kernel: Alignment trap: not handling instruction edd37a00 at [<0001098c>]
Jul 23 17:00:45 raspberrypi kernel: Unhandled fault: alignment exception (0x001) at 0x01cd005a
Jul 23 17:00:45 raspberrypi kernel: pgd = 3e055f1c
Jul 23 17:00:45 raspberrypi kernel: [01cd005a] *pgd=15bf3835, *pte=1322d75f, *ppte=1322dc7f

Additional context
Alignment trap is setup to warn+fixup

pi@raspberrypi:~ $ cat /proc/cpu/alignment
User:           7
System:         0 (  (null))
Skipped:        7
Half:           0
Word:           0
DWord:          0
Multi:          0
User faults:    3 (fixup+warn)

0001094c <foo::x() const>:
   1094c:	e52db004 	push	{fp}		; (str fp, [sp, #-4]!)
   10950:	e28db000 	add	fp, sp, #0
   10954:	e24dd00c 	sub	sp, sp, #12
   10958:	e50b0008 	str	r0, [fp, #-8]
   1095c:	e51b3008 	ldr	r3, [fp, #-8]
   10960:	e5933000 	ldr	r3, [r3]
   10964:	ee073a90 	vmov	s15, r3
   10968:	eeb00a67 	vmov.f32	s0, s15
   1096c:	e28bd000 	add	sp, fp, #0
   10970:	e49db004 	pop	{fp}		; (ldr fp, [sp], #4)
   10974:	e12fff1e 	bx	lr

00010978 <foo::x_id() const>:
   10978:	e92d4800 	push	{fp, lr}
   1097c:	e28db004 	add	fp, sp, #4
   10980:	e24dd008 	sub	sp, sp, #8
   10984:	e50b0008 	str	r0, [fp, #-8]
   10988:	e51b3008 	ldr	r3, [fp, #-8]
   1098c:	edd37a00 	vldr	s15, [r3]
   10990:	eeb00a67 	vmov.f32	s0, s15
   10994:	ebffff78 	bl	1077c <ident(float)>
   10998:	eef07a40 	vmov.f32	s15, s0
   1099c:	eeb00a67 	vmov.f32	s0, s15
   109a0:	e24bd004 	sub	sp, fp, #4
   109a4:	e8bd8800 	pop	{fp, pc}

[trejan]

ARMv7-A architecture ref manual says VLDR must be word aligned or you'll cause an alignment fault (A3.2.1)

[felixfrank]

You're right, I missed that. My question remains though:
Can't the alignment trap detect those unaligned accesses and fix them? Looking at the assembly code, a simple recipe would be to jump over the vldr instruction and use ldr + vmov to load the value into the floating point register. The reference manual states in A3.2.1 that ldr can handle unaligned memory.

[pelwell]

If you want to contribute VLDR alignment fixups to the Linux kernel then go ahead, but we aren't going to be spending our time on it.

[felixfrank]

I case anyone stumbles on this...
I talked to a linux kernel maintainer: the alignment trap could be used to handle this problem, but it's awfully slow because it requires a context switch. gcc assumes memory is aligned naturally, so if you know that specific structs may be unaligned declare them with __attribute__((packed)). Then gcc will only use instructions which work with unaligned memory as well.