Performance: Layout::align() can assume, that the alignment is a power of two

[TimDiekmann]

An alignment is guaranteed to be a power of two. However without intrinsics it's not possible to use this guarantee when constructing a new Layout from an old one only changing the size. For testing I used the following code:

pub fn dynamic_slow(layout: Layout, size: usize) -> Result<Layout, LayoutErr>{
    let align = layout.align();
    Layout::from_size_align(size, align)
}

pub fn dynamic_fast(layout: Layout, size: usize) -> Result<Layout, LayoutErr>{
    let align = layout.align();
    unsafe { core::intrinsics::assume(align.is_power_of_two()); }
    Layout::from_size_align(size, align)
}

pub fn static_slow() -> Result<Layout, LayoutErr> {
    dynamic_slow(Layout::new::<u32>(), 12)
}

pub fn static_fast() -> Result<Layout, LayoutErr> {
    dynamic_fast(Layout::new::<u32>(), 12)
}

dynamic_fast assumes, that the alignment returned from Layout is a power of two. This results in this assembly:

example::dynamic_slow:
        lea     rax, [rsi - 1]
        mov     r8, rsi
        neg     r8
        xor     ecx, ecx
        test    rsi, rax
        mov     rdi, rsi
        cmovne  rdi, rcx
        test    rsi, rsi
        cmove   rdi, rsi
        mov     rax, rdx
        cmp     r8, rdx
        cmovae  rcx, rdi
        mov     rdx, rcx
        ret

example::dynamic_fast:
        mov     rax, rdx
        mov     rcx, rsi
        neg     rcx
        xor     edx, edx
        cmp     rcx, rax
        cmovae  rdx, rsi
        ret

example::static_slow:
        mov     eax, 12
        mov     edx, 4
        ret

example::static_fast:
        mov     eax, 12
        mov     edx, 4
        ret

While in the static case, this does not matter, the dynamic case don't have to check align.is_power_of_two().

Normally, I would create a pull request, which adds the intrinsic to Layout::align(), but core::intrinsics::assume is not available in const fns.

[nbdd0121]

We could make assume const, similar to likely/unlikely in #73778.

[TimDiekmann]

It's possible to bypass this with core::hint::unreachable_unchecked() which is also stable (at least it's non-const part):

pub fn dynamic_fast(layout: Layout, size: usize) -> Result<Layout, LayoutErr>{
    let align = layout.align();
    if !align.is_power_of_two() {
        unsafe { core::hint::unreachable_unchecked(); }
    }
    Layout::from_size_align(size, align)
}

which results in the same assembly as using assume():

example::dynamic_fast_alt:
        mov     rax, rdx
        mov     rcx, rsi
        neg     rcx
        xor     edx, edx
        cmp     rcx, rax
        cmovae  rdx, rsi
        ret

[nbdd0121]

Ah indeed, and core::hint::unreachable_unchecked is unstably const already (#74459), so we can use that. But it wouldn't hurt to make assume unstably const anyway.

[TimDiekmann]

I never understood, why we have unreachable_unchecked and assume as intrinsic as one could be implemented using the other. But, yes, it wouldn't hurt to make it const 😉

[nbdd0121]

That's simply because LLVM has an assume intrinsic (https://llvm.org/docs/LangRef.html), so there is a Rust version of it. Semantically unreachable_unchecked is more versatile but that would probably require LLVM to do some transformation to turn it into assume.