RUSTFLAGS="-Z cf-protection=full" gives prologue in naked functions for x86_64-unknown-none

[haraldh]

I tried this code:

#![feature(naked_functions)]
#![feature(start)]
#![no_std]
#![no_main]

use core::arch::asm;

#[panic_handler]
fn panic(_info: &core::panic::PanicInfo<'_>) -> ! {
    loop {}
}

#[no_mangle]
#[naked]
pub unsafe extern "sysv64" fn _hlt() -> ! {
    asm!("hlt", options(noreturn))
}

#[start]
#[no_mangle]
pub fn _start(_a: isize, _b: *const *const u8) -> isize {
    unsafe { _hlt() }
}

I expected to see this happen:

no endbr64 prologue in _hlt naked function.

❯ RUSTFLAGS="-Z cf-protection=full" cargo +nightly build --target x86_64-unknown-none 
   Compiling cet-naked v0.1.0 (/home/harald/CLionProjects/cet-naked)
    Finished dev [unoptimized + debuginfo] target(s) in 0.44s

❯ objdump --disassembler-options=intel --disassemble target/x86_64-unknown-none/debug/cet-naked

target/x86_64-unknown-none/debug/cet-naked:     file format elf64-x86-64


Disassembly of section .text:

0000000000001210 <_hlt>:
    1210:	f4                   	hlt    
    1211:	0f 0b                	ud2    
    1213:	cc                   	int3   
    1214:	cc                   	int3   
    1215:	cc                   	int3   
    1216:	cc                   	int3   
    1217:	cc                   	int3   
    1218:	cc                   	int3   
    1219:	cc                   	int3   
    121a:	cc                   	int3   
    121b:	cc                   	int3   
    121c:	cc                   	int3   
    121d:	cc                   	int3   
    121e:	cc                   	int3   
    121f:	cc                   	int3   

0000000000001220 <_start>:
    1220:	f3 0f 1e fa          	endbr64 
    1224:	48 83 ec 18          	sub    rsp,0x18
    1228:	48 89 7c 24 08       	mov    QWORD PTR [rsp+0x8],rdi
    122d:	48 89 74 24 10       	mov    QWORD PTR [rsp+0x10],rsi
    1232:	e8 d9 ff ff ff       	call   1210 <_hlt>
    1237:	0f 0b                	ud2    

Instead, this happened: an unwanted endbr64 prologue in a naked function

❯ RUSTFLAGS="-Z cf-protection=full" cargo +nightly build --target x86_64-unknown-none 
   Compiling cet-naked v0.1.0 (/home/harald/CLionProjects/cet-naked)
    Finished dev [unoptimized + debuginfo] target(s) in 0.44s

❯ objdump --disassembler-options=intel --disassemble target/x86_64-unknown-none/debug/cet-naked

target/x86_64-unknown-none/debug/cet-naked:     file format elf64-x86-64


Disassembly of section .text:

0000000000001210 <_hlt>:
    1210:	f3 0f 1e fa          	endbr64 
    1214:	f4                   	hlt    
    1215:	0f 0b                	ud2    
    1217:	cc                   	int3   
    1218:	cc                   	int3   
    1219:	cc                   	int3   
    121a:	cc                   	int3   
    121b:	cc                   	int3   
    121c:	cc                   	int3   
    121d:	cc                   	int3   
    121e:	cc                   	int3   
    121f:	cc                   	int3   

0000000000001220 <_start>:
    1220:	f3 0f 1e fa          	endbr64 
    1224:	48 83 ec 18          	sub    rsp,0x18
    1228:	48 89 7c 24 08       	mov    QWORD PTR [rsp+0x8],rdi
    122d:	48 89 74 24 10       	mov    QWORD PTR [rsp+0x10],rsi
    1232:	e8 d9 ff ff ff       	call   1210 <_hlt>
    1237:	0f 0b                	ud2    

Meta

rustc --version --verbose:

rustc 1.64.0-nightly (7425fb293 2022-06-30)
binary: rustc
commit-hash: 7425fb293f510a6f138e82a963a3bc599a5b9e1c
commit-date: 2022-06-30
host: x86_64-unknown-linux-gnu
release: 1.64.0-nightly
LLVM version: 14.0.6

Backtrace

<backtrace>

[haraldh]

CC @npmccallum

[workingjubilee]

Not immediately clear which one is the bug. Naked functions are supposed to be prologue-free in the sense of having no instructions that e.g. modify parameter passing, but cf-protection=full or cf-protection=branch require marking branch targets in code using endbr. This can be seen as a "prologue" in a strict sense but it is required in order to fulfill the requests that are made to the compiler.

To clarify: Do you want the ability to make a naked function that, with CET enabled, cannot be indirectly branched to without IBT kicking in, then?

[haraldh]

either way, it would be nice to have a cfg to know, if the endbr64 happens or not.

[haraldh]

In my case I was resembling the reset vector page with a naked function, so any addition voids the expectation. I can live with the endbr64 if there was a cfg for the CET flags, which results in the prolog.

[npmccallum]

Not immediately clear which one is the bug. Naked functions are supposed to be prologue-free in the sense of having no instructions that e.g. modify parameter passing, but cf-protection=full or cf-protection=branch require marking branch targets in code using endbr. This can be seen as a "prologue" in a strict sense but it is required in order to fulfill the requests that are made to the compiler.

This is not the definition of constrained naked functions that has been adopted via the RFC. That definition is stronger and insists that no instructions are emitted before the inline assembly in naked functions.

The reason for this is obvious: any other behavior creates an unknown set of interactions between the generated instructions multiplied across all possible compiler features.

The author that chooses a naked function gets no help from the compiler and has to implement all such features manually. This is the only way to control the interactions.

(Note that the author CAN detect, using compile-time features whether to emit a CET-enabled block of asm or not.)

[workingjubilee]

Fair enough. 🤔 Then the question is, why would this happen with the x86_64-unknown-none target and not the -unknown-linux-gnu target?

...Actually, checking Godbolt, there seems to be no difference?
https://rust.godbolt.org/z/WMqe85G38

So that answers that.
It looks like cf-protection-branch is applied as a module-level attribute here. Extracting naked functions into their own modules just to get them to compile correctly seems a bit heavyweight. But we could choose to also attach nocf_check as a flag to naked functions. That may have additional consequences, which I am not clear on, but it seems to actually strip the resulting endbr64:
https://llvm.godbolt.org/z/3dhj91YY8

It might also be useful to query upstream with LLVM as well if this is intended behavior on their part to begin with, as it's not clear to me that LLVM's naked function attribute should not be stripping endbr on its own. They are, after all, supposed to be naked.

[workingjubilee]

I have a patch which fixes this for x86, but with -Zbranch-protection="bti" (essentially the AArch64 version the same option) and otherwise equivalent Rust code, I get this assembly (Godbolt):

rust_begin_unwind:
        hint    #34
        b       .LBB0_1
.LBB0_1:
        b       .LBB0_1

_hlt:
        hint    #34
        hlt     #0x1
        brk     #0x1

_start:
        hint    #34
        str     x30, [sp, #-16]!
        bl      _hlt
        brk     #0x1

llc does not appear to respect the NoCfCheck attribute for non-x86 platforms. And yes, the hint instructions disappear if -Zbranch-protection is not enabled... it seems to be the same issue. Setting "branch-target-enforcement"="0" (or "false") on the function seems to fix this, also. This might get quite annoying if not patched in LLVM instead...

[haraldh]

Fair enough. thinking Then the question is, why would this happen with the x86_64-unknown-none target and not the -unknown-linux-gnu target?

...Actually, checking Godbolt, there seems to be no difference? https://rust.godbolt.org/z/WMqe85G38

Sorry, might have been a glitch in my configuration. Can't reproduce it again either.

[workingjubilee]

Fixed in #98998. An alternative proposal for how to fix this and other issues with naked fn, that was brought up in that PR, is Rust lowering naked functions into global asm blocks which then can get compiled separately-ish.