Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indirect branch tracking #13665

Open
jcs opened this issue Jul 14, 2023 · 9 comments · May be fixed by #15122
Open

Indirect branch tracking #13665

jcs opened this issue Jul 14, 2023 · 9 comments · May be fixed by #15122

Comments

@jcs
Copy link
Contributor

jcs commented Jul 14, 2023

OpenBSD now enforces indirect branch tracking (BTI on arm64, IBT on x86_64) which breaks Crystal on OpenBSD going forward.

I'm not sure whether this would all be handled by llvm and Crystal's compiler just needs to turn it on (in clang it's -fcf-protection=branch) or if there is assembly in Crystal that needs to be modified with the new instructions.

@HertzDevil
Copy link
Contributor

It says that indirect branch tracking is already on by default unless there is an opt-out linker flag, so do we actually need to do anything?

@jcs
Copy link
Contributor Author

jcs commented Jul 14, 2023

Yes, the opt-out is temporary.

@GeopJr
Copy link
Contributor

GeopJr commented Nov 29, 2023

OpenBSD enforces it now and Crystal has been marked as broken

@joshturge
Copy link

I'd like to help get Crystal supported on OpenBSD, what work needs to be done in order to get IBT support? I'm not sure where to start.

@wecing
Copy link

wecing commented Apr 19, 2024

@joshturge maybe check out QBE's implementation

Note that it was reverted due to tooling reasons, but I am trying to do a roll-forward since the blocker seems gone.

UPDATE: or, call linker with -z,nobtcfi (see openbsd/ports@71f21f5)

@ysbaddaden
Copy link
Contributor

We need to tell LLVM to generate the correct instructions on the x86_64 and aarch64 targets —in gcc/clang this is the -fcf-protection=branch option.

I understand we also need to fix the fiber context switch asm for x86_64 and aarch64 to have these too, so we start & resume a fiber?

ysbaddaden added a commit to ysbaddaden/crystal that referenced this issue Oct 22, 2024
OpenBSD 7.4+ enforces Indirect Branch Tracking but we're not yet
compatible in Crystal, so we disable the feature for the time being.
@ysbaddaden
Copy link
Contributor

ysbaddaden commented Oct 23, 2024

Searching for examples, I found out that Rust adds the cf-protection-branch=1 flag when creating a LLVM module when we pass -Z cf-protection=branch to rustc. This is equivalent to -fcf-protection=branch in clang.

See the source code and Rust book.

@ysbaddaden
Copy link
Contributor

The -fcf-protection=branch option enables the control-flow enforcement technology (CET) on x86 only. For AArch64 this is the -mbranch-protection=bti option to enable the branch authentication instructions 😩

Looking at the disassembly of a compiled C program, the x86 option starts all functions with endbr64 while the aarch64 option starts all functions with bti c. We might just need to add these in the ASM of fiber context switches (as the first resume instruction) when the feature is enabled?

There are more specific values to each option, to enable more features. See the links above for details. I don't know about their impact on the ASM of fiber context switches. They're not needed to support OpenBSD; in fact IBT/BTI isn't needed and we could keep it disabled, but it's interesting to follow on the security and see & measure the impact.

@ysbaddaden
Copy link
Contributor

ysbaddaden commented Oct 23, 2024

Initial attempt: c973b01

It only works with --single-module otherwise we fail with incompatible LLVM module / function / context / type, which means I fail to set the values on every LLVM module (I guess). I must have missed something.

I also didn't test fibers. Only a few test files from nanolib and they didn't fail on OpenBSD 🤞

@ysbaddaden ysbaddaden linked a pull request Oct 24, 2024 that will close this issue
ysbaddaden added a commit to ysbaddaden/crystal that referenced this issue Oct 24, 2024
OpenBSD 7.4+ enforces Indirect Branch Tracking but we're not yet
compatible in Crystal, so we disable the feature for the time being.
straight-shoota pushed a commit that referenced this issue Oct 25, 2024
Fixes compatibility with OpenBSD 7.4 that enforced indirect branch tracking by default but we're not compatible yet (see #13665), so we must disable it.

With this patch I can run the std specs, except for the SSL specs because of openssl/libressl mess, as well as the compiler specs, except for the interpreter specs that regularly crash with SIGABRT (may help to debug issues in the interpreter).

Note: the segfault handler is broken on OpenBSD and processes eventually crash with SIGILL after receiving SIGSEGV.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants