Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for control-flow protection #93439

Merged
merged 1 commit into from
Feb 15, 2022
Merged

Conversation

abrown
Copy link
Contributor

@abrown abrown commented Jan 28, 2022

This change adds a flag for configuring control-flow protection in the LLVM backend. In Clang, this flag is exposed as -fcf-protection with options none|branch|return|full. This convention is followed for rustc, though as a codegen option: rustc -Z cf-protection=<none|branch|return|full>. Tracking issue for future work is #93754.

@rustbot rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Jan 28, 2022
@rust-highfive
Copy link
Collaborator

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @petrochenkov (or someone else) soon.

Please see the contribution instructions for more information.

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jan 28, 2022
@abrown abrown marked this pull request as draft January 28, 2022 19:17
@abrown abrown force-pushed the cf-protection branch 2 times, most recently from dfe5a84 to 20c4504 Compare January 28, 2022 19:43
@rust-log-analyzer

This comment has been minimized.

@fweimer-rh
Copy link

  • A more serious issue, I think: if other libraries are not also built with these flags enabled, then LLVM will seek the "lowest-common denominator" and compile without cf-protection enabled; what I observed is that compiling a simple example with -C cf-protection=full doesn't result in the correct ELF tags, presumably because the standard libraries were not compiled with the same flags--any thoughts on this?

The statically linked bits in current Linux distributions are CET-enabled. With binutils ld, you can use -z cet-report=warning to get a report of object files in a link that lack CET markup, and investigate further from there.

compiler/rustc_session/src/options.rs Show resolved Hide resolved
compiler/rustc_session/src/options.rs Outdated Show resolved Hide resolved
compiler/rustc_session/src/options.rs Outdated Show resolved Hide resolved
compiler/rustc_session/src/config.rs Outdated Show resolved Hide resolved
compiler/rustc_codegen_llvm/src/context.rs Outdated Show resolved Hide resolved
@petrochenkov
Copy link
Contributor

I will need someone to point out where such a test would go as I'm not too familiar with the rustc repository

You could search for -C control-flow-guard in ./src/test/codegen, this feature probably can be tested in a similar way, but I'm not sure.

@petrochenkov petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 29, 2022
*slot = match v {
None | Some("none") => CFProtection::None,
Some("branch") => CFProtection::Branch,
Some("return") => CFProtection::Return,
Some("full") => CFProtection::Full,
Some(_) => return false,
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like it'd be pretty hard to extend this kind of CLI in a sensible manner in the future if the underlying CF protection concept changes in any way.

Copy link
Contributor Author

@abrown abrown Feb 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To address this and your comments below about wanting a unified -Zbranch-protection: I agree that it would be nice to unify these various properties (see -Ccf-guard as well) but I think that could reasonably be discussed in a separate issue, mainly because I suspect the various flags are not exactly equivalent and are implemented with different assumptions (e.g., cf-protection expects certain ELF tags, kernel support, and architecture support). If such an issue exists, I would be glad to comment there. But I would not expect to have to figure that out in this PR. I was hoping this could just add support with the same flags that Clang exposes (-fcf-protection=none|branch|return|full).

compiler/rustc_codegen_llvm/src/context.rs Show resolved Hide resolved
compiler/rustc_codegen_llvm/src/context.rs Outdated Show resolved Hide resolved
@nagisa
Copy link
Member

nagisa commented Jan 31, 2022

as a side note, I think ideally we'd figure out an unified interface with -Zbranch-protection, though probably not part of this PR. A flag for every architecture for broadly related features seems very suboptimal.

@abrown
Copy link
Contributor Author

abrown commented Feb 1, 2022

The statically linked bits in current Linux distributions are CET-enabled. With binutils ld, you can use -z cet-report=warning to get a report of object files in a link that lack CET markup, and investigate further from there.

@fweimer-rh: I started down this path but I first ran into the issue that I didn't even know how the linker is being called. Using the rustc built at build/x86_64-unknown-linux-gnu/stage1/bin/rustc, I tried out -Z print-link-args (no effect), eventually settling on the following:

$ RUSTC_LOG=rustc_codegen_ssa::back::link=info rustc-custom -v -Z cf-protection=full -o empty-custom-rust empty.rs

 INFO rustc_codegen_ssa::back::link preparing Executable to "empty-custom-rust"
 INFO rustc_codegen_ssa::back::link "cc" "-m64" "empty-custom-rust.empty.ea9ce8e5-cgu.0.rcgu.o" "empty-custom-rust.empty.ea9ce8e5-cgu.1.rcgu.o" "empty-custom-rust.empty.ea9ce8e5-cgu.2.rcgu.o" "empty-custom-rust.empty.ea9ce8e5-cgu.3.rcgu.o" "empty-custom-rust.empty.ea9ce8e5-cgu.4.rcgu.o" "empty-custom-rust.empty.ea9ce8e5-cgu.5.rcgu.o" "empty-custom-rust.empty.ea9ce8e5-cgu.6.rcgu.o" "empty-custom-rust.30bkn99o4ddymxvy.rcgu.o" "-Wl,--as-needed" "-L" [... some rlib files]

I could then try to run ld -z cet-report=warning on each of these object files but I presume you were thinking this could be done more efficiently? E.g., switch to ld as the linker and then pass arguments to that?

@petrochenkov
Copy link
Contributor

r? @nagisa

@rust-highfive rust-highfive assigned nagisa and unassigned petrochenkov Feb 1, 2022
@fweimer-rh
Copy link

I could then try to run ld -z cet-report=warning on each of these object files but I presume you were thinking this could be done more efficiently? E.g., switch to ld as the linker and then pass arguments to that?

@abrown Presumably cc refers to gcc or clang, then you can use cc -Wl,-z,cet-report=warning. If the linker is BFD-compatible, it should print the warnings. If it does not, you may have to switch to BFD ld using cc -fuse-ld=bfd -Wl,-z,cet-report=warning.

@abrown
Copy link
Contributor Author

abrown commented Feb 2, 2022

@fweimer-rh, thanks, that worked! When I run RUSTC_LOG=rustc_codegen_ssa::back::link=info rustc-custom -v -Z cf-protection=full -C link-arg="-Wl,-z,cet-report=warning" -o empty-custom-rust empty.rs, I can now see all of the modules not compiled with CET enabled that prevent empty.rs from being CET enabled:

Click to expand the rather large list of non-CET objects
INFO rustc_codegen_ssa::back::link linker stderr:
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.12.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.13.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.15.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.2.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.3.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.4.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.5.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.6.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.7.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.8.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.9.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.0.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.1.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.10.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.11.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libstd-d73f7266be14cb8b.rlib(std-d73f7266be14cb8b.std.f7443020-cgu.14.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-95f9ae1f72899b39.rlib(panic_unwind-95f9ae1f72899b39.panic_unwind.861d5c0d-cgu.1.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-95f9ae1f72899b39.rlib(panic_unwind-95f9ae1f72899b39.panic_unwind.861d5c0d-cgu.3.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libpanic_unwind-95f9ae1f72899b39.rlib(panic_unwind-95f9ae1f72899b39.panic_unwind.861d5c0d-cgu.0.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libminiz_oxide-5fecd091d52fbe30.rlib(miniz_oxide-5fecd091d52fbe30.miniz_oxide.1ba51e95-cgu.1.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-f61a898a4c8e4d5d.rlib(adler-f61a898a4c8e4d5d.adler.b9100e7e-cgu.2.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libadler-f61a898a4c8e4d5d.rlib(adler-f61a898a4c8e4d5d.adler.b9100e7e-cgu.4.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-2dde36d2dabf6685.rlib(object-2dde36d2dabf6685.object.77b6cb11-cgu.10.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-2dde36d2dabf6685.rlib(object-2dde36d2dabf6685.object.77b6cb11-cgu.9.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-2dde36d2dabf6685.rlib(object-2dde36d2dabf6685.object.77b6cb11-cgu.8.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-2dde36d2dabf6685.rlib(object-2dde36d2dabf6685.object.77b6cb11-cgu.11.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-2dde36d2dabf6685.rlib(object-2dde36d2dabf6685.object.77b6cb11-cgu.6.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-2dde36d2dabf6685.rlib(object-2dde36d2dabf6685.object.77b6cb11-cgu.7.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-2dde36d2dabf6685.rlib(object-2dde36d2dabf6685.object.77b6cb11-cgu.0.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-2dde36d2dabf6685.rlib(object-2dde36d2dabf6685.object.77b6cb11-cgu.1.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-2dde36d2dabf6685.rlib(object-2dde36d2dabf6685.object.77b6cb11-cgu.14.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-2dde36d2dabf6685.rlib(object-2dde36d2dabf6685.object.77b6cb11-cgu.15.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-2dde36d2dabf6685.rlib(object-2dde36d2dabf6685.object.77b6cb11-cgu.2.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libobject-2dde36d2dabf6685.rlib(object-2dde36d2dabf6685.object.77b6cb11-cgu.3.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libmemchr-31c1e8486a1782f1.rlib(memchr-31c1e8486a1782f1.memchr.dfa49cae-cgu.0.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-db992a4c8ad1d30d.rlib(addr2line-db992a4c8ad1d30d.addr2line.b65cc714-cgu.0.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libaddr2line-db992a4c8ad1d30d.rlib(addr2line-db992a4c8ad1d30d.addr2line.b65cc714-cgu.2.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.10.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.11.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.13.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.14.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.15.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.2.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.5.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.7.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.8.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.9.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.0.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.12.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libgimli-61dad49d899d3572.rlib(gimli-61dad49d899d3572.gimli.43ed73af-cgu.6.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-7cb15cfd59dcb70f.rlib(rustc_demangle-7cb15cfd59dcb70f.rustc_demangle.273f8612-cgu.2.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-7cb15cfd59dcb70f.rlib(rustc_demangle-7cb15cfd59dcb70f.rustc_demangle.273f8612-cgu.3.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-7cb15cfd59dcb70f.rlib(rustc_demangle-7cb15cfd59dcb70f.rustc_demangle.273f8612-cgu.8.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-7cb15cfd59dcb70f.rlib(rustc_demangle-7cb15cfd59dcb70f.rustc_demangle.273f8612-cgu.0.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-7cb15cfd59dcb70f.rlib(rustc_demangle-7cb15cfd59dcb70f.rustc_demangle.273f8612-cgu.1.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-7cb15cfd59dcb70f.rlib(rustc_demangle-7cb15cfd59dcb70f.rustc_demangle.273f8612-cgu.14.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/librustc_demangle-7cb15cfd59dcb70f.rlib(rustc_demangle-7cb15cfd59dcb70f.rustc_demangle.273f8612-cgu.6.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-8e524ba73e835f1e.rlib(alloc-8e524ba73e835f1e.alloc.a1686a60-cgu.0.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-8e524ba73e835f1e.rlib(alloc-8e524ba73e835f1e.alloc.a1686a60-cgu.1.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-8e524ba73e835f1e.rlib(alloc-8e524ba73e835f1e.alloc.a1686a60-cgu.10.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-8e524ba73e835f1e.rlib(alloc-8e524ba73e835f1e.alloc.a1686a60-cgu.12.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-8e524ba73e835f1e.rlib(alloc-8e524ba73e835f1e.alloc.a1686a60-cgu.13.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-8e524ba73e835f1e.rlib(alloc-8e524ba73e835f1e.alloc.a1686a60-cgu.15.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-8e524ba73e835f1e.rlib(alloc-8e524ba73e835f1e.alloc.a1686a60-cgu.2.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/liballoc-8e524ba73e835f1e.rlib(alloc-8e524ba73e835f1e.alloc.a1686a60-cgu.9.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.1.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.10.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.11.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.12.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.14.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.15.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.2.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.4.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.6.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.7.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.8.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.9.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.3.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcore-aac13a4a74d65b56.rlib(core-aac13a4a74d65b56.core.07b87a26-cgu.5.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-5baac75f3dfbf78d.rlib(compiler_builtins-5baac75f3dfbf78d.compiler_builtins.a867b2fc-cgu.29.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-5baac75f3dfbf78d.rlib(compiler_builtins-5baac75f3dfbf78d.compiler_builtins.a867b2fc-cgu.42.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-5baac75f3dfbf78d.rlib(compiler_builtins-5baac75f3dfbf78d.compiler_builtins.a867b2fc-cgu.43.rcgu.o): warning: missing IBT and SHSTK properties
/usr/bin/ld: /home/abrown/Code/rust/build/x86_64-unknown-linux-gnu/stage1/lib/rustlib/x86_64-unknown-linux-gnu/lib/libcompiler_builtins-5baac75f3dfbf78d.rlib(compiler_builtins-5baac75f3dfbf78d.compiler_builtins.a867b2fc-cgu.81.rcgu.o): warning: missing IBT and SHSTK properties

The summary of this is what I suspected: none of the dependencies are CET-enabled so LLVM must seek the the "lowest common denominator." These dependencies include libstd, libpanic_unwind, libobject, libgimli, liballoc, libcore, etc.--pretty much everything. Is there any way in the rustc toolchain to say something like "recompile all dependencies of this object with this flag"?

Alternately, it has been suggested to me that this CET support might require a new target arch, e.g. x86_64cet, in order for all code in all modules to be compiled with the expected LLVM flags. Any thoughts on that?

@nagisa
Copy link
Member

nagisa commented Feb 2, 2022

There is -Zbuild-std option for cargo which allows (re-)building custom standard libraries. A new target or changing default build configuration for existing targets would be the only way to make it work on stable. However the specific approach can be figured out before the underlying functionality is stabilized, and this PR is not stabilizing anything.

@fweimer-rh
Copy link

Alternately, it has been suggested to me that this CET support might require a new target arch, e.g. x86_64cet, in order for all code in all modules to be compiled with the expected LLVM flags. Any thoughts on that?

@abrown Distributions have switched to building statically linked bits with CET enabled (and shared objects anyway), maybe the Rust distribution should do the same. It's relatively easy to do because all x86-64 implementations support long NOPs. It's more difficult for i586 because long NOPs are not part of the ISA there.

@abrown
Copy link
Contributor Author

abrown commented Feb 2, 2022

@nagisa, @fweimer-rh: thanks for the help. Indeed the following command generates a CET-enabled binary:

$ RUSTFLAGS="-Z cf-protection=full" RUSTC="rustc-custom" cargo +nightly build -Z build-std --target x86_64-unknown-linux-gnu
...

$ readelf -a target/x86_64-unknown-linux-gnu/debug/empty | grep feature:
      Properties: x86 feature: IBT, SHSTK

What else do you think is needed in this PR for it to be ready for review--docs, tests?

[edit: @fweimer-rh, I think the idea of switching the default to CET-enabled could be discussed in a separate issue/PR?]

@nagisa
Copy link
Member

nagisa commented Feb 5, 2022

What else do you think is needed in this PR for it to be ready for review--docs, tests?

Unstable features do benefit from docs, yeah. These generally go to the unstable book. Source for it is in src/doc/unstable-book.

You may also want to create an issue with the C-tracking-issue label for tracking remaining work, outstanding questions and the stabilization process for this feature.

Exhaustive tests can be added as a follow-up, I feel, but here are some basic ones which I think are a must-have before this PR can land:

  1. src/test/codegen test for llvm attribute emission;
  2. src/test/codegen test checking that we don't emit any additional attributes unless the flag is specified (at least while the feature is unstable);
  3. you may or may not wish to add a test verifying the assembly output as well.

For the first two I recommend the branch-protection test as an example to follow.

I think the idea of switching the default to CET-enabled could be discussed in a separate issue/PR?

Switching whatever defaults as a follow-up change would make landing this initial work and whatever experimentation much more straightforward for all parties involved.

@abrown
Copy link
Contributor Author

abrown commented Feb 8, 2022

@nagisa: I am going to mark this ready for review. It now has tests, documentation, and I added an issue for tracking further work (#93754; feel free to edit!).

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 14, 2022
@bors
Copy link
Contributor

bors commented Feb 14, 2022

⌛ Testing commit 8d6c973 with merge abd8bddece4c55023e7031274944620dd3799e56...

@bors
Copy link
Contributor

bors commented Feb 14, 2022

💔 Test failed - checks-actions

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 14, 2022
@rust-log-analyzer
Copy link
Collaborator

The job x86_64-gnu-aux failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
error: RPC failed; curl 56 GnuTLS recv error (-54): Error in the pull function.
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
thread 'main' panicked at 'assertion failed: status.success()', src/tools/cargotest/main.rs:125:13
Build completed unsuccessfully in 0:20:49
Build completed unsuccessfully in 0:20:49
make: *** [check-aux] Error 1
Makefile:44: recipe for target 'check-aux' failed

@nagisa
Copy link
Member

nagisa commented Feb 14, 2022

@bors retry

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 14, 2022
@bors
Copy link
Contributor

bors commented Feb 14, 2022

⌛ Testing commit 8d6c973 with merge 5c0ef97094e15c18102978f62856cb6fdf059dc8...

@bors
Copy link
Contributor

bors commented Feb 15, 2022

💥 Test timed out

@bors bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 15, 2022
@rust-log-analyzer
Copy link
Collaborator

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

@nagisa
Copy link
Member

nagisa commented Feb 15, 2022

@bors retry doesn't necessarily look related to the changes here unless we got some sort of a llvm pathology.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 15, 2022
@bors
Copy link
Contributor

bors commented Feb 15, 2022

⌛ Testing commit 8d6c973 with merge 09cb29c...

@bors
Copy link
Contributor

bors commented Feb 15, 2022

☀️ Test successful - checks-actions
Approved by: nagisa
Pushing 09cb29c to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Feb 15, 2022
@bors bors merged commit 09cb29c into rust-lang:master Feb 15, 2022
@rustbot rustbot added this to the 1.60.0 milestone Feb 15, 2022
@bors bors mentioned this pull request Feb 15, 2022
@abrown abrown deleted the cf-protection branch February 16, 2022 01:16
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (09cb29c): comparison url.

Summary: This benchmark run did not return any relevant results.

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.