Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[llvm][rtsan] Add transform pass for sanitize_realtime_unsafe #109543

Merged
merged 10 commits into from
Oct 3, 2024
68 changes: 48 additions & 20 deletions llvm/lib/Transforms/Instrumentation/RealtimeSanitizer.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -15,49 +15,77 @@

#include "llvm/IR/Analysis.h"
#include "llvm/IR/IRBuilder.h"
#include "llvm/IR/InstIterator.h"
#include "llvm/IR/Module.h"

#include "llvm/Demangle/Demangle.h"
#include "llvm/Transforms/Instrumentation/RealtimeSanitizer.h"

using namespace llvm;

static SmallVector<Type *> getArgTypes(ArrayRef<Value *> FunctionArgs) {
SmallVector<Type *> Types;
for (Value *Arg : FunctionArgs)
cjappl marked this conversation as resolved.
Show resolved Hide resolved
Types.push_back(Arg->getType());
return Types;
}

static void insertCallBeforeInstruction(Function &Fn, Instruction &Instruction,
const char *FunctionName) {
const char *FunctionName,
ArrayRef<Value *> FunctionArgs) {
LLVMContext &Context = Fn.getContext();
FunctionType *FuncType = FunctionType::get(Type::getVoidTy(Context), false);
FunctionType *FuncType = FunctionType::get(Type::getVoidTy(Context),
getArgTypes(FunctionArgs), false);
FunctionCallee Func =
Fn.getParent()->getOrInsertFunction(FunctionName, FuncType);
IRBuilder<> Builder{&Instruction};
Builder.CreateCall(Func, {});
Builder.CreateCall(Func, FunctionArgs);
}

static void insertCallAtFunctionEntryPoint(Function &Fn,
const char *InsertFnName) {

insertCallBeforeInstruction(Fn, Fn.front().front(), InsertFnName);
const char *InsertFnName,
ArrayRef<Value *> FunctionArgs) {
insertCallBeforeInstruction(Fn, Fn.front().front(), InsertFnName,
FunctionArgs);
}

static void insertCallAtAllFunctionExitPoints(Function &Fn,
const char *InsertFnName) {
for (auto &BB : Fn)
for (auto &I : BB)
if (isa<ReturnInst>(&I))
insertCallBeforeInstruction(Fn, I, InsertFnName);
const char *InsertFnName,
ArrayRef<Value *> FunctionArgs) {
for (auto &I : instructions(Fn))
if (isa<ReturnInst>(&I))
insertCallBeforeInstruction(Fn, I, InsertFnName, FunctionArgs);
}

static PreservedAnalyses rtsanPreservedCFGAnalyses() {
PreservedAnalyses PA;
PA.preserveSet<CFGAnalyses>();
return PA;
}
cjappl marked this conversation as resolved.
Show resolved Hide resolved

static PreservedAnalyses runSanitizeRealtime(Function &Fn) {
insertCallAtFunctionEntryPoint(Fn, "__rtsan_realtime_enter", {});
insertCallAtAllFunctionExitPoints(Fn, "__rtsan_realtime_exit", {});
return rtsanPreservedCFGAnalyses();
}

static PreservedAnalyses runSanitizeRealtimeUnsafe(Function &Fn) {
IRBuilder<> Builder(&Fn.front().front());
Value *Name = Builder.CreateGlobalString(demangle(Fn.getName()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly out of interest, why do we de-mangle here and not in the reporting?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a really great question. In all honesty I didn't originally think about the possibility of de-mangling during the report stage. After thinking about it further, I have an idea on which one will probably be less work overall, but I'd need advice on whether the distribution of work is appropriate.

Here's how I see the work involved:

  • for demangling in the pass:
    • no run-time work: demangles once at compile time, but
    • could increase compile-time work (marginally?)
  • for demangling in the reporting:
    • adds to run-time work: demangles every time an error is registered (which could be many times if halt_on_error is off)
    • won't add to compile-time work

My very naive intuition is that demangling in the pass is preferable, but very happy to move it to the reporting if I have my priorities a bit upside down. Thanks in advance for any suggestions here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you are missing the downside of binary size, because demangled name takes up more space, for demangling in the pass.

I am not sure if demangling will cause any noticable overhead over unwinding and symbolizing the stack frame.

Taking a step back, why do we pass this name and not use the PC and symbolize that?

Copy link
Contributor Author

@davidtrevelyan davidtrevelyan Sep 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this question and prompting us to look into this in more detail. I had indeed not considered the issue of binary size, so went away and thought about it a bit harder with @cjappl. There's lots to think about here, especially in terms of how we expect the sanitizer to be used.

We anticipate that the sanitizer users will only be marking a handful of functions they are worried about with the [[clang::blocking]] attribute, which is what ultimately results in sanitize_realtime_unsafe being added in IR. We think this because any functions that do the normal bad real-time stuff like allocation will be picked up by the libc interceptors anyway. Adding the [[clang::blocking]] attribute is more for user functions that could block entirely within user space, like a spin mutex lock or similar busy-wait scenario. Conservatively estimating that a user has 10 concerning [[blocking]] functions, each with demangled symbol length 25 characters - I think that would be (at least ideally) of the order 250 bytes of extra binary size to store. The rtsan runtime library is (when stripped) currently about 750 kB already, so we think this is about a 0.03% size increment - and only for sanitized builds.

We tried the PC approach in a branch and confirmed it works, but we were sad to lose the function name for printing diagnostics in release builds, and also to have to do the extra run-time work to unwind the stack. Being able to run the sanitizer with release builds is one of our design goals - we'd like to achieve as-near-normal performance as possible, so that users can meaningfully QA their real-time systems with RealtimeSanitizer enabled.

Given:

  • the extra work at run time necessary to unwind the stack,
  • the simpler run-time implementation when accepting a function name string,
  • the loss of being able to print the blocking function name in release builds, and
  • the almost-negligible expected binary increment,

... our moderate preference is currently to keep the approach how we have it outlined in this PR. However, this is of course caveated with the appreciation that we're very new here and inexperienced with the codebase, so I'm very keen to take as much advice as possible on this. Many thanks for your help so far - keen to hear your thoughts!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We tried the PC approach in a branch and confirmed it works, but we were sad to lose the function name for printing diagnostics in release builds

Confirming that this also meant that the whole stack trace symbolization was gone, but that was okay.

do the extra run-time work to unwind the stack.

I don't understand. To symbolize a PC, we don't need to unwind the stack. From the runtime CLs I saw go buy, it seems like you unwind the stack on error regardless though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirming that this also meant that the whole stack trace symbolization was gone, but that was okay.

Yes - thanks - that was indeed my intended meaning.

I don't understand. To symbolize a PC, we don't need to unwind the stack. From the runtime CLs I saw go buy, it seems like you unwind the stack on error regardless though

That's correct, my mistake. We can discount the runtime cost of unwinding the stack and just concentrate on the functionality/binary size trade-off. After some more thought, we also realised that the symbolised PC approach would make suppression of these errors difficult to implement, because we would no longer have a function name to match against a suppression list. For this reason, we're leaning more strongly towards the existing implmentation and taking the small hit on binary size. I noticed that you approved this PR last night, so I guess you're happy enough with it as it is - please let me know if you have any further concerns. Thanks again for the review 👍

insertCallAtFunctionEntryPoint(Fn, "__rtsan_notify_blocking_call", {Name});
return rtsanPreservedCFGAnalyses();
}

RealtimeSanitizerPass::RealtimeSanitizerPass(
const RealtimeSanitizerOptions &Options) {}

PreservedAnalyses RealtimeSanitizerPass::run(Function &F,
PreservedAnalyses RealtimeSanitizerPass::run(Function &Fn,
AnalysisManager<Function> &AM) {
if (F.hasFnAttribute(Attribute::SanitizeRealtime)) {
insertCallAtFunctionEntryPoint(F, "__rtsan_realtime_enter");
insertCallAtAllFunctionExitPoints(F, "__rtsan_realtime_exit");

PreservedAnalyses PA;
PA.preserveSet<CFGAnalyses>();
return PA;
}
if (Fn.hasFnAttribute(Attribute::SanitizeRealtime))
return runSanitizeRealtime(Fn);

if (Fn.hasFnAttribute(Attribute::SanitizeRealtimeUnsafe))
return runSanitizeRealtimeUnsafe(Fn);

return PreservedAnalyses::all();
}
31 changes: 31 additions & 0 deletions llvm/test/Instrumentation/RealtimeSanitizer/rtsan_unsafe.ll
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --check-globals all --version 5
; RUN: opt < %s -passes=rtsan -S | FileCheck %s
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use update_test_checks.py, unless there's some reason this is not possible here.

Copy link
Contributor Author

@davidtrevelyan davidtrevelyan Sep 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, great tool. Thanks for showing me this script - I've updated this test in 0775459


; RealtimeSanitizer pass should create the demangled function name as a global string and,
; at the function entrypoint, pass it as an argument to the rtsan notify method

;.
; CHECK: @[[GLOB0:[0-9]+]] = private unnamed_addr constant [20 x i8] c"blocking_function()\00", align 1
;.
define void @_Z17blocking_functionv() #0 {
; CHECK-LABEL: define void @_Z17blocking_functionv(
; CHECK-SAME: ) #[[ATTR0:[0-9]+]] {
; CHECK-NEXT: call void @__rtsan_notify_blocking_call(ptr @[[GLOB0]])
; CHECK-NEXT: ret void
;
ret void
}

define noundef i32 @main() #2 {
; CHECK-LABEL: define noundef i32 @main() {
; CHECK-NEXT: call void @_Z17blocking_functionv()
; CHECK-NEXT: ret i32 0
;
call void @_Z17blocking_functionv() #4
ret i32 0
}

attributes #0 = { mustprogress noinline sanitize_realtime_unsafe optnone ssp uwtable(sync) }
cjappl marked this conversation as resolved.
Show resolved Hide resolved
;.
; CHECK: attributes #[[ATTR0]] = { mustprogress noinline optnone sanitize_realtime_unsafe ssp uwtable(sync) }
;.