When are static symbols guaranteed to show up in the final binary?

[RalfJung]

This came up in rust-lang/miri#450 but is really a t-opsem question, not a Miri question.

Obviously anything marked #[used] is guaranteed to show up (though it seems there are issues that make this not always true, but this seems like a rustc implementation issue to me, the spec is unambiguous).

But @joboet wants this to be guaranteed for more cases. I'll let them do the motivation and summary for that.

[joboet]

Thanks, Ralf!

Motivation

std's TLS support on Windows is very hacky. Unfortunately, these hacks are required in order to implement TLS destructors, which can only be registered through linker magic. One of the requirements of that linker magic is that we need to reference a special symbol (_tls_used) in the binary, otherwise the TLS destructor function (a pointer to which is placed in a special section) will not be called.

As far as I am aware, the only way to guarantee that a symbol is present without compiler magic is by performing a read_volatile with the address of the symbol. LLVM cannot optimize this out, thus ensuring that the symbol reference makes it to the linker.

Unfortunately, Rust's semantics do not guarantee "read_volatile => used", breaking the Windows destructor support.

std currently also relies on the assumption to skip the #[used] attribute on the static with function pointer in the special linker section (see rust-lang/rust#121596). This is "nice-to-have" as it allows removing the whole destructor infrastructure if it is unused, but not strictly necessary.

Potential solution

To solve this problem, I propose to make used-ness a dynamic property of symbols that is either provided a priori by #[used] or at runtime by volatile operations.

In the motivating example, the read_volatile of _tls_used in the TLS code would atomically mark this symbol as used, so when the abstract machine or other code then queried for that symbol, it would always get a positive result. When a symbol is not used, its presence is not guaranteed, so the answer would be non-deterministic.

[bjorn3]

In the motivating example, the read_volatile of _tls_used in the TLS code would atomically mark this symbol as used, so when the abstract machine or other code then queried for that symbol, it would always get a positive result.

If the act of executing the read_volatile marks the static as used, that is still incorrect. The tls callback is called both when spawning a thread and when destroying it, but the read_volatile would only get executed after spawning the first thread and thus after the first time the callback should have been called. While it may happen to work here because we only care about the thread exit callback for a destructor, it would not work at all for constructors.

[joboet]

Yes, indeed! But that seems fine to me. I really cannot come up with an opsem that would allow constructors to work with read_volatile. #[used] on imports seems like the only option there.

Still, I'd argue that dynamic used-ness should be used, not just because it'd fix std but also because it fits with what I'd expect from read_volatile.¹ In contrast to normal loads, whose result the AM could just guess, read_volatile is guaranteed to "touch bytes in memory", so any (foreign) allocation it touches cannot be elided, otherwise its side-effects could be missed. Therefore, the current situation is very weird: miri performs a read_volatile that touches an allocation, but at the same time says that that allocation doesn't exist.

Footnotes

1. IIuc, volatile hasn't really defined yet for Rust, so these are just my assumptions. It became abundantly clear to me during my discussion with Ralf that I do not know enough about opsem to be considered anything but a beginner in this area, so please take anything I say with a grain of salt. ↩

[chorman0773]

I'm not sure any definition of "final binary" is an opsem question. Insofar as it affects constructor execution, C++ (which does have static constructors as a language feature) couldn't do much about linkers and runtime loaders, and just made it "happens-before the first odr-use of anything in the same translation unit".

[CAD97]

Since the goal is manipulating the generated binary, I would think the "correct" way for std to ensure mention, if #[used] can't do the correct thing, would be with global_asm!. With that it should in theory be straightforward to just use the necessary assembler directives.

Also, I don't see why #[used] can't be used to ensure mention of an externally defined symbol; you should be able to put #[used] on an extern static.

---

I don't feel like "is a symbol mentioned" is a meaningful question to ask at the AM layer; it's a property of the concrete machine's object format after lowering from the AM. As a low level language we do care about guaranteeing some properties of that lowering, but it might be acceptable if those are a bit less formal than the abstract opsem.

To extend Connor's comment, in theory it should be the compiler's responsibility to ensure any magic symbol mentions needed to enable TLS happen, not std's, as running of TLS dtors is a language feature requiring cooperation with the host. (But in fairness std is privileged.)

[ChrisDenton]

it should be the compiler's responsibility to ensure any magic symbol mentions needed to enable TLS happen

This is indeed a point I've made before. But this is something that requires both compiler work and at least some domain knowledge whereas hacks in std are "easy" and they've "worked" for years so there's little motivating it.

[chorman0773]

Eh, I don't agree that it's fundamentally not stds problem, because the implementation of the standard library is part of the implementation as a whole. It still does need to cooperate with the compiler-proper though, so rustc isn't completely free of responsibility when something doesn't work (but neither is libstd).

[ChrisDenton]

Well in this case, llvm does in fact do the right thing. You can use #[thread_local] and it'll make sure _tls_used is included. The problem is when we don't use llvm for this (e.g. has_thread_local == false). The rustc compiler has no real knowledge of Windows TLS so it's put fully on std to resolve this.

TLS destructors are slightly different. In C/C++ these are provided by the runtime (e.g. msvc) in library files. These have object files compiled in such a way that including the function to register a destructor also includes the thing that runs destructors. But again, std can't really control which functions are "connected" to which other functions.

[RalfJung]

As far as I am aware, the only way to guarantee that a symbol is present without compiler magic is by performing a read_volatile with the address of the symbol. LLVM cannot optimize this out, thus ensuring that the symbol reference makes it to the linker.

AFAIK there is a way to do this, which us #[used]. Using read_volatile is just a nice-to-have optimization, isn't it? But it sounds like #[used] does not work for _tls_used so I am confused.

To extend Connor's comment, in theory it should be the compiler's responsibility to ensure any magic symbol mentions needed to enable TLS happen, not std's, as running of TLS dtors is a language feature requiring cooperation with the host.

For TLS dtors that may be the best solution. But what if someone needs some other special magic linker symbol in the future and wants it to be "used" in a way that can be optimized, with tricks similar to read_volatile?

I'm not sure any definition of "final binary" is an opsem question.

That's really the key point. opsem defines what happens in a single execution, and in all possible executions. The binary is an intermediate artifact that opsem has no bearing on (and in fact opsem applies to ways of running Rust code where there is no binary, such as via Miri).

So there's a fundamental mismatch here where in practice the symbol will show up if any execution of the program may do the volatile read -- so even if the read happens half-way through execution, the corresponding callback will have been invoked already at program startup, before there was any way to know whether the read will happen! All Miri can do is check whether the current execution has done the read in the past. So when we go look up a linker array, we could say that all statics that had volatile reads in the past are definitely included (though this is a really strange coupling of what should be unrelated concepts) -- but we can't do anything about symbols where the volatile read will happen in the future.

If this could be made a compiler primitive, that would indeed make things simpler. But then I worry that this will not be the last time someone wants to play tricks like this, and we can't add a new compiler primitive each time.

[bjorn3]

But it sounds like #[used] does not work for _tls_used so I am confused.

It probably needed #[used(linker)] rather than #[used].

[ChrisDenton]

If this could be made a compiler primitive, that would indeed make things simpler. But then I worry that this will not be the last time someone wants to play tricks like this, and we can't add a new compiler primitive each time.

A more general mechanism would be something like #[symbol_group(some_unique_name_or_path)]. Then everything within the same symbol_group gets linked as a unit.