Add new tier 3 target: x86_64-unknown-none

[mikeleany]

Adds support for compiling OS kernels or other bare-metal applications for the x86-64 architecture.

Below are details on how this target meets the requirements for tier 3:

A tier 3 target must have a designated developer or developers (the "target maintainers") on record to be CCed when issues arise regarding the target. (The mechanism to track and CC such developers may evolve over time.)

I would be willing to be a target maintainer, though I would appreciate if others volunteered to help with that as well.

Targets must use naming consistent with any existing targets; for instance, a target for the same CPU or OS as an existing Rust target should use the same name for that CPU or OS. Targets should normally use the same names and naming conventions as used elsewhere in the broader ecosystem beyond Rust (such as in other toolchains), unless they have a very good reason to diverge. Changing the name of a target can be highly disruptive, especially once the target reaches a higher tier, so getting the name right is important even for a tier 3 target.

Uses the same naming as the LLVM target, and the same convention as many other bare-metal targets.

Target names should not introduce undue confusion or ambiguity unless absolutely necessary to maintain ecosystem compatibility. For example, if the name of the target makes people extremely likely to form incorrect beliefs about what it targets, the name should be changed or augmented to disambiguate it.

I don't believe there is any ambiguity here.

Tier 3 targets may have unusual requirements to build or use, but must not create legal issues or impose onerous legal terms for the Rust project or for Rust developers or users.

I don't see any legal issues here.

The target must not introduce license incompatibilities.
Anything added to the Rust repository must be under the standard Rust license (MIT OR Apache-2.0).
The target must not cause the Rust tools or libraries built for any other host (even when supporting cross-compilation to the target) to depend on any new dependency less permissive than the Rust licensing policy. This applies whether the dependency is a Rust crate that would require adding new license exceptions (as specified by the tidy tool in the rust-lang/rust repository), or whether the dependency is a native library or binary. In other words, the introduction of the target must not cause a user installing or running a version of Rust or the Rust tools to be subject to any new license requirements.
If the target supports building host tools (such as rustc or cargo), those host tools must not depend on proprietary (non-FOSS) libraries, other than ordinary runtime libraries supplied by the platform and commonly used by other binaries built for the target. For instance, rustc built for the target may depend on a common proprietary C runtime library or console output library, but must not depend on a proprietary code generation library or code optimization library. Rust's license permits such combinations, but the Rust project has no interest in maintaining such combinations within the scope of Rust itself, even at tier 3.
Targets should not require proprietary (non-FOSS) components to link a functional binary or library.
"onerous" here is an intentionally subjective term. At a minimum, "onerous" legal/licensing terms include but are not limited to: non-disclosure requirements, non-compete requirements, contributor license agreements (CLAs) or equivalent, "non-commercial"/"research-only"/etc terms, requirements conditional on the employer or employment of any particular Rust developers, revocable terms, any requirements that create liability for the Rust project or its developers or users, or any requirements that adversely affect the livelihood or prospects of the Rust project or its developers or users.

I see no issues with any of the above.

Neither this policy nor any decisions made regarding targets shall create any binding agreement or estoppel by any party. If any member of an approving Rust team serves as one of the maintainers of a target, or has any legal or employment requirement (explicit or implicit) that might affect their decisions regarding a target, they must recuse themselves from any approval decisions regarding the target's tier status, though they may otherwise participate in discussions.
This requirement does not prevent part or all of this policy from being cited in an explicit contract or work agreement (e.g. to implement or maintain support for a target). This requirement exists to ensure that a developer or team responsible for reviewing and approving a target does not face any legal threats or obligations that would prevent them from freely exercising their judgment in such approval, even if such judgment involves subjective matters or goes beyond the letter of these requirements.

Only relevant to those making approval decisions.

Tier 3 targets should attempt to implement as much of the standard libraries as possible and appropriate (core for most targets, alloc for targets that can support dynamic memory allocation, std for targets with an operating system or equivalent layer of system-provided functionality), but may leave some code unimplemented (either unavailable or stubbed out as appropriate), whether because the target makes it impossible to implement or challenging to implement. The authors of pull requests are not obligated to avoid calling any portions of the standard library on the basis of a tier 3 target not implementing those portions.

core and alloc can be used. std cannot be used as this is a bare-metal target.

The target must provide documentation for the Rust community explaining how to build for the target, using cross-compilation if possible. If the target supports running tests (even if they do not pass), the documentation must explain how to run tests for the target, using emulation if possible or dedicated hardware if necessary.

Use --target=x86_64-unknown-none-elf option to cross compile, just like any target. The target does not support running tests.

Tier 3 targets must not impose burden on the authors of pull requests, or other developers in the community, to maintain the target. In particular, do not post comments (automated or manual) on a PR that derail or suggest a block on the PR based on a tier 3 target. Do not send automated messages or notifications (via any medium, including via @) to a PR author or others involved with a PR regarding a tier 3 target, unless they have opted into such messages.
Backlinks such as those generated by the issue/PR tracker when linking to an issue or PR are not considered a violation of this policy, within reason. However, such messages (even on a separate repository) must not generate notifications to anyone involved with a PR who has not requested such notifications.

I don't foresee this being a problem.

Patches adding or updating tier 3 targets must not break any existing tier 2 or tier 1 target, and must not knowingly break another tier 3 target without approval of either the compiler team or the maintainers of the other tier 3 target.
In particular, this may come up when working on closely related targets, such as variations of the same architecture with different features. Avoid introducing unconditional uses of features that another variation of the target may not have; use conditional compilation or runtime detection, as appropriate, to let each target run code supported by that target.

No other targets should be affected by the pull request.

[rust-highfive]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @cjgillot (or someone else) soon.

Please see the contribution instructions for more information.

[rust-highfive]

⚠️ Warning ⚠️

• These commits modify compiler targets. (See the Target Tier Policy.)

[joshtriplett]

Thank you for working on this. I have a proposal in progress myself for a similar target, and there are a couple of changes I'd propose; I also have target documentation for the target. Let me get that together, please, and I'll post PR review within a day or two.

[joshtriplett]

(One such change: I believe this should use CodeModel::Kernel.)

[mikeleany]

Thank you for working on this. I have a proposal in progress myself for a similar target, and there are a couple of changes I'd propose; I also have target documentation for the target. Let me get that together, please, and I'll post PR review within a day or two.

Thanks I appreciate the help.

(One such change: I believe this should use CodeModel::Kernel.)

Yes, I agree. I've noticed that some JSON files out there have that and others don't. I had intended to include it here, but somehow missed it. That is corrected now.

[joshtriplett]

I sent a PR to this PR, at mikeleany#1 with additions and modifications to this proposed target. This also addresses all of the outstanding feedback (including my own), and adds target-specific documentation.

I also submitted the target tier request as a compiler MCP at rust-lang/compiler-team#462 . That proposal adds a second target maintainer, @haraldh.

[mikeleany]

I sent a PR to this PR, at mikeleany#1 with additions and modifications to this proposed target. This also addresses all of the outstanding feedback (including my own), and adds target-specific documentation.

I also submitted the target tier request as a compiler MCP at rust-lang/compiler-team#462 . That proposal adds a second target maintainer, @haraldh.

Thanks for this and everyone else's feedback.

I've removed -elf from the PR title to reflect that change. I had debated whether that should be included or not. I can see the point that it may imply a level of library support.

[joshtriplett]

The current failure looks like a bug in the linkchecker with mailto links. We can work around it, but I think the linkchecker may need improvement here.

[joshtriplett]

On Mon, Sep 20, 2021 at 12:21:00AM -0700, Harald Hoyer wrote: @haraldh commented on this pull request. > + max_atomic_width: Some(64), + // don't use probe-stack=inline-asm until rust#83139 and rust#84667 are resolved + stack_probes: StackProbeType::Call, + position_independent_executables: true, + static_position_independent_executables: true, + relro_level: RelroLevel::Full, + relocation_model: RelocModel::Static, + linker_flavor: LinkerFlavor::Lld(LldFlavor::Ld), + linker: Some("rust-lld".to_owned()), + features: + "-mmx,-sse,-sse2,-sse3,-ssse3,-sse4.1,-sse4.2,-3dnow,-3dnowa,-avx,-avx2,+soft-float" + .to_string(), + executables: true, + disable_redzone: true, + panic_strategy: PanicStrategy::Abort, + code_model: Some(CodeModel::Kernel), Do we really want `CodeModel::Kernel` ??

For freestanding/bare-metal usage, I think we do, yes. Code running in ring 0 most often wants to be using this, especially if it supports ring 3 but often even if it doesn't. A project can always change the code model, but I think this is the right default for this target.

[jackpot51]

If anyone needs another reference for this, here is the Redox kernel's default target spec. It would be nice to be able to use a built-in one.

{
    "llvm-target": "x86_64-unknown-none",
    "target-endian": "little",
    "target-pointer-width": "64",
    "target-c-int-width": "32",
    "data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128",
    "arch": "x86_64",
    "os": "none",
    "env": "",
    "vendor": "unknown",
    "linker-flavor": "gcc",
    "target-family": "redox",
    "pre-link-args": {
        "gcc": ["-m64", "-nostdlib", "-static"]
    },
    "features": "-mmx,-sse,-sse2,-sse3,-ssse3,-sse4.1,-sse4.2,-3dnow,-3dnowa,-avx,-avx2,+soft-float",
    "dynamic-linking": false,
    "executables": false,
    "relocation-model": "pic",
    "code-model": "kernel",
    "disable-redzone": true,
    "eliminate-frame-pointer": false,
    "exe-suffix": "",
    "has-rpath": false,
    "no-compiler-rt": true,
    "no-default-libraries": true,
    "position-independent-executables": false,
    "has-elf-tls": true
}

[jackpot51]

I defined this target spec above as x86_64-unknown-kernel. Perhaps there is value in having x86_64-unknown-none use the default code model, but have another target like x86_64-unknown-kernel which uses the kernel code model?

[npmccallum]

@cjgillot @joshtriplett What is the status of this? The MCP was accepted. It looks like we're just blocked on merging this PR. Once this is merged we can do smaller iterative changes to coalesce the other related targets. Also, we (Enarx) are blocked waiting on this.

Can someone please help unblock this?

[joshtriplett]

Current status: the compiler team MCP has been fully approved, and all the comments on this PR have been addressed, so someone needs to do the final r+ on it. I'll track down someone to do so.

[cjgillot]

The MCP has been accepted, and the feedback appears to have been addressed. This is Tier 3, and there does not appear to be an impact on other targets.
r? @cjgillot
@bors r+

[bors]

📌 Commit 11d54dc has been approved by cjgillot

[bors]

⌛ Testing commit 11d54dc with merge ff0e148...

[bors]

☀️ Test successful - checks-actions
Approved by: cjgillot
Pushing ff0e148 to master...

[rust-highfive]

📣 Toolstate changed by #89062!

Tested on commit ff0e148.
Direct link to PR: #89062

💔 rls on linux: test-pass → test-fail (cc @Xanewok).

[rust-timer]

Finished benchmarking commit (ff0e148): comparison url.

Summary: This benchmark run did not return any relevant changes.

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

[haraldh]

@npmccallum I removed TLS support in x86_64-unknown-none-hermitkernel (see #89941). It isn't necessary.

@stlankes , so, I tried rusty_demo with:

❯ git diff
diff --git a/hermit-sys/build.rs b/hermit-sys/build.rs
index 391fae2..3be0e47 100644
--- a/hermit-sys/build.rs
+++ b/hermit-sys/build.rs
@@ -35,7 +35,7 @@ fn build_hermit(src_dir: &Path, target_dir_opt: Option<&Path>) {
                        .arg("-Z")
                        .arg("build-std=core,alloc")
                        .arg("--target")
-                       .arg("x86_64-unknown-none-hermitkernel");
+                       .arg("x86_64-unknown-none");
        } else if target_arch == "aarch64" {
                cmd.current_dir(src_dir)
                        .arg("build")
@@ -132,7 +132,7 @@ fn build_hermit(src_dir: &Path, target_dir_opt: Option<&Path>) {
 
        let lib_location = if target_arch == "x86_64" {
                target_dir
-                       .join("x86_64-unknown-none-hermitkernel")
+                       .join("x86_64-unknown-none")
                        .join(&profile)
                        .canonicalize()
                        .unwrap() // Must exist after building

and it seems to work. Any blockers on using x86_64-unknown-none?

[stlankes]

@haraldh I will check it, but I don't see any blocker. But a double check will be great. @mkroening Do you see any issue?

[stlankes]

@haraldh I thought a little bit about your request. The main issue is that our llvm target is also defined in the spec file (see https://github.com/rust-lang/rust/blob/master/compiler/rustc_target/src/spec/x86_64_unknown_none_hermitkernel.rs#L14). This is used to define our kernel as standalone kernel (https://github.com/llvm/llvm-project/blob/e356027016c6365b3d8924f54c33e2c63d931492/llvm/include/llvm/MC/MCELFObjectWriter.h#L76-L77). In our case, we develop a library operating systems and everything is linked to one single boot image. Maybe it is currently working, but if I understand everything correctly, every part of the image should use the same OSABI. What do you think?

[stlankes]

By the way, my Linux defines the OSABI (see /usr/include/elf.h) as following:

#define EI_OSABI 7 /* OS ABI identification /
#define ELFOSABI_NONE 0 / UNIX System V ABI /
#define ELFOSABI_SYSV 0 / Alias. /
#define ELFOSABI_HPUX 1 / HP-UX /
#define ELFOSABI_NETBSD 2 / NetBSD. /
#define ELFOSABI_GNU 3 / Object uses GNU ELF extensions. /
#define ELFOSABI_LINUX ELFOSABI_GNU / Compatibility alias. /
#define ELFOSABI_SOLARIS 6 / Sun Solaris. /
#define ELFOSABI_AIX 7 / IBM AIX. /
#define ELFOSABI_IRIX 8 / SGI Irix. /
#define ELFOSABI_FREEBSD 9 / FreeBSD. /
#define ELFOSABI_TRU64 10 / Compaq TRU64 UNIX. /
#define ELFOSABI_MODESTO 11 / Novell Modesto. /
#define ELFOSABI_OPENBSD 12 / OpenBSD. /
#define ELFOSABI_ARM_AEABI 64 / ARM EABI /
#define ELFOSABI_ARM 97 / ARM /
#define ELFOSABI_STANDALONE 255 / Standalone (embedded) application */

[haraldh]

Moved this discussion here: hermit-os/hermit-rs#197