Move appropriate parts of Microsoft.DotNet.PlatformAbstractions.RuntimeEnvironment into the shared framework

[tannergooding]

Rationale

It is often the case, when working with .NET Core that you want to do something based on the current platform/architecture. In many cases, this can include needing to interop with tools published using a particular "runtime identifier".

Today, the RuntimeIdentifier is only exposed through the Microsoft.DotNet.PlatformAbstractions.RuntimeEnvironment.GetRuntimeIdentifier() API and is not exposed as part of CoreFX. This requires referencing an additional package, and prevents it from being used in certain scenarios (such as evaluation time in MSBuild).

As such, I propose that we expose a mechanism to get the RID for the current execution environment at runtime.

Proposed API

#nullable enabled
namespace System.Runtime.InteropServices
{
    public static class RuntimeInformation
    {
        // The current OS RID, e.g.: win7-x64, osx.10.11-x64, ubuntu.18.04-arm64, rhel.7-x64
        public static string RuntimeIdentifier { get; }
    }
}

The above API will return the value of a new AppContext variable RUNTIME_IDENTIFIER. This variable will be passed by the dotnet.exe host, and can be passed by any other native host when the runtime is loaded.

DISCUSSION: Should we still maintain a managed fallback code path for when this AppContext variable isn't present? For example if the app was loaded from a different native host that didn't set this variable.

Additional Notes

This would allow switching on the active RID from an MSBuild project/props/targets file, which would allow greater versatility in consuming native tools from a package during build time (or from a user program at runtime).

It may be worth reviewing the other APIs exposed via Microsoft.DotNet.PlatformAbstractions.RuntimeEnvironment and determining which others, if any, should also be exposed: https://github.com/dotnet/core-setup/blob/master/src/managed/Microsoft.DotNet.PlatformAbstractions/RuntimeEnvironment.cs

Updates

• eerhardt: Add proposal for OperatingSystem and OperatingSystemVersion so Microsoft.DotNet.PlatformAbstractions.RuntimeEnvironment can be completely removed/deprecated/etc.

• vihofer: Choosing OS over OperatingSystem as we already use OS in IsOSPlatform. OperatingSystemPlatform is already accessible through IsOSPlatform(Platform). RuntimeArchitecture is already exposed as ProcessArchitecture. The RuntimeVersions proposal is already covered by dotnet/coreclr#22664 and doesn't require an api review.

• eerhardt: Remove proposal for OSName and OSVersion, we will drop support for these APIs. Callers can instead call Ntdll.RtlGetVersionEx on Windows, or read /etc/os-release on Unix.
  Change RuntimeIdentifier from a property to a method call, since the proposed behavior will invoke a method: AppContext.GetData(), which isn't appropriate for a property.

[tannergooding]

FYI. @eerhardt

[tannergooding]

@nguerrera, @KathleenDollard, @livarcocc, @dsplaisted. I know there has been a similar request on the SDK/CLI team before

[weshaggard]

We already have other related issues like https://github.com/dotnet/corefx/issues/28620.

[dsplaisted]

Is just getting the current RID helpful? Or do you usually also need to find a best match from a list of RIDs you have resources available for, using the RID graph?

[tannergooding]

Is just getting the current RID helpful? Or do you usually also need to find a best match from a list of RIDs you have resources available for, using the RID graph?

Yes, getting the best match from a list of RIDs is even better. Not sure how closely tied that is to NuGet, however.

[tannergooding]

We already have other related issues like #25681.

@weshaggard, right. But I didn't see anything that looked like the "recommended" API proposal layout and that covered the RuntimeIdentifier.

[eerhardt]

It may be worth reviewing the other APIs exposed via Microsoft.DotNet.PlatformAbstractions.RuntimeEnvironment and determining which others, if any, should also be exposed: https://github.com/dotnet/core-setup/blob/master/src/managed/Microsoft.DotNet.PlatformAbstractions/RuntimeEnvironment.cs

See my comments here: https://github.com/dotnet/corefx/issues/28620#issuecomment-377421773, copied below for ease in reading:

@eerhardt what is your thoughts on bring Microsoft.DotNet.PlatformAbstractions into corefx?

I would absolutely LOVE for this to happen.

However, I don't think it should go wholesale into corefx. I think pieces of it should be removed altogether (I'm looking at you HashCodeCombiner and Platform).

And some things should be folded into System.Runtime.InteropServices.RuntimeInformation, like RuntimeEnvironment.

ApplicationEnvironment can probably be deleted as well now that we have netstandard2.0. I'm not exactly sure what it's main purpose was - mostly for net45 and netstandard1.3 code to be able to call 1 method to get the BaseDirectory without having to #if themselves. With netstandard2.0, users can now call AppContext.BaseDirectory, or AppDomain.CurrentDomain.BaseDirectory for this.

So thinking about it more - probably move RuntimeEnvironment into RuntimeInformation, and delete the rest. Obviously the library will still live in servicing branches, if we ever need to service the already released versions of it. And users can continue to use the already released versions all they want.

---

The other public members of RuntimeEnvironment:

        public static Platform OperatingSystemPlatform { get; } = PlatformApis.GetOSPlatform();

        public static string OperatingSystemVersion { get; } = PlatformApis.GetOSVersion();

        public static string OperatingSystem { get; } = PlatformApis.GetOSName();

        public static string RuntimeArchitecture { get; } = GetArch();

The following properties aren't already exposed in corefx:

• OperatingSystem should be moved to RuntimeInformation.
• OperatingSystemVersion should be moved to RuntimeInformation.

The following properties are dupes of corefx APIs:

• OperatingSystemPlatform can be removed since we already have OSPlatform.
• RuntimeArchitecture can be removed since we already have RuntimeInformation.ProcessArchitecture.

[eerhardt]

One question I have about the RID API:

Today, we have an environment variable that allows someone to set the RID:

private static readonly string OverrideEnvironmentVariableName = "DOTNET_RUNTIME_ID";

Do we plan on preserving this behavior with the proposed API? And if we do, would it make more sense for the API to be a method instead of a property? Since it would always need to check the env var.

[tannergooding]

Do we plan on preserving this behavior with the proposed API? And if we do, would it make more sense for the API to be a method instead of a property? Since it would always need to check the env var.

I think a method makes sense, if the behavior is preserved. Otherwise, a property seems the better choice.

[pjanotti]

In general I don't like the idea of applications taking functionality dependency based on reported OS version at runtime: there is a history of broken features because the OS can fake its answers, or later starts to support some feature and the app then requires recompilation. If possible testing/trying the feature itself is the preferred way. That said decisions based on broad categories (e.g.: Windows vs Unix) or more fine grained for tests, not production, are reasonable. Of course using RID info is totally fine in the context of building/packaging/etc.

Do we really need this as a public API? Can we instead bring it down to some place that provides the gains for building/packaging without exposing a public API?

[eerhardt]

I don't think it is a great idea for code to take functionality dependency based on reported OS version at runtime either. But the fact of the matter is, it happens. The RID has the OS version inside of it. So anything you do with the RID will now mean you are dependent on the reported OS version.

We've been down the "do we need the OS Version public API?" path before, and it wasn't added as a "corefx" API previously. And that decision is what led to the creation of the PlatformAbstractions library in the first place. So whether we like it or not, we already have this public API. The issue is the API lives outside of corefx, and outside the "typical" place where people look for this type of information - the RuntimeInformation class. If all the existing callers of RuntimeEnvironment.OperatingSystemVersion can be convinced that they actually don't need to call it, then I think it can be removed.

Here are other places where people are asking for OS version:

https://github.com/dotnet/corefx/issues/16629
https://github.com/dotnet/corefx/issues/29395
https://github.com/dotnet/corefx/issues/8099
https://github.com/dotnet/corefx/issues/4741
https://github.com/dotnet/corefx/issues/12737
https://github.com/dotnet/corefx/issues/9011
https://github.com/dotnet/corefx/issues/1017
https://github.com/dotnet/cli/issues/2754

(note this isn't an exhaustive list, I stopped looking as I think the point is made)

The main usage of OperatingSystemVersion is logging/reporting types of scenarios. Here are the 2 places in the dotnet/cli that call RuntimeEnvironment.OperatingSystemVersion:

• When handling dotnet --info
• To send to telemetry information

I know that we already have Environment.OSVersion. However, Environment.OSVersion is not implemented the same way as RuntimeEnvironment.OperatingSystemVersion. RuntimeEnvironment.OperatingSystemVersion will give you the version part of the RID, which comes from /etc/os-release and then also has some special logic to "normalize" the version on alpine and RHEL. Whereas Environment.OSVersion gets the version from calling uname and returning the .release portion directly.

Lastly, I think these methods/properties (GetRuntimeIdentifier(), OperatingSystem, OperatingSystemVersion) need to live together since they need to be consistent. If the RID says it is rhel.7-x64, then the OperatingSystemVersion needs to return 7. And getting the RID from corefx's RuntimeInformation, but needing to go to PlatformAbstraction's RuntimeEnvironment to get the OperatingSystemVersion doesn't make any sense. If we are going to move the API to get the current machine's RID into RuntimeInformation, then PlatformAbstraction's RuntimeEnvironment should be removed.

p.s.
On the topic of using reported versions in code, I think this comment from @terrajobst is relevant

Working around implementation level issues. Sometimes you need to work around specific behavior that is displayed by specific platform versions.

Version APIs are problematic. Believe me, we know very well how version checks are fragile. We've tried several times to remove version checks from our API surface. Be it in .NET Core or in Windows. But the reality is that some problems are just too hard to do reliably -- regardless of the approach. Giving developers the ability to perform version checks as the ultimate escape hatch is a sensible compromise. There is a reason why we don't want to make this API super discoverable and also turn the logic inside out, i.e. let the developer pass in the value they care about so we can do the version checking for them.

[pjanotti]

Thanks for digging the history and showing that this was considered many times before 😄 you convinced me.

[eerhardt]

I've added to the proposal to include OperatingSystem, and OperatingSystemVersion strings. However, thinking more about this, would it make sense to create a new type that encapsulated the RID APIs? ex.

public struct RuntimeIdentifier
{
    public string Id { get; }
    public string OperatingSystem { get; }
    public string OperatingSystemVersion  { get; }
}

or potentially just a completely new static class that contained RID specific information:

public static class RuntimeIdentifier
{
    public static string GetCurrent();
    public static string OperatingSystem { get; }
    public static string OperatingSystemVersion { get; }
}

This would convey that the OperatingSystem and OperatingSystemVersion strings are directly tied to the RID's concept of what OperatingSystem and OperatingSystemVersion are. And it would be understandable why they are different than say Environment.OSVersion, or RuntimeInformation.IsOSPlatform (ex. RuntimeInformation.IsOSPlatform(OSPlatform.Create("ubuntu")) returns false even when running on ubuntu.)

[weshaggard]

I think it might make some since to have a type for RuntimeIdentifier if we plan to support other scenarios like checking compatibility with other RIDs, if we only care about getting the current RID then I'd stick with a static class.

I do think we need to be a little careful about making assumptions about the structure of a RID. While it is true it generally contains an OS and version it isn't always the case as a RID is just an string. It also commonly contains the architecture so if you are breaking it apart you may want to add that value as well.

ex. RuntimeInformation.IsOSPlatform(OSPlatform.Create("ubuntu")) returns false even when running on ubuntu

We should consider actually fixing that if we can someone embed the RID graph we could use that information to check these different hierarchies.

[simplexidev]

This is something that would be useful, especially when writing bindings for other native libraries. Has there been any update on this?

[ViktorHofer]

Updated the api proposal and changed the label to api-ready-for-review.

[am11]

OperatingSystemPlatform is already accessible through IsOSPlatform(Platform)

Is there any API that returns the OSPlatform object representing current platform?

For example an API that could expose the value of s_osPlatformName from https://github.com/dotnet/corefx/blob/a10890f/src/System.Runtime.InteropServices.RuntimeInformation/src/System/Runtime/InteropServices/RuntimeInformation/RuntimeInformation.Unix.cs

[jkotas]

public static string RuntimeIdentifier { get; }

What do we expect this API to be used for? Is this API useful without the library that allows you to reason about the RID graph?

The RIDs come with a lot of problems because of they form a graph. You cannot really do anything useful with the RID without having the library that is able to deal with the RID graph.

[jkotas]

Is there any API that returns the OSPlatform object representing current platform?

@ViktorHofer Is that basically the proposed OSName API, or is OSName something else from OSPlatform? Could you please update the proposal with representative set of examples of values that each of the APIs return to make it clear what they do?

[danmoseley]

The move of Microsoft.DotNet.PlatformAbstractions out of dotnet/core-setup, which partly motivates this API, is tracked in https://github.com/dotnet/core-setup/issues/5213

[jkotas]

The GetRuntimeIdentifier() in Microsoft.DotNet.PlatformAbstractions does not return what the calling code wants some of time, and it has a hack that lets you override to returned value via Environment variable to componsate for that. If we add this API to CoreFX, do we expect to keep this hack? (It comes down to what we expect this API to be actually used for... .)

[eerhardt]

Is that basically the proposed OSName API, or is OSName something else from OSPlatform?

No, that is not the proposed OSName.

The s_osPlatformName is something like Linux, OSX, FreeBSD.

The proposed OSName is something like ubuntu, rhel, centos, Mac OS X, FreeBSD. On Linux systems, this is the ID field in /etc/os-release (or /etc/readhat-release). Typically, it is the first part in the RuntimeIdentifier string for Linux systems.

public static string RuntimeIdentifier { get; }

What do we expect this API to be used for? Is this API useful without the library that allows you to reason about the RID graph?

We have it today in Microsoft.DotNet.PlatformAbstractions, and it is useful. There is a Rationale at the top of this issue. There are also numerous linked issues to this issue, here is one of them:

Any managed library (such as msbuild task) that loads native library needs to know RID. This need came up recently in libgit2sharp. We should add GetRuntimeId method.

Here is ASP.NET's build system calling it.

The CLI uses it to print it out during --info, so the user knows what .NET thinks is the current RID of the machine.

Today, you can reason about the RID graph by doing something similar to what the CLI does here.

We can add more "RID-graph reasoning" APIs in the future, if necessary. But starting with "what is the current machine's RID?" seems like a good starting point.

[eerhardt]

The GetRuntimeIdentifier() in Microsoft.DotNet.PlatformAbstractions does not return what the calling code wants some of time

It has to return exactly what the ".NET runtime" thinks is the RID of the machine. This API and the dotnet host must always be in sync, or else there will be problems.

it has a hack that lets you override to returned value via Environment variable to componsate for that

That "hack" has proven useful very often. Just recently, it was proven useful because we didn't have alpine 3.8 in the RID graph.

[jkotas]

Here is ASP.NET's build system calling it.

This is old code in a private repo. I do not see it in live ASP.NET Core. Also, it is a great example that shows why the RID alone is useless. It has ad-hoc list of cases to compensate for when the RID is not what you want.

Should the API to give you best guess about the current RID be rather part of the RID graph reasoning library?

[eerhardt]

Sorry, that was a bad link to the ASP.NET build system. Here is the real link: https://github.com/aspnet/BuildTools/blob/b7b88d08d55abc8b71de9abf16e26fc713e332cd/src/ApiCheck.Console/NuGet/RuntimeGraph.cs#L93

Here is dotnet/arcade calling it: https://github.com/dotnet/arcade/blob/2d52dfb5fca30c0e64454ce01a1e9c81d7e75989/src/Microsoft.DotNet.Build.Tasks.Configuration/src/GenerateConfigurationProps.cs#L137

Should the API to give you best guess about the current RID be rather part of the RID graph reasoning library?

No, I believe it needs to be sync'd with what the runtime thinks is the current RID. Again, if these are different values (what the runtime thinks, and what this API produces) we are going to have problems.

[jkotas]

sync'd with what the runtime thinks is the current RID
If the RID says it is rhel.7-x64, then the OperatingSystemVersion needs to return 7

Ok, so these APIs are really just tunnel to the host.

• The implementation of all these 3 APIs is just going to be a thin wrapper over the host methods, like https://github.com/dotnet/core-setup/blob/dbb1c1e55eab169f4af370e9a61b85a982ff1105/src/corehost/cli/deps_format.cpp#L139 .

• Should the returned RID be filtered against the RID graph that the host happens to have, or should it be the raw first guess?

@eerhardt ?

cc @vitek-karas @jeffschwMSFT

[eerhardt]

Ok, so these APIs are really just tunnel to the host.

On .NET Core, yes. We would have to decide what to do on other places - throw PNSE, fallback to some C# implementation, etc. Today PlatformAbstractions has the logic in C# and is netstandard1.3....

Should the returned RID be filtered against the RID graph that the host happens to have, or should it be the raw first guess?

For use cases like the CLI showing it during --info, I think it needs to be the raw value. That is what is getting returned in PlatformAbstractions today.

[jkotas]

We would have to decide what to do on other places

System.Runtime.InteropServices.RuntimeInformation is not a package anymore. This API would be .NET Core 3.0+ only (and maybe uap/uapaot - but that one is really just a flavor of .NET Core).

[eerhardt]

This API would be .NET Core 3.0+ only

How about Mono? I thought they used the libraries in corefx. Or any other netstandard2.1 platform?

Also note the breaking change here if we go through with https://github.com/dotnet/core-setup/issues/5213. Previously you could call this API on netstandard1.3. But that will be up to the core-setup issue to resolve. It shouldn't stop corefx from adding the API.

[jkotas]

How about Mono? I thought they used the libraries in corefx.

Once Mono chooses to implement .NET Core App 3.0 API set, they will implement this API too.

Or any other netstandard2.1 platform?

They will not get this API.

[jkotas]

The fallback really means "don't know". If you pass any to other APIs that expect actual meaningful RID, they are likely going to crash anyway.

[eerhardt]

The issue in the API design meeting regarding this API being defined as:

public static string? RuntimeIdentifier { get; }

Is that it is an exceptional case when the AppContext variable isn't set. When an app is launched by our app host, the RUNTIME_IDENTIFIER AppContext variable will be set. If an app is using the new preferred native Hosting APIs, it will be set. The only time RUNTIME_IDENTIFIER won't be set is if someone is using the CoreClrHost.h or mscoree.h native hosting, which shouldn't be a lot of places. Forcing callers of this method to have to handle null when it is an exceptional situation, and there is a possible non-null fallback, wasn't desired.

If you pass any to other APIs that expect actual meaningful RID, they are likely going to crash anyway.

I would think that depends on the API. Since any is a valid RID, I don't see why it would crash. If it can't find any in its graph, it would treat it the same as any invalid RID it couldn't find.

However, the other API is probably defined as

#nullable enbale
void MyRidApi(string rid)

And we couldn't say MyRidApi(RuntimeInformation.RuntimeIdentifier), without the compiler complaining for everyone.

---

The current RuntimeEnvironment API returns unknown-{arch} when it can't figure it out. Perhaps keeping that same behavior is the best approach for this new API.

[jkotas]

The only time RUNTIME_IDENTIFIER won't be set is if someone is using the CoreClrHost.h or mscoree.h native hosting

That's not the only case. The broader .NET ecosystem includes e.g. Unity that has its own runtime and number of Unity target platforms do not even have well-defined RIDs by design.

I am fine with this being implemented as `GetData("RUNTIME_IDENTIFIER") ?? "any"`` (or "unknown" if that works better).

[terrajobst]

I think the primary conclusion is that we'd consider such hosts as having a bug. Frankly, I'd even go so far and say we should crash on startup when a RID isn't specified. Or said differently, for this API to be reliable, the host must set it meaningfully. That's why we should force the change there, rather than asking everyone to guard for null.

[am11]

We like that we don't duplicate code between the native and the managed implementation

If by native we also consider the control flow stemming from

runtime/src/installer/corehost/cli/hostmisc/pal.h

Line 67 in 3d8073d

#if defined(_WIN32)

as a problem, then +1. For operating systems, that are not officially supported, such as BSD lane or even other SystemV likes such as Illumous, these kind of hardcoded constraint unnecessary puts another barrier on already complicated system for porting work, which is suffering for a long time due to:

• over-engineered RID barriers everywhere: starting from post- CoreCLR native compilation to the package delivery via NuGet to build publish dotnet apps.
• heavy (mandatory?) dependency on CI: so if AzDevOps (since it has replaced its more capable counterpart Jenkins in 2019) does not support the operating system (i.e. anything beyond Windows, macOS, Linux), we have no dice to get dotnet(1) running and will have our fingers crossed forever since dotnet/source-build also relies on AzDevOps and is locked down to those three systems.

By the time we get any version working on other OS, it is already too late in lifetime of that version, due to breaking changes in infra for the next version.

IMO, keep RID story super simple; restrictions wise and otherwise so it can scale better in what it was originally designed to do. This is mitigate some pain points in porting work for vast range of targets.

[jkotas]

@terrajobst As @am11 said, the RIDs are ever-engineered, do not scale and major barrier for entry for non-Microsoft driven ports. It is why I believe that it is important that this API is allowed to return "don't know answer". If we dislike null, it is ok for this don't know answer to be "any", "unknown" or even empty string.

[terrajobst]

@terrajobst As @am11 said, the RIDs are ever-engineered, do not scale and major barrier for entry for non-Microsoft driven ports. It is why I believe that it is important that this API is allowed to return "don't know answer". If we dislike null, it is ok for this don't know answer to be "any", "unknown" or even empty string.

The host doesn't need to validate the string against the graph; it just needs to return a string that is unique and can be used to identity the runtime. However, for libraries with native dependencies it's important that "sensible" answers are returned. As you said, returning "any" is unlikely to be a good answer.

[jkotas]

However, for libraries with native dependencies it's important that "sensible" answers are returned

I would like to see programs and libraries that try to do their own native library resolution based on RIDs to assume that the native library is supplied by other means for unknown RID. Nobody really needs to their own native library resolution in .NET Core, this is very niche case - often just a left over from .NET Framework. I know that there are a few libraries that do that for "reasons", and I would not feel bad that the potential null is in their face.

I would like to avoid the situation we have with Assembly.Location today where every other call to this API does not handle the case of empty path correctly that results in a lot of breaks with single-file publishing.

[tmds]

@eerhardt the value returned for RuntimeIdentifier depends on the distributions known to .NET Core?
For example, if I have installed Microsoft linux-x64 .NET Core build on Ubuntu 18.04, this returns ubuntu.18.04-x64?
If I install that same version on Ubuntu 20.04, this will return linux-x64 (because ubuntu.20.04-x64 is not a known rid)?

From a use-case perspective, it may be interesting to be able to query whether the platform is compatible with an rid. Both above are compatible with linux-x64.

[jkotas]

This API is not doing any filtering against any known lists. It is going to return ubuntu.20.04-x64 in your example.

[tmds]

@jkotas Ah, that's good! The processing is limited to some normalization of /etc/os-release/ (like stripping the minor for rhel7).

[tmds]

fwiw, since this returns an rid for the OS, OSIdentifier would be a more indicative name.

[eerhardt]

From a use-case perspective, it may be interesting to be able to query whether the platform is compatible with an rid.

Today, you can use the framework's .deps.json file, or the app's .deps.json file if it is a self-contained app (the RID graph was added for self-contained apps with dotnet/sdk#3361), to get the RID graph. .deps.json files can be loaded with the DependencyModel library. See the RuntimeGraph property on DependencyContext. You can use that to see what RID is compatible with what for the current platform.

If you need a full RID graph (ex. win information on unix), we don't have an API to get this information. You would have to ship and update the runtime.json file yourself.

The processing is limited to some normalization of /etc/os-release/ (like stripping the minor for rhel7).

Yes, it still does that normalization for rhel and alpine.

since this returns an rid for the OS, OSIdentifier would be a more indicative name.

I believe that would be confusing since everywhere else in the system (CLI options, SDK properties, Documentation, etc) calls this RID or RuntimeIdentifier.