Champion "blittable"/"unmanaged" constraint (15.7)

[gafter]

The blittable feature will give language enforcement to the class of types known as "unmanaged types" in the C# language spec. This is defined in section 18.2 as a type which is not a reference type and doesn't contain reference type fields at any level of nesting.

Allows where T : unmanaged

• Proposal added (https://github.com/dotnet/csharplang/blob/master/proposals/csharp-7.3/blittable.md)
• Discussed in LDM
• Decision in LDM
• Finalized (done, rejected, inactive)
• Spec'ed

[iam3yal]

Is this a proposal to improve existing blittable types or something? what's this about? :)

[vbcodec]

@eyalsk It is [Redacted] ::)

[ufcpp]

Is this related to dotnet/roslyn#2209 ?

[jaredpar]

Sorry for the confusion. Will reopen when we have the proposal ready.

[jaredpar]

Sorry there isn't more details here. This is being used as a reminder for me to add more details.

[jaredpar]

Proposal added here #206

[ufcpp]

This is good idea! but I have some questions

• How is this related to Blittable Types in Interop Marshaling? The same? or similar but bit different?
• Can IntPtr and UIntPtr be blittable? They are blittable in the Interop Marshaling and unmanaged in the current C# spec.

[iam3yal]

@jaredpar Just a thought.

Why can't it be an attribute?

[Blittable]
struct Point 
{
    public int X;
    public int Y;
}

And instead of adding a new blittable constraint such in the following case:

void Hash<T>(T value) where T : blittable struct
{
    using (T* p = &value) { 
        ...
    }
}

Allow attributes to be used as a generic constraint? so something like this.

void Hash<T>(T value) where T : struct, Blittable
{
    using (T* p = &value) { 
        ...
    }
}

This seems like a more general feature that can be used here and elsewhere in the framework. :)

Alternatively, use a marker interface?

[iam3yal]

@bbarry Why the downvote? what's the point of adding a new keyword? and then adding a special constraint when existing features can actually do the job or/and improved? not to mention that blittable struct is at odds with existing constraints, for example, why can't we check that the generic parameter is abstract class? what makes blittable any special?

Maybe I'm overlooking something so if you can, please do elaborate. :)

[KodrAus]

@eyalsk I think it's reasonable as a keyword, because adding it to a structure changes its compile-time behaviour (it means adding non-blittable fields will fail to compile). It seems like a nice fit to me.

[bbarry]

@eyalsk, it is nothing personal, my downvote is just that I'd rather see this be a keyword over an attribute. It is admittedly a fine line in the distinction (surely the compiled code would contain some attribute so that this can be enforced across assemblies, so the compiler must in fact know about that as it is enforced).

Adding a special case to permit the use of one particular attribute for this purpose seems arbitrary and counter to syntax consistency and rules that devs can internalize when writing code (I dislike all the attributes that do this already). Allowing general constraints in the form where T : Something where Something maps to an attribute type SomethingAttribute which must be defined on the type T is at first mildly interesting (I am not sure it has advantages over a marker interface, but I don't really like those), but is significantly a wider scope than this proposal and at best should be separate.

My gut says the former is yucky and the latter is going to introduce either ambiguities or breaking changes. A keyword isn't significantly better, but I can reasonably expect that a keyword means something to a compiler and will have an impact on how my code builds.

---

If there were the ability to have generalized attribute constraints, Blittable would still need to be handled in a special way by the compiler because it must enforce certain constraints on the struct type where it is used (for example disallowing non-blittable fields or auto struct layout).

[iam3yal]

@KodrAus, @bbarry Okay, fair enough, thank you both for the explanation. :)

[HaloFour]

The name blittable is a bit weird. To newcomers it doesn't remotely convey what it means. I'm not convinced that "blitting" is a known concept outside of a specific subset of Win32 developers who happened to know that "BitBlt" inspired a verb, and I'm one of them.

The name unmanaged is a little better, but it doesn't really grab me. I don't know that I have a better option. Maybe fixed, but that seems to imply more than this does as well.

Also, how is the generic constraint enforced? I don't see a mention of a CLR change to accompany this, so would this be encoded on the generic type/method as an attribute and enforced only by supporting compilers? What would happen if an errant compiler were to use one of these generic types/methods specifying a non-blittable struct generic type argument?

[ufcpp]

@HaloFour

What would happen if an errant compiler

This problem already exists in the current C# compiler. Using the Unsafe class, managed types can be converted to pointer types. This could cause GC crash.

using System;
using System.Runtime.CompilerServices;

struct UnmanagedStruct
{
    public int X;
    public int Y;
    public override string ToString() => (X, Y).ToString();
}

struct ManagedStruct
{
    public string X;
    public string Y;
    public override string ToString() => (X ?? "NULL", Y ?? "NULL").ToString();
}

unsafe class Program
{
    static void Main()
    {
        // OK
        // An unmanaged type can be used with pointers
        UnmanagedStruct us = new UnmanagedStruct { X = 1, Y = 2 };
        var usp = (int*)Unsafe.AsPointer(ref us);

        usp[0] = 100;
        usp[1] = 200;
        Console.WriteLine(us); // (100, 200)

        // OK!
        // This is an abuse of the Unsafe class.
        // The Unsafe class is implemented by IL and allows some problematic codes.
        // For instance, it can make managed type pointer.
        ManagedStruct ms = new ManagedStruct { X = "abc", Y = "cde" };
        var msp = (IntPtr*)Unsafe.AsPointer(ref ms);

        msp[0] = (IntPtr)0;
        msp[1] = (IntPtr)0;
        Console.WriteLine(ms); // (NULL, NULL)

        msp[0] = (IntPtr)1234567; // invalid pointer. GC could be crashed
    }
}

[ufcpp]

Related to https://github.com/dotnet/corefxlab/blob/e109353256875ee42e6365c5facd243b1f0497ce/src/System.Slices/System/SpanExtensions_binary.cs#L21

[HaloFour]

@ufcpp

Yes, but the existing code doesn't have any declared "constraints" intended to prevent that so the developer is definitely walking into unsafe territory there.

[jaredpar]

@ufcpp

How is this related to Blittable Types in Interop Marshaling? The same? or similar but bit different?

It's the same.

Can IntPtr and UIntPtr be blittable? They are blittable in the Interop Marshaling and unmanaged in the current C# spec.

Yes these are blittable. I missed that in the proposal. Thanks!

[jaredpar]

@eyalsk

Why can't it be an attribute?

In general the C# language avoids attaching semantics to attributes attached to types / methods. The preference is to use explicit language syntax to implement semantic changes in behavior.

[iam3yal]

@jaredpar Understood, thanks. :)

[jaredpar]

@HaloFour

The name blittable is a bit weird.
The name unmanaged is a little better, but it doesn't really grab me.

Yep ... naming is hard. 😄

One of the other names we considered in Midori was primitive.

[jnm2]

I like blittable. My estimate of developers would be more liberal, but it's easy enough to Google if you are unfamiliar with a term. The win is that blittable is the most directly precise term.

[HaloFour]

@jaredpar

Agreed, and it's a minor point at least at this stage of the proposal.

@jnm2

Even if that were the case, blit by itself doesn't mean anything. It happens to be the pseudo-pronouciation given to 1970s-era graphics compositing routines that Win32 happened to inherit but are largely obsolete today. We might as well call them sprite structs. I'd rather a name that better captures the intent of constraining the type to primitives or composites of primitives.

[jnm2]

I've often seen "blit" used without a graphics connotation, more of a "spit out these raw bytes fast without parsing" connotation which is even slightly contradictory to the original meaning.
I believe blit by itself has come to have a meaning of its own distinct from its origins (as all words do).

[orthoxerox]

Yeah, 'blittable' means 'can be safely treated as a single contiguous chunk of bits' to me.

[jnm2]

public static class Foo => public static struct Foo => wheee!

[MarkPflug]

Some structs that contain references can still be blittable (strings, once pinned, and assuming they are passed as readonly are blittable, for example).

Is that not true for any pinned reference, a pinned byte[] array? Or are strings truly special in this regard? Wouldn't this just be a effectively a void* or a IntPtr then? Still sounds "value-ish" to me.

[jnm2]

Wouldn't this just be a effectively a void* or a IntPtr then? Still sounds "value-ish" to me.

To me, value-ish, no. Blit-ish, yes. For what it's worth. 😄

I think blit has more apropos meaning in people's minds (“this blob can be used without transformation”) compared to value (“this structure has value semantics rather than identity semantics”). or even val which some propose to mean readonly var.

[tannergooding]

@alzr, I'm not convinced static delegate is right either. I had initially proposed value delegate, but some discussion ended up changing that (I don't recall what it was at the time).

Code that you aren't familiar with is going to be searched and static struct will likely cause ambiguity and confusion. I think static struct will cause large amounts of confusion, especially when compared to static class.

[tannergooding]

@MarkPflug, not all pinned references are blittable. Strings are sometimes blittable, single dimensional 0-based arrays of blittable types are themselves blittable:
https://docs.microsoft.com/en-us/dotnet/framework/interop/blittable-and-non-blittable-types
https://github.com/dotnet/corefx/blob/04b84e0206bbb3d126fdc3329b6417971d3bc895/Documentation/coding-guidelines/interop-pinvokes.md
https://github.com/dotnet/corefx/blob/42ce0e696e08e190aceaf8613e47e67d57178550/Documentation/coding-guidelines/interop-guidelines.md

[nietras]

Should the title of this not be renamed to lose the "blittable" naming now that the wording has, thank god, become unmanaged? 😄

[OmarTawfik]

Assigning back to champion since implementation is done.

[xoofx]

Taking a pointer to a generic with unmanaged constraint (or indirectly on a struct that is implicitly unmanaged transitively as for issue #1504) is the very primary usage of the unmanaged constraint. I can only consider the unmanaged constraint done if T* and fixed on T are allowed at C# level. This is possible at the IL level, this should be unlocked at C# level so that we can get rid off once and for good of the IL tricks we have been using for years to workaround this problem. It should allow to remove a dependency to System.Runtime.CompilerServices.Unsafe.
Is there any thing blocking this at the language spec and furthermore at the roslyn level?

[xoofx]

Oops, nevermind, retrying this with latest and it seems that pointers/fixed are supported. The only remaining problem is #1504 which can be very annoying but less critical

[theraot]

Did blittable lose just because naming? I mean, blittable and unmanaged are not the same thing. E.g. char is unmanaged but not blittable.

Blittable gives me the guarantee that I can take a pointer to it, read it, write it to a stream and recover it somewhere else without losing the meaning. I do not think that is true of char, I should be using a character encoding.

What do we earn with unmanaged? If I have a generic type that has the constraint of being unmanaged... what are the things that I can safetly do with it thanks to that? Edit: Ah, pointers.

Addendum: Can blittable exist as a subset of unmanaged?

[yaakov-h]

Blittable gives me the guarantee that I can take a pointer to it, read it, write it to a stream and recover it somewhere else without losing the meaning.

Assuming same CPU bitness and endian-ness?

I do not think that is true of char, I should be using a character encoding.

System.Char is UTF-16, IIRC.

[PathogenDavid]

System.Char is UTF-16, IIRC.

Yup, from the spec:

Character and string processing in C# uses Unicode encoding. The char type represents a UTF-16 code unit, and the string type represents a sequence of UTF-16 code units.

---

Did blittable lose just because naming?

Yes. unmanaged was chosen to match F#, CLR, and existing C# spec terminology.

I do not think that is true of char, I should be using a character encoding.

Character encoding is a concern of marshaling. If you aren't using marshaling (which you probably aren't when you're using the unmanaged constraint), you're dealing with UTF-16.

Also char is satisfies the unmanaged constraint, and is absolutely blittable.

Addendum: Can blittable exist as a subset of unmanaged?

You need to provide examples where unamanaged isn't satisfying your current needs.

[theraot]

@PathogenDavid I'm not thinking of interop with unmanaged APIs, but of writing a struct to a stream (disk or network).

Also char is satisfies the unmanaged constraint, and is absolutely blittable.

I must be remembering wrong. Was there not a problem of endianess? At Blittable and Non-Blittable Types char is listed not blittable. Similarly bool is listed not blittable, while it is unmanaged.

You need to provide examples where unamanaged isn't satisfying your current needs.

Let us say that char is blittable as you claim...

Pointers are unamanaged, are they not problem? If I want to write a generic method that writes a blittable struct to disk and have it recovered by a different proccess at a later time, I should not be accepting pointers.

As far as I can tell, that means that I have to check that the type is supported (well, at least that I did not get a pointer) at runtime via reflection.

[PathogenDavid]

I'm not thinking of interop with unmanaged APIs, but of writing a struct to a stream (disk or network).

Sure, the issues are largely the same. Nothing about "marshaling" necessarily implies interop. You can use marshaling for writing data to the disk or network too.

At Blittable and Non-Blittable Types char is listed not blittable

This documentation is specific to the interop marshaler, which does not consider bool or char to be blittable because the default marshaling behavior of those types does not match their in-memory representation in CLR.

Was there not a problem of endianess?

If you're worried about endianess, you can't ever handle most structs in the ways you're describing. Any primitive type larger than a byte is going to be hosed. I'd strongly suggest evaluating whether you actually care about endianess. Unless you have specific needs, it's very unlikely you actually do.

I suggest you review what endianess actually is if you think it'd only apply to char.

Pointers are unamanaged, are they not problem?

It depends on what that pointer (or more specifically, the native-sized integer) represents. You can absolutely have pointers which will match between runs of the application or between multiple machines. (A few examples: VirtualAlloc at a specific location, memory-mapped registers on an embedded system, the "pointer" doesn't actually represent a pointer to memory, memory allocated before a call to fork().)

This is an issue with your specific needs. Blindly writing out a struct that contains an integer field that represents an index into a separate array that wasn't written out is just as problematic.

As far as I can tell, that means that I have to check that the type is supported (well, at least that I did not get a pointer) at runtime via reflection.

At the end of the day, you can only do so much to protect the developer from themselves. In my opinion, passing process-specific information to some library to send over the network falls under "play stupid games, win stupid prizes".

If you're working in a domain where pointer fields in structs are common and you want to avoid them getting passed to APIs that can't handle them properly, I'd suggest writing a Roslyn analyzer so you can detect this issue at compile-time.