Broken custom IDataObject implementations

[filipnavara]

• .NET Core Version: 5.0

• Have you experienced this same bug with .NET Framework?: No

Problem description:

PR #3388 removed the ComVisible attribute from System.Windows.Forms.IDataObject. That largely breaks the marshaling custom implementation of IDataObject through clipboard within the same application. The concern about marshaling of individual calls is not applicable in that case because the runtime could resolve the IDataObject to the original .NET object implementation.

It likely broke this case in Clipboard.GetDataObject and now all returned objects are wrapped in the default DataObject which doesn't support all the custom types:

winforms/src/System.Windows.Forms/src/System/Windows/Forms/Clipboard.cs

Lines 170 to 173 in 2fd2a58

	if (dataObject is IDataObject ido && !Marshal.IsComObject(dataObject))
	{
	return ido;
	}

Expected behavior:

Minimal repro:

TBD

[weltkante]

Doesn't in-process usage of COM query for a private interface to determine its a managed object and let .NET take over from there? In other words you shouldn't be operating on a COM wrapper in the first place if its a managed object, so ComVisible shouldn't matter? I'll try to create a repro and debug to determine whats going on.

ComVisible is problematic because support for TLB generation has been removed, so even adding ComVisible back won't work for cross thread clipboard usage within the same process (fails marshaling). Instead of making it ComVisible WinForms could [ComImport(...)] the IDataObject from Desktop Framework to rely on its TLB, or manually generate a TLB (via MIDL) and embed it, then ComImport it. (but I'm not yet convinced that is necessary, as explained above)

[filipnavara]

It's really tricky. We reuse the same data object implementation for drag&drop and clipboard. In the case of drag&drop the object identities somehow kick in. That same behavior does not happen when going through clipboard and using the OleGetClipboard API. In that case you always get __ComObject back. Maybe the .NET runtime actually does marshalling in this case and my original description is not entirely correct.

We never do cross-thread or cross-application calls with this. I don't even think it's reasonably valid to access clipboard from non-UI thread.

Our use-case is that we expose an operation akin to "copy email into clipboard". For the operating system (COM IDataObject) we expose the formats to interpret is as file, text, etc. However for copying within the application and pasting into different mailbox we store few additional properties that are exposed only through System.Windows.Forms.IDataObject methods. These are now lost due to the additional wrapping.

[weltkante]

That same behavior does not happen when going through clipboard and using the OleGetClipboard API. In that case you always get __ComObject back.

I get this behavior on Desktop Framework as well, i.e. Clipboard.GetDataObject returns a wrapped COM object, not the original WinForms DataObject. I assume OLE does not pass the DataObject on directly but provides a wrapper for isolation.

If you had this working on Desktop Framework maybe something else is going wrong, if you can provide a repro or more details I'm happy to take a look, but from a quick test I could not find a difference between Desktop and Core.

[filipnavara]

The difference is started happening between .NET Core 3.1 and .NET 5 with the removal of the ComVisible attribute. Both .NET Framework and .NET Core 3.1 behaved identically. I'll try to come up with some minimal sample.

[filipnavara]

This demonstrates the issue we originally hit:
WinFormsApp1.zip

The app multi-targets .NET Core 3.1 and .NET 5. In 3.1 it will pass fine through all the asserts, in 5.0 it will crash. However, it relies on directly calling the OleGetClipboard API instead of Clipboard.GetDataObject to reproduce the issue. I'd have to dig deeper to see why Clipboard.GetDataObject doesn't work as expected. We knew it didn't work on .NET Framework which is why we use the OleGetClipboard API directly in our app but the code in this repository made me believe that it was something that was already fixed.

[weltkante]

Thanks, I need some time to look closer at this, there are multiple issues happening, especially since the last assert never worked on Desktop Framework in the first place. Restoring ComVisible is definitely not the right call (beyond TLB issues), it may have looked to you like "it just worked" but that probably was just accidental alignment of multiple bugs. It will break if you start using the interface slightly differently than you are doing (for example note the Type arguments in the interface, code paths inside WinForms which use them will cause issues).

If you want a quick workaround you could define your own COM visible interface on your data object (in addition to the interfaces you already implement) and cast to that, since you are already doing custom interop anyways. This way you keep relying on the "accidental alignment of bugs" but can keep working like you did before.

I'll get back to this once I have a more complete analysis of what happens, and hopefully a suggestion how to improve the situation. (I'd like to get at least Clipboard.GetDataObject working with custom IDataObject implementations, assuming thats possible and not by design or an implementation mistake in your custom data object.)

[filipnavara]

for example note the Type arguments in the interface, code paths inside WinForms which use them will cause issues

I intentionally left that out in the example to make it more concise. We do implement these methods as well although I doubt we actually call them directly anywhere. I assume there could be some gotchas with these methods.

If you want a quick workaround you could define your own COM visible interface on your data object

That's exactly what we did as a workaround. We needed to do minimal change to avoid breaking the dependent code:

	[ComVisible(true)]
	public interface IComVisibleDataObject
	{
		object GetData(string format, bool autoConvert);
		object GetData(string format);
		object GetData(Type format);
		bool GetDataPresent(string format, bool autoConvert);
		bool GetDataPresent(string format);
		bool GetDataPresent(Type format);
		string[] GetFormats(bool autoConvert);
		string[] GetFormats();
		void SetData(string format, bool autoConvert, object data);
		void SetData(string format, object data);
		void SetData(Type format, object data);
		void SetData(object data);
	}

	class ComVisibleDataObjectWrapper : System.Windows.Forms.IDataObject
	{
		IComVisibleDataObject innerDataObject;

		public ComVisibleDataObjectWrapper(IComVisibleDataObject innerDataObject)
		{
			this.innerDataObject = innerDataObject;
		}

		public object GetData(string format, bool autoConvert) => innerDataObject.GetData(format, autoConvert);
		public object GetData(string format) => innerDataObject.GetData(format);
		public object GetData(Type format) => innerDataObject.GetData(format);
		public bool GetDataPresent(string format, bool autoConvert) => innerDataObject.GetDataPresent(format, autoConvert);
		public bool GetDataPresent(string format) => innerDataObject.GetDataPresent(format);
		public bool GetDataPresent(Type format) => innerDataObject.GetDataPresent(format);
		public string[] GetFormats(bool autoConvert) => innerDataObject.GetFormats(autoConvert);
		public string[] GetFormats() => innerDataObject.GetFormats();
		public void SetData(string format, bool autoConvert, object data) => innerDataObject.SetData(format, autoConvert, data);
		public void SetData(string format, object data) => innerDataObject.SetData(format, data);
		public void SetData(Type format, object data) => innerDataObject.SetData(format, data);
		public void SetData(object data) => innerDataObject.SetData(data);
	}

	/// <summary>
	/// Class implementing drag/drop and clipboard support for virtual files.
	/// Also offers an alternate interface to the IDataObject interface.
	/// </summary>
	public class BaseDataObject :
		IComVisibleDataObject,
		System.Windows.Forms.IDataObject,
		System.Runtime.InteropServices.ComTypes.IDataObject,
		IAsyncOperation
	{
                ...

		public static System.Windows.Forms.IDataObject GetClipboardDataObject()
		{
			try
			{
				System.Runtime.InteropServices.ComTypes.IDataObject dataObject = null;
				Win32.OleGetClipboard(ref dataObject);
				if (dataObject is IComVisibleDataObject comVisibleDataObject /*&& !Marshal.IsComObject(dataObject)*/)
					return new ComVisibleDataObjectWrapper(comVisibleDataObject);
				return Clipboard.GetDataObject();
			}
			catch
			{
				return Clipboard.GetDataObject();
			}
		}
        }

I'll get back to this once I have a more complete analysis of what happens, and hopefully a suggestion how to improve the situation. (I'd like to get at least Clipboard.GetDataObject working with custom IDataObject implementations, assuming thats possible and not by design or an implementation mistake in your custom data object.)

Thanks! We know we depend on underspecified part of the code but this is something that dates back to the .NET Framework days. Part of the custom data object implementation likely even comes from some CodeProject article or StackOverflow answer so it's likely to affect other people than us. I don't expect it to be very common construct though.

The underlying reason for even going to this great length is that we need to expose virtual files where content is generated during the paste operation and not earlier. The standard WinForms DataObject does not have that capability.

[weltkante]

The sample you provided has a faulty implementation of both the managed IDataObject and the native IDataObject - both are fixable and when done Clipboard.GetDataObject (your third assert) works. In other words, WinForms can and will consume a properly implemented custom data object.

details about the bugs in the sample

• you implement both native and WinForms IDataObject, the native IDataObject takes precedence in the provided sample. If you remove the native interface and only implement the managed IDataObject you can verify that WinForms correctly falls back to using the managed IDataObject implementation on your custom data object, regardless of it not being ComVisible (it will pipe it through a system-provided native IDataObject implementation, which is the same behavior DesktopFramework is providing, since OLE does not roundtrip the data object for clipboard and puts a wrapper object in between)

• in the native IDataObject your EnumFormatEtc implementation does not enumerate Foo as supported format, so GetDataPresent("Foo") will say its not supported. Fixing this fixes the third assert, if you implement the native interface you want to implement it correctly, or not implement it at all. If you need an example how the FORMATETC has to look here is what WinForms would produce, but notice that you still need to implement the other interface members correctly to actually retrieve the data.

if (direction == DATADIR.DATADIR_GET)
{
    var formatList = new List<FORMATETC>
    {
        new FORMATETC
        {
            cfFormat = (short)DataFormats.GetFormat("Foo").Id,
            dwAspect = DVASPECT.DVASPECT_CONTENT,
            lindex = -1,
            ptd = IntPtr.Zero,
            tymed = TYMED.TYMED_HGLOBAL,
        }
    };

    if (SHCreateStdEnumFmtEtc((uint)formatList.Count, formatList.ToArray(), out var enumerator) == S_OK)
        return enumerator;
}

• in the managed IDataObject implementation of the sample the code for GetFormats returns "Foo " with space, thus WinForms will return false for GetDataPresent("Foo"). Fixing this (and removing the native IDataObject implementation to make WinForms use the managed one) will properly report data is present.

Of course I can't tell if that solves your actual problem since it just fixes the sample, there may be other issues preventing you from using WinForms to consume your custom data object. However the initially quoted snippet is not faulty, the second condition will prevent direct usage of managed IDataObject so WinForms itself will never run into a problem from it not being ComVisible.

As far as the second assert is concerned, you are actively going out of your way to not use any WinForms code besides the declaration of the interface. If the entire implementation is already coding around WinForms instead of using it, I believe its ok to expect you to also use a custom interface for that external communication. In other words the "workaround" we identified above is most likely also the correct implementation for what you are doing. (Note that the interface you copy/pasted is probably broken even if it currently works for you, see below, and in general the approach you are taking is probably not the best solution, there are better ways to establish a channel through a data object by putting the communication interface into the data object instead of implementing it on the data object.)

So if there is no bug with custom IDataObject implementations that leaves us with the breaking change of removing ComVisible on the managed IDataObject. This was an intentional and important change to make sure Desktop Framework and .NET Core can coexist on the same machine without getting in each others way.

Besides issues with type libraries discussed in the PR/issue for the change, also note that having Type as arguments in the interface method signatures means the interface can never be compatible to Desktop Framework

• Desktop Framework and .NET Core will not be able to communicate due to using different type systems
• The clipboard (and drag'n'drop) are a global resource, so they will inevitably try to communicate
• .NET Core does not even have Type marked as ComVisible, so the interface as it is defined will not be a proper ComVisible interface (you need to replace Type arguments with IUnknown-marshaled objects).

The last point means source compatibility and runtime compatibility at the same time are not achievable, so I believe keeping managed IDataObject not ComVisible is the "best" solution. It allows Desktop Framework applications to keep using the interface globally as they used to, and .NET Core applications can use the interface in-process to feed custom data objects. Anyone already going out of their way to work around the common usage patterns probably has to adapt their code.

---

PS: I tried to keep this response as short as possible, so if you want me to go into details or want to update the sample, you are welcome to keep the discussion going, the above is just my current opinion on the matter

[filipnavara]

Firstly, thanks for the detailed write up!

you implement both native and WinForms IDataObject, the native IDataObject takes precedence in the provided sample

That's intentional even if it trips the WinForms logic. In the actual code we implement logic for CFSTR_FILEDESCRIPTORW and delayed serving of data from Stream objects. That's something that cannot be accomplished by System.Windows.Forms.IDataObject and implementing both IDataObject interfaces accurately reflects what we are trying to do.

in the native IDataObject your EnumFormatEtc implementation does not enumerate Foo as supported format, so GetDataPresent("Foo") will say its not supported.

It's something that I tried to explain earlier. We have a set of formats that we want to work for COM and between processes (text, RTF, HTML, file descriptors). That's what is implemented in the COM IDataObject which, for brevity, I left empty in the reduced sample. Then we have a second set of properties that are intended only for in-process clipboard, ie. copy & paste within the same instance of the same application. Content of these properties is meaningless for other application instances since it contains stuff like database object IDs. This allows us to optimize certain scenarios instead of actually transferring megabytes or gigabytes of data through COM interfaces with marshaling.

I am not necessarily saying that's a good way to accomplish the goal of having in-process formats and out-of-process formats (a subset of the in-process ones). It's simply how it was implemented in our application for ±10 years and it broke with the removal of the ComVisible attribute which was unexpected. I am very much open to writing it in a different way as long as it still supports the requirement of CFSTR_FILEDESCRIPTORW support.

in the managed IDataObject implementation of the sample the code for GetFormats returns "Foo " with space

Probably a late typo in the editor just before I posted it. Sorry.

As for the rest, I understand the reasoning behind the removal of the ComVisible attribute. However, most or all the compatibility problems you mentioned (between desktop NetFX and NetCore) are not relevant to how we used the data objects. We only ever used the System.Windows.Forms.IDataObject for clipboard operations within the same process.

I am not sure whether there is anything actionable for this issue at the moment. Maybe a documentation mentioning the change about the ComVisible attribute but since you can now Google this issue it's probably moot. Perhaps I should file a feature request to improve the clipboard APIs to add support for the "file descriptors" which is the only reason why we use this convoluted code in the first place.

[weltkante]

I definitely understand your intentions of why you were doing it, and if you want a low effort solution for maintaining the previous behavior then the custom interface is definitely ok. Just saying that for an actively developed application its possible to implement this in a better way.

Then we have a second set of properties that are intended only for in-process clipboard

We use "private" data formats for this, which I believe is the intended way to pass this kind of information. Using a GUID or a sufficiently custom data format name does the trick, as long as you don't need to pass sensitive data. Also makes it easier if you want to allow limited interaction of multiple instances of your application.

On Desktop Framework things were easily done by using the remoting infrastructure (inherit from MarshalByRefObject or make your objects [Serializable] as appropriate) but the former is already gone in .NET Core and the latter is going to be removed over the next releases. There is still COM as "out of the box" communication mechanism but its a little more work to set up. (Though if you implement the data object yourself anyways you could use its support for placing COM objects in data slots, another thing WinForms didn't expose to .NET, saving you from implementing the lifetime management you have to do when using the default DataObject).

I'll see if I can find some time to write some examples how we are doing things and what alternatives exist, maybe it gives you some ideas for alternate implementations around your problem.

Maybe a documentation mentioning the change about the ComVisible attribute

That actually was recently added: dotnet/docs#22423

Perhaps I should file a feature request to improve the clipboard APIs to add support for the "file descriptors" which is the only reason why we use this convoluted code in the first place.

I'd definitely welcome a discussion about this. Maybe together with updating the clipboard formats, the .NET Core team driving the removal of BinaryFormatter means another major breaking change is incoming in one of the next releases.

[RussKie]

/cc: @AaronRobinsonMSFT

[RussKie]

FWIW, we're working on making the clipboard API more resilient to address issues like this.
I have a very rough draft (thanks to @AaronRobinsonMSFT) on my fork, and @JeremyKuhne is tinkering in this area as well.

[weltkante]

@filipnavara I want to follow up on your original issue, as promised

First here's what we are doing: just store a MarshalByRefObject (Desktop Framework only) or a serialized COM reference into a data object slot with a custom format. This could even be a back-reference to the custom data object itself if you want it to.

example for putting a COM reference into a data object

Some notes about the code below:

• Stream is the only object that can be transferred safely between frameworks for custom formats, as all other object types are serialized via BinaryFormatter (even strings) which is not available in 3rd party frameworks that may want to talk to the clipboard and even is going away in future .NET Core. Though if you only care about in-process usage that may not matter to you and you can just stick in the moniker as a string.
• any stream you put in is going to come out as a MemoryStream since its transferred through a HGLOBAL memory block, so I'm always working with MemoryStream to simplify the code
• if you want additional protection you can add a private GUID as prefix to the moniker and strip it on reading after using it to verify its actually in your format (but IMHO using a sufficiently unique custom format string should be enough protection against accidental slot colissions)
• to add even more protection you could use a hash or encryption to protect/validate the moniker but at this point it becomes silly, just mentioning it for completeness, if the user (or a program run by the user) wants to break you they have easier ways to do so
• you can restrict effective visibility to your own process by using a non-marshalable interface (i.e. something just defined in your assembly via ComImport, using a private GUID and without TLB registered)
• alternatively you can just use below code to put an arbitrary .NET object into the data object, if the receiving side is in the same process it will unpack as a .NET object, but if you see a COM reference then its in a different process
  • of course if you treat arbitrary objects as COM objects that relies on specific behavior of the framework you are using. It may change in future frameworks and no longer allow getting an IUnknown on an arbitrary managed object, in which case you need to do the boxing yourself. Just put your value in a .NET object you can expose to COM instead of relying on the framework creating an implicit COM object

public static class DataObjectHelper
{
    private static Guid IUnknownIID = new Guid("00000000-0000-0000-C000-000000000046");

    [DllImport("ole32", ExactSpelling = true, CharSet = CharSet.Unicode, PreserveSig = false)]
    public static extern System.Runtime.InteropServices.ComTypes.IBindCtx CreateBindCtx(int reserved = 0);

    [DllImport("ole32", ExactSpelling = true, CharSet = CharSet.Unicode, PreserveSig = false)]
    public static extern System.Runtime.InteropServices.ComTypes.IMoniker CreateObjrefMoniker([MarshalAs(UnmanagedType.IUnknown)] object punk);

    [DllImport("ole32", ExactSpelling = true, CharSet = CharSet.Unicode, PreserveSig = false)]
    public static extern System.Runtime.InteropServices.ComTypes.IMoniker MkParseDisplayName(System.Runtime.InteropServices.ComTypes.IBindCtx pbc, [MarshalAs(UnmanagedType.BStr)] string szUserName, out uint pchEaten);

    public static void SetObjectReference(this IDataObject obj, string format, object reference)
    {
        if (reference == null)
            throw new ArgumentNullException(nameof(reference));

        System.Runtime.InteropServices.ComTypes.IMoniker moniker = null;
        System.Runtime.InteropServices.ComTypes.IBindCtx context = null;
        try
        {
            moniker = CreateObjrefMoniker(reference);
            context = CreateBindCtx();
            moniker.GetDisplayName(context, null, out var id);
            obj.SetData(format, new MemoryStream(Encoding.UTF8.GetBytes(id)));
        }
        finally
        {
            if (context != null)
                Marshal.ReleaseComObject(context);

            if (moniker != null)
                Marshal.ReleaseComObject(moniker);
        }
    }

    public static object GetObjectReference(this IDataObject obj, string format)
    {
        var data = obj?.GetData(format) as MemoryStream;
        if (data is null)
            return null;

        return GetObjectReference(Encoding.UTF8.GetString(data.ToArray()));
    }

    private static object GetObjectReference(string monikerStr)
    {
        if (string.IsNullOrEmpty(monikerStr))
            return null;

        System.Runtime.InteropServices.ComTypes.IMoniker moniker = null;
        System.Runtime.InteropServices.ComTypes.IBindCtx context = null;

        try
        {
            context = CreateBindCtx();
            moniker = MkParseDisplayName(context, monikerStr, out _);
            moniker.BindToObject(context, null, ref IUnknownIID, out var obj);
            return obj;
        }
        finally
        {
            if (moniker != null)
                Marshal.ReleaseComObject(moniker);

            if (context != null)
                Marshal.ReleaseComObject(context);
        }
    }
}

Having given that example of how we do it currently, there is a certain simplicity in your broken solution, especially performance-wise, so I have spent more time to research why it was working for you and if theres a way to make it work reliable.

more technical details of what happens

So lets break down what you were doing again:

• you were calling OleGetClipboard, which returned a COM object instead of your managed object
• you were casting to IDataObject (or later to your custom copy of the interface)

Note that the second step still returns a COM object and not a managed object, regardless of which framework I tried it on, which indicates that you do not have a reference to your original object, but to "something else".

So I researched what OleGetClipboard is actually doing, here are the results:

• OleGetClipboard constructs an intermediate implementation of the native IDataObject to put between the producer and consumer. The clipboard is a global resource and implementations can have bugs or be incomplete, this intermediate objects tries to fix certain issues, mostly performing TYMED conversion the provider didn't bother to implement.
• The intermediate data object intentionally violates COM rules by allowing QueryInterface calls it doesn't understand to be forwarded to the original data object. This is not allowed by COM because you can't get back to the IUnknown of the intermediate data object (and QueryInterface for IUnknown must always return the same instance, so this is where it violates COM rules)
• the .NET runtime gets incredibly confused by this violation of COM rules, so even though the intermediate implementation forwards QueryInterface for the internal marker interface the runtime can't manage to unwrap the managed object
• your cast to IDataObject is also broken, the .NET runtime can't figure out the cast (still doesn't unwrap the object), and you get weird behavior. The behavior you observe only works for the first interface implemented on the data object, if you change interface order you suddenly get errors that you can't cast, even if it is ComVisible

So the result of that research is that OleGetClipboard returns a wrapper which will forward QueryInterface calls (violating COM rules) and the .NET runtime is too confused to resolve the reference.

The good news is that you can rely on the behavior of OleGetClipboard (its always been there and won't go away for compatibility reasons, applications rely on the ability to QI into the underlying object). The only thing you need to solve is help the .NET runtime to not be confused anymore. This is simple, just do the QI manually to move away from the wrapper object, once you are on the inner data object you have a "normal" COM object that plays by the rules and can be understood by the .NET runtime.

• Note that this solution does not depend on IDataObject being ComVisible - it just needs an arbitrary interface implemented on your data object but not implemented by the OLE data object.
• Note that this solution has a performance gain over your previous solution, your implementation did not unpack the managed object so you had a roundtrip through COM (including marshaling and passing through some sort of wrapper) for every function call. By unpacking the managed object properly you can talk directly to it in managed code, including casting to IDataObject regardless of it being ComVisible or not

helper code

[ComImport]
[Guid("59FFC7AA-BE08-4279-AC5E-C734D55429FD")] // random private GUID
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
interface ICustomDataObjectMarker { }

public static class ClipboardHelper
{
    [DllImport("ole32", ExactSpelling = true, CharSet = CharSet.Unicode)]
    private static extern int OleGetClipboard(out IntPtr data);

    public static IDataObject TryGetCustomDataObject()
    {
        IntPtr pWrappedDataObject = IntPtr.Zero;
        IntPtr pInnerDataObject = IntPtr.Zero;
        Guid knownInnerInterface = typeof(ICustomDataObjectMarker).GUID;

        try
        {
            // returns a wrapper object which violates COM rules so we use IntPtr to handle it
            if (OleGetClipboard(out pWrappedDataObject) < 0)
                return null;

            // manually do a QI to an interface not implemented by the wrapper, but implemented by your object
            if (Marshal.QueryInterface(pWrappedDataObject, ref knownInnerInterface, out pInnerDataObject) < 0)
                return null;

            // then let the .NET runtime take over, it unpacks the object and lets you cast to managed interfaces
            return Marshal.GetObjectForIUnknown(pInnerDataObject) as IDataObject;
        }
        finally
        {
            if (pInnerDataObject != IntPtr.Zero)
                Marshal.Release(pInnerDataObject);

            if (pWrappedDataObject != IntPtr.Zero)
                Marshal.Release(pWrappedDataObject);
        }
    }
}

[filipnavara]

Thanks for the incredibly detailed response. I especially like the clever solution at the end.

I guess, my final question is, can the code mentioned in the original post ever succeed? Given the removal of ComVisible attribute on the IDataObject and the behavior of OleGetClipboard I think the cast will always fail.

[weltkante]

I guess, my final question is, can the code mentioned in the original post ever succeed? Given the removal of ComVisible attribute on the IDataObject and the behavior of OleGetClipboard I think the cast will always fail.

Maybe previous versions of Windows didn't return a wrapper object, or previous versions of .NET didn't get confused by it violating COM rules, but in the current state of affairs, yes I believe this is dead and cannot enter that branch.

I think thats a good point though, it would be possible to use an internal marker interface as outlined above to bring this optimization back to life, for the stock DataObject at least.

While it would be technically possible I wouldn't want to use the WinForms IDataObject itself as marker interface, making it ComVisible again, because having it ComVisible would either break runtime compatibility or require breaking source compatibility as explained earlier.

@RussKie @JeremyKuhne what do you think?

[filipnavara]

Maybe previous versions of Windows didn't return a wrapper object

JFYI they did, at least all the way to Windows XP. I am too lazy to verify it any further since older versions are unsupported anyway.

[RussKie]

@kant2002 since you're doing work with COM wrappers, you may be interested in this discussion.

[filipnavara]

Haha. Thanks for looping @kant2002 in. I wanted to post a link to this issue to the Twitter thread but I got distracted and forgot about it 😅

[RussKie]

He seemed like the best candidate for nerd sniping :)

[kant2002]

Do I properly understand that initial request is to have fact below working.

[WinFormsFact]
public void DataObject_CustomDataObjectRoundTrip()
{
    var realDataObject = new CustomDataObject();
    Clipboard.SetDataObject(realDataObject);
    bool hasFoo0 = ((System.Windows.Forms.IDataObject)realDataObject).GetDataPresent("Foo");
    Assert.True(hasFoo0);

    var dataObject = Clipboard.GetDataObject();
    bool hasFoo1 = dataObject.GetDataPresent("Foo");
    Assert.True(hasFoo1);
}

And maybe for same-process calls, realDataObject == dataObject. Correct?

[weltkante]

Yes, but from what I gathered at other threads you aren't going to be able to support it, since it means user-defined custom COM objects and IDispatch support are required. It basically means full support for COM interop instead of handrolled inlined internal implementations you're currently doing.

Also this thread is a very advanced usecase, usually people don't implement their own COM objects, getting general clipboard and drag'n'drop working with all the WinForms-internal COM objects would probably be much more important in a first iteration.

[kant2002]

@weltkante Where IDispatch comes from? I was looking at IDataObject and cannot find any reference to it, or to VARIANT from which it can jump on my face. Obviously I may miss some special case. Please point to me which API may trigger requirement for IDispatch implementation since I trying to avoid that for now.

since it means user-defined custom COM objects and IDispatch support are required

What you mean user-defined custom COM objects? Regular CCW (COM callable wrappers), or something more, like exposing .NET object as COM object to external processes?

Regarding IDispatch in general it just doubles work required to implement RCW and CCW, since I have to support two methods of invocation. That's pain for now, but doable. I do not remember if I share with you, but there sample howto do IDIspatch https://github.com/dotnet/samples/tree/main/core/interop/comwrappers/IDispatch. Please correct me if I'm miss some subtleties which make things more complicated.

[weltkante]

Where IDispatch comes from? I was looking at IDataObject and cannot find any reference to it

Its the default when you make an object ComVisible(true) without explicitly opting out of it. I think I remember debugging the OLE code of the clipboard and noting OLE will query for IDispatch, but I may misremember. I've looked through the discussion, since IDispatch wasn't mentioned in this thread it may not be relevant for this particular usage - we had several issues in WinForms where it was relevant though, so maybe I'm starting to mix them up, sorry.

Regular CCW (COM callable wrappers), or something more, like exposing .NET object as COM object to external processes?

As far as this use case is concerned, exposing a .NET object to COM as CCW and unpacking it to the same managed object in the same process. The CCW is on a user-defined class and there also are user-defined interfaces (in addition to implementing the OLE IDataObject and WinForms IDataObject interfaces). If thats all possible to make working thats great, but having user-defined COM interop has previously been mentioned as an issue for the new ComWrappers model.

Regarding IDispatch in general it just doubles work required to implement RCW and CCW, since I have to support two methods of invocation. That's pain for now, but doable.

Thanks, I see, maybe I got the wrong impression then. That example (WebBrowser scripting) is certainly one of the cases which came up with requiring IDispatch support on arbitrary (ComVisible) managed objects. Don't know how high priority it is to keep the classic WebBrowser control working when there are new ones, but offtopic to DataObject/Clipboard/Drag'n'drop.

[RussKie]

Just to throw the wrench into works, the strategic target we'd want to get to is to re-write the current implementation of the Clipboard using a custom CCW/RCW shared by @AaronRobinsonMSFT (see: #4555 (comment)).

[weltkante]

ust to throw the wrench into works, the strategic target we'd want to get to is to re-write the current implementation of the Clipboard using a custom CCW/RCW

Thats fine and matches what AOT needs, but without AOT you can do an incremental implementation and fall back to asking the builtin marshaler for a CCW when getting passed a user-defined object - the problem is that (as far as I understand) AOT doesn't like that very much and prefers everything being custom CCWs/RCWs which is tough when you have to support user-defined ComVisible objects and interfaces (you basically have to reimplement the interop mappings as far as I understand, not impossible but certainly a lot of work nobody wants)

of course WinForms can declare those scenarios unsupported under AOT if it has to

or you can take a breaking change to the API and no longer allow custom data objects in a future .NET version (so anyone needing them would have to do Windows interop directly using their own custom CCWs instead of going through WinForms). That would probably break some advanced use cases like this issue though, which rely on being able to unwrap a CCW of a custom DataObject back to a managed object so WinForms can use the managed IDataObject implementation instead of the COM interop.

[kant2002]

I think .NET 6 will solve custom IDataObject support if we retreive objects using ComWrappers.GetOrCreateObjectForComInstance(ptr, CreateObjectFlags.Unwrap) where CreateObjectFlags.Unwrap is new flags
https://github.com/dotnet/runtime/blob/57bfe474518ab5b7cfe6bf7424a79ce3af9d6657/src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/ComWrappers.cs#L66

My understanding that if ptr is CCW to existing runtime object, it will take that runtime object and return it, instead of creation new RCW for COM instance.

[weltkante]

yes, unwrapping is possible, but the question is who creates the CCW of a custom DataObject? It needs to respond to interface queries to things other than IDataObject, including user-defined interfaces. I don't know the new API enough to know if the WinForms ComWrappers can unwrap an RCW obtained from a user-defined ComWrapper CCW. The alternative would be the WinForms ComWrapper to be solely responsible for clipboard CCWs and having to dynamically generate interface tables like the old marshaling layer did.

[kant2002]

I think if we talk about implementations like that
https://github.com/AvaloniaUI/Avalonia/blob/ef39b9487897fe5a7ae9c4502f0e8ce05c8f4612/src/Windows/Avalonia.Win32/DataObject.cs
I see two options, not mutually exclusive.

1. Then WinForms should either provider a way to specify ComWrapper to use for wrapping/unwrapping clipboard objects to implementor of the library. Maybe something like Clipboard.GetDataObject(ComWrappers) and Clipboard.SetDataObject(object, ComWrappers) methods.
2. Other option, is to have well known set of interfaces, and provide CCW only to them when call OleSetClipboard. For RCW this can be solved by implementing IDynamicInterfaceCastable like here https://github.com/dotnet/samples/tree/2cf486af936261b04a438ea44779cdc26c613f98/core/interop/IDynamicInterfaceCastable/src/ManagedApp

I do not think about cases like in Avalonia. That's make things a bit complicated for ComWrappers 😞 and especially for NativeAOT I have to think a bit more.

[RussKie]

This is the general direction we'll likely be taking the clipboard in the future: https://github.com/RussKie/ClipboardRedux

[RussKie]

Is #6976 having any effect on this issue?

[kant2002]

No, the #6976 does not have effect on the issue, but #7087 is.
After #7087 lands, you can start juggling how to create CCW for DataObject. right now I support only one interface - IComDataObject, but we can create CCW which support 2 interfaces IComDataObject + IWinFormsDataObject. For IWinFormsDataObject we can take GUID from Desktop framework. And then new proxy can be used to create necessary behavior

[RussKie]

Related to #6269