Replies: 1 comment 17 replies
-
Looks cool! I do mention this later on, but I did want to mention up front that the focus for Biohazrd has been the .NET 5 CoreCLR runtime. This is partially due to the fact that it uses C#9 function pointers, which officially-speaking are only supported on .NET 5. If I am remembering correctly, some bugs in CoreCLR around CLI function pointers had to be fixed. (The feature already existed within the CLR to support C++/CLI and the internals of delegates, but was never exposed in C# until recently.) I bring this up since I saw you're using Mono. I have 0 clue what the situation is with Mono, but there is a chance that could be a problem. Mono is a bit of a dead end though, so maybe this is a good excuse to replace it with CoreCLR? 😁 (I know, big ask. Plus I've heard CoreCLR is harder to embed compared to Mono, but hey maybe the added performance benefits make it worth it.) As I note below, Biohazrd could potentially be retooled to use delegate-marshaled function pointers, but those have a bunch of problems and don't jive well with Biohazrd's architecture.
Yup, I'm familiar with that mood. I tired both before creating Biohazrd.
Would definitely love to have you! Every passing day I'm worried Biohazrd is way too "me" and as much as I love using it, I want a framework that benefits everyone not just me. I've had a handful of other people put eyes on it, but I'm still the only one who's made substantial use of it.
I definitely need to spend more time on documentation, will definitely use this conversation and those on imgui_private to inform areas I should focus on. Object Lifetime
So the answer currently is...it kind-of doesn't? Not to say I haven't thought about it, it's more that Biohazrd takes a different approach to this issue than SWIG or CppSharp do. As you've noted, there's no one-size-fits all approach for handling the relationship between managed and unmanaged object lifetimes. Biohazrd does not attempt to try and prescribe a specific philosophy because no matter what that philosophy would not work well for everyone. A good example is Direct3D. A dictionary associating the unmanaged identity to a managed identity would be unecessarily inefficient here because it'd make a lot more sense to just store a Similarly, in Direct3D 12 the lifetimes of objects needs to be carefully considered so it doesn't really make sense to try and add garbage collection capabilities to the managed identity of D3D12 objects. Conversely though, Direct3D 11 is much more liberal about object lifetimes and the driver handles most of it for you, so allowing a D3D11 object to be released when the managed identity is collected might be desirable. Another important thing to consider is that different abstractions have different performance characteristics. For instance a tiny wrapper struct that just hides the C++ pointer is basically free, whereas a fat object with a finalizer makes allocating the managed identity of an object much more expensive: (Source for this benchmark)
Plus certain designs for the managed identity enable different capabilities which may or may not be important depending on the library. A good example here is the `ManagedLoggerBase in InfectedTensorRT -- That repo is still private so check your email for access. That type allows a managed C# object to inherit from a C++ class and override its virtual methods. All that boilerplate can add cost and this extra type adds noise to the API surface, so in cases where it will never be used it doesn't make sense to generate it for no reason. (As an aside, I want to note that InfectedTensorRT's output is mega jank because it uses a dirty hack to add namespaces from before Biohazrd supported namespaces.) Biohazrd's philosophyMy general philosophy for interop libraries is that they should be done in two distinct layers:
The main reason why I think this distinction is important and that having both available is important is that well-meaning safe layers routinely inhibit certain capabilities of an API. One example I've seen prevalent in a lot of interop libraries are APIs which take pointers that can be null to indicate that parameter shouldn't be used. These types of APIs are (generally) only discoverable by reading the documentation, so safe interop layers almost always miss a few. Sometimes the lines can blur a little bit where the safe(r) helper methods are emitted alongside the dirty unsafe methods. For example, here's what my private Direct3D12 wrapper generates for [DllImport("d3d12.dll", CallingConvention = CallingConvention.Cdecl, EntryPoint = "#101", ExactSpelling = true)]
public static extern HRESULT D3D12CreateDevice(IUnknown* pAdapter, D3D_FEATURE_LEVEL MinimumFeatureLevel, in Guid riid, void** ppDevice);
[MethodImpl(MethodImplOptions.AggressiveInlining), DebuggerStepThrough, DebuggerHidden]
public unsafe static HRESULT D3D12CreateDevice<TD3D12Device>(IUnknown* pAdapter, D3D_FEATURE_LEVEL MinimumFeatureLevel, out TD3D12Device* ppDevice)
where TD3D12Device : unmanaged, IID3D12Device
{
TD3D12Device* __ppDevice;
HRESULT __returnValue = D3D12CreateDevice(pAdapter, MinimumFeatureLevel, IUnknown<TD3D12Device>.InterfaceId, (void**)&__ppDevice);
ppDevice = __ppDevice;
return __returnValue;
}
[MethodImpl(MethodImplOptions.AggressiveInlining), DebuggerStepThrough, DebuggerHidden]
public unsafe static TD3D12Device* D3D12CreateDevice<TD3D12Device>(IUnknown* pAdapter, D3D_FEATURE_LEVEL MinimumFeatureLevel)
where TD3D12Device : unmanaged, IID3D12Device
{
TD3D12Device* __ppDevice;
D3D12CreateDevice(pAdapter, MinimumFeatureLevel, IUnknown<TD3D12Device>.InterfaceId, (void**)&__ppDevice).ThrowIfFailure();
return __ppDevice;
} Basically I've taught my generator to recognize the How Biohazrd helps with the managed interop layerCurrently Biohazrd focuses on that unsafe layer, but enables you to augment generation to generate the safe layer. The plan is that eventually Biohazrd will ship with some common patterns and/or provide guidance on how to generate your safe interop layer. (For instance, Biohazrd could pretty easily generate a
To answer this question directly: Yes. You do it using custom declarations. I don't have any great examples in a public generator, but here's the one used for that previous Direct3D 12 factory pattern example. (Apologies that example references some stuff not in this repo. I haven't finalized the shape of things like Templates
The story around templates is definitely still evolving as none of the current target libraries make heavy use of them except for OpenCV. Biohazrd still mostly leans on the generator author to decide how they will be translated to C# and doesn't provide much help in doing so. In my experience templates come in a handful of shapes:
Loose generic functions are not handled at all right now out of pure virtue that they haven't been needed yet. As you note, these will likely just involve shoving them into an additional .cpp file to explicitly instantiate them as needed. Generic POD types fall into two categories depending on whether or not they can be represented by .NET generics. If they can, ideally you would just make them C# generic structs so they're just as nice to work with in C# as they are in C++. This mostly just depends on whether they use numeric constants as template parameters. (Think of a matrix type with arbitrary dimensions.) Biohazrd contains some very experimental support for automatically instantiating all implicit template instantations and generating types for them. You can see hints of this in this test, and here's an example from OpenCV where a templated Finally there's those generic classes. Which are basically a combination of both. As noted earlier, some templates can be represented in C# as generic types. A good example of this is the Eventually I'd like it so that Biohazrd will automatically instantiate all of the implicitly instantiated templates and expose them as rigid types in C# with the option for the generator author to replace them with something more friendly as I did with Inline functions
Yup, this is basically how Biohazrd handles inline methods today. Here's an example from InfectedPhysX. (Taking a pointer to an inline function causes it to be included in the DLL thanks to a convenient technicality in the C++ spec stating that function pointers to inline methods must all be the same.) Wrapping STL containers
Edit: I noticed after writing all of this that you're using EASTL. If you use it everywhere that Biohazrd would be involved, that may simplify things since it's not as much of a moving target as whatever arbitrary STL gets picked up by whatever C++ compiler was used to build the project. As you've probably gathered, Biohazrd does nothing special to make life with STL containerrs tolerable/usable, but this is definitely something I'd like to do whenever I get around to them. I already do something similar for the generated constant array helpers. (I know there's some evil things happening in that code that assume the constant array types never get thrown on the managed heap, it's on my mental roadmap. Ideally they'd be Manual implementation
Yup! Biohazrd is designed to allow pretty arbitrary manipulation of the declaration tree. Going back to the Another example of this is that in my private InfectedWin32 wrapper, I manually define this type for using System;
namespace InfectedWin32
{
public readonly struct ATOM
{
private readonly UInt16 Value;
private ATOM(UInt16 value)
=> Value = value;
public static explicit operator UInt16(ATOM atom) => atom.Value;
public static explicit operator ATOM(UInt16 value) => new ATOM(value);
// Some functions like UnregisterClassW or CreateWindowExW can take a class name char* or an ATOM. This makes it easier to use those APIs.
public unsafe static implicit operator char*(ATOM atom) => (char*)atom.Value;
public unsafe static explicit operator ATOM(char* value) => new ATOM(checked((UInt16)value));
public bool IsValid => Value != 0;
public static readonly ATOM Zero = new ATOM(0);
}
} Within the InfectedWin32 generator I have a transformation which simply replaces the Strings!
Same. This sort-of fits into the safe interop layer situation I mentioned where Biohazrd doesn't do a whole lot to help you out yet.
|
Beta Was this translation helpful? Give feedback.
-
Hey!
After reading your message i am very intrigued by your project! I myself maintain a project (game engine) of considerable size which has C# bindings. Road was long and complicated. I tried rolling my own bindings generator. I tried CppSharp. I finally ended up using SWIG. It is a great project, but age really shows, and when it comes to a point where extra features are needed - there is no support and i hate every second working with SWIG codebase. Something better would be nice and Biohazrd looks like something much better! Lack of C layer is especially exciting! CppSharp also avoids C layer, however i had quite a hard time with it. Biohazrd seems to have a potential to accomplish the task much better and we would love to switch if we could cover all our bases. And i would not mind contributing my time towards the project now that i would not have to fix SWIG ;)
So i have some questions (and maybe answer could end up in readme as a feature table) as i have not found a detailed list of supported features. Here is a list of things i have encountered wrapping a non-trivial project:
My project extensively uses intrusive refcounting by storing C# object handle in the native class in order to prevent garbage collection of managed instance if user loses all references to it. Engine may keep a native reference alive, and managed object may be providing logic implementation via overriden virtuals. This is achieved by inserting some custom logic at the points where C++ pointer is getting translated to C# object and in
Dispose()
/Finalize()
. Good thing about all of this is that we can always return same C# object for a certain C++ object pointer, by fetching it from object handle stored inside C++ object. A natural consequence of this is that we can mark C# object as "expired" if native side frees last reference and deallocates C++ object. Unity works this way AFAIK.Another important machinery i implemented is caching of C# object instances. This is required for non-refcounted objects since there is no space to store handle of managed object in them. This is important, because we use event handlers that take a hashmap as a parameter. Such handlers get called many times per frame and we can not afford allocating a new C# wrapper object for same hashmap pointer many times per frame. This is implemented same way like refcounted object customization - injecting logic at T* -> C# object translation and
Dispose()
/Finalize()
.Is it possible to do such custom code injections to customize how wrapper works?
Templates. In my experience it is enough if we could generate unique wrappers for each template class instantiation. However this would require generating an additional .cpp file which explicitly instantiates these templates. Maybe a similar technique would work for functions that dont get code generated (inline for example)? This is closely related to wrapping native containers. SWIG does it quite nicely by making native interface private, and injecting a public C#-like interface, so wrapped containers inherit
IEnumerable<>
and everything.Implementing certain types manually in C#. For performance reasons we have some classes implemented straight in C#, to avoid paying a cost of crossing managed/native boundary each time method is called. Can we do this for our own types?
Strings! Oh i hate strings. How is Biohazrd dealing with them? Not only we need to translate utf-8 string to whatever C# uses (and that costs), but we also need to handle:
std::string&
const std::string&
const char*
char*
would also be nicechar*
and it is user's responsibility to free itFunction pointers. Any support for those?
Virtual method overriding. I saw vtables are generated, which is great. I did not see if we can override them. I only looked a bit at physx wrapper though.
Are operators handled at all?
Any support for arrays? (Both as func parameter and as field in an object).
Default function parameter values? SWIG generates overloads. My custom bindings generator did a mix of
Nullable<>
+ naive parsing of default value expression strings. Now that i think of it, these native expressions could totally end up in a .cpp function and get returned.What about static constants of complex type? For this once again seems like .cpp returning constant would be required.
Forgot multiple inheritance. SWIG generates interfaces for classes used as a second+ base. This introduces some restrictions to other steps (like hiding getters/setters for of generated properties if a class inherits from interface). Another important detail is that SWIG keeps a C++ pointer to each base. Biohazrd probably knows how to upcast without going through C++? This was possible in CppSharp at least.
There are many more things we need, but from bird's eye view it seems like Biohazrd already handles them, or at least should, given how its implemented. Thank you for working on this, world really needs a good and modern bindings generator.
Beta Was this translation helpful? Give feedback.
All reactions