C interop today and in the future.

[dabrahams]

Unfortunately, calling a C-language function from LLVM is not as simple as creating a trivially-corresponding LLVM declaration for the C function and calling that. There are all kinds of irregularities in how a given C call should be lowered to LLVM across target architectures and platforms, and those details are currently captured in the Clang codebase.

However, LLVM does make an informal guarantee of regularity for calls taking pointer and pointer-sized integer parameters.

Summarizing John McCall from the linked talk,

When you have an LLVM type that would naturally turn into a bunch of pointers or pointer size integers on standard cpu architectures, the naive lowering… will actually work. So that's… an informal guarantee that you can rely on across platforms

(John defines register-filling types as pointer types and pointer-sized integer types; let's additionally define an LLVM-trivial function as any C function whose parameter types are register-filling, and whose return type is register-filling or void.)

but he goes on to caution,

that guarantee is extremely limited. Don't try to stretch too far… when you're interacting with a[n arbitrary] c declaration you really need to be using Clang to lower the interaction.

The future

As the linked talk says, that is exactly what Swift does, and it seems apparent that in order to ingest C headers and be able to use the types and functions declared there efficiently and cleanly from Hylo, we will eventually want to do the same. That will free us from relying on LLVM's informal guarantees and allow us to call into C from Hylo in the most efficient and straightforward way, without writing any glue code. It will also give us tools we'll want for building Hylo modules and consuming C modules. Swift/C++ interop should be a big help in this effort.

Stopgaps for calling C functions

But what do we do in the meantime? I can see several possible approaches, each building on the previous one:

1. Document LLVM's informal rules, using something like our existing @ffi declarations to make LLVM-trivial C functions visible to Hylo. Since few existing C functions are LLVM-trivial, we would often manually write LLVM-trivial trampoline functions in C and link them into a hylo program.
2. If we sacrifice the ability to make trampoline-free calls to the few existing C functions that happen to be already LLVM-trivial, we can dispense with ForeignConvertible. Instead, our @ffi declarations can simply declare the Hylo signature of a trampoline that uses Hylo's calling convention (which creates only LLVM-trivial functions, because everything is passed and returned by pointer), with the C code of a trampoline dereferencing each pointer argument as necessary to extract C argument values and store the C return type.
3. Create C++ templates that can easily generate an appropriate trampoline for any arbitrary C function, making trampolines far more straightforward to write.
4. Have the hylo compiler:
   1. generate trampolines in C or C++ source code from @ffi declarations
   2. invoke the appropriate compiler to create an object file
   3. automatically link the result into the executable.

How many of these steps to actually pursue before we integrate Clang's source into Hylo is an open question.

Calling Hylo functions from C

In general, function pointers handled by C code are not expected to use Hylo calling conventions, so a C function pointer that calls Hylo must refer to a callback wrapper, written in C (or generated using Clang), that marshals argument and return values into/out of a call to Hylo. In the most general case, the callback wrapper must be written with explicit knowledge of the Hylo function, which must have a mangled name that can be referred to from a C declaration. In many cases, though, the C signature includes a void* referring to “captures” of a notional closure of which the function pointer is the other part. In these cases it may be possible to store the address of the Hylo function to be called inside the captures, and a single callback wrapper for a given C signature can be used with multiple Hylo functions.

I expect many of the details of how this should work to be revealed by the process of manually creating callback wrappers.

[DibyaXPP]

Will their be Modern C++ Transpilling? Like I can write Hylo code with VSCODE, Transpile to C++ CPP File & Modules then compile with my old code in Visual Studio with MSVC Compiler.
I think destructive move is biggest concern for Hylo but i guess their proposal for that in C++ https://github.com/tringi/papers/blob/main/cxx-destructive-move.md

[dabrahams]

It's not on the roadmap, but C++ interop is (eventually) very interesting to us and we're not opposed to having transpilation as part of that picture. At this point we don't have the manpower to work on the design and implementation, but if you'd like to contribute, we could give your designs and PRs some attention.

[DibyaXPP]

I will love to help, but i need to understand how the language works first including mutable value semantics. Some more documentation will be really nice but obviously documentation is secondary, first language needs to evolve.

I love what u gals & guys doing.