[WIP] Implement FFI for GDScript.

[bruvzg]

Implement simple FFI for the GDScript using libffi.

• Adds FFILibrary class.
• Adds functions to get writable pointer of the PackedByteArray and copy data from a pointer to a PackedByteArray.
• C-struct argument/return types are encoded/decoded to the GDScript Array.

Simple demo - Click to expand

#if defined(_MSC_VER)
    //  Microsoft 
    #define EXPORT __declspec(dllexport)
#elif defined(__GNUC__)
    //  GCC
    #define EXPORT __attribute__((visibility("default")))
#else
    #define EXPORT
#endif

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

struct TestStruct {
    int a;
    long b;
    float c;
};

EXPORT struct TestStruct function_a(int p_a, float p_b) {
    struct TestStruct s;
    s.a = p_a;
    s.b = p_a * 10;
    s.c = p_b / 2.0;

    return s;
}

EXPORT float function_b(struct TestStruct p_s, int p_a) {
    float a = p_s.c + p_s.b * 10 + p_s.a * 100 - p_a;
    printf("==== %d %d %f %d = %f ====\n", p_s.a, p_s.b, p_s.c, p_a, a);
    return a;
}

EXPORT void function_c(int p_a, float p_b, char *p_str) {
    printf("==== %d %f %s ====\n", p_a, p_b, p_str);
    memcpy(p_str, "output test", 11);
}

func _test(path):
    # Open library.
    var lib = FFILibrary.new()
    lib.open(path, FFILibrary.ABI_DEFAULT)

    # Bind C-struct type.
    # args: name, element types...
    lib.bind_struct("TestStruct", FFILibrary.TYPE_SINT, FFILibrary.TYPE_SLONG, FFILibrary.TYPE_FLOAT)

    # Bind C-functions.
    # args: return type, name, argument types...
    var function_a = lib.bind_method("TestStruct", "function_a", FFILibrary.TYPE_SINT, FFILibrary.TYPE_FLOAT)
    var function_b = lib.bind_method(FFILibrary.TYPE_FLOAT, "function_b", "TestStruct", FFILibrary.TYPE_SINT)
    var function_c = lib.bind_method(FFILibrary.TYPE_VOID, "function_c", FFILibrary.TYPE_SINT, FFILibrary.TYPE_FLOAT, FFILibrary.TYPE_POINTER)

    # Call function A.
    print(function_a.call(44, 89.6))
    print("should be: ", [44, 440, 44.8])

    # Call function B.
    print(function_b.call([10, 20, 30.5], 233))
    print("should be: ", 997.5)

    # Call function C.
    var s = "Test String".to_utf8_buffer()
    s.push_back(0) # Add terminating zero.
    var ptr = s.unsafe_get_ptr() # Get writable pointer.
    function_c.call(10, 20.5, ptr)
    print(s.get_string_from_utf8())
    print("should be: ", "output test")

Tested (with the aforementioned demo only) on:

• macOS x86_64 and ARM64
• Windows (MSVC) x86_64 and x86_32
• Linux x86_64 and ARM64

TODO:

• Documentation.
• Test all primitive types and nested structures.
• Build flags need to be carefully evaluated and tested on each platform and architecture, since it involves pre-processing and building assembly sources.
• Evaluate thread safety.

[vnen]

This is interesting. I had an idea like this for quite some time but never got around to implementing something.

[dsnopek]

I haven't looked at the code, but conceptually I think this is a great idea! Integrating with 3rd party libraries today requires using GDExtension, but this would provide an alternative GDScript-only path to do it.

[Calinou]

Integrating with 3rd party libraries today requires using GDExtension, but this would provide an alternative GDScript-only path to do it.

This is interesting in particular for people who've had to use C# builds of Godot in the past just to be able to use its Invoke functionality, which calls into native libraries.

[MarioLiebisch]

Neat, missed this so far and will have a look later.

IMO important concern: It might be reasonable to disable this functionality by default in the editor (and editor started projects; similar to how you have to enable/opt-in addons), simply because it could allow malicious code to do all kind of nasty stuff default tool scripts can't do.

Maybe decoupling this here, could be worth having a "Do you trust this source?" kind of banner once certain function calls or classes are used in a project?

[bruvzg]

IMO important concern: It might be reasonable to disable this functionality by default in the editor (and editor started projects; similar to how you have to enable/opt-in addons), simply because it could allow malicious code to do all kind of nasty stuff default tool scripts can't do.

GDScript can download and run executable or load GDExtension (which can have arbitrary native code) so FFI won't make it more insecure than it already is.

[MarioLiebisch]

Hm, true.

[Calinou]

Maybe decoupling this here, could be worth having a "Do you trust this source?" kind of banner once certain function calls or classes are used in a project?

See godotengine/godot-proposals#5010.

[MarioLiebisch]

What's the status on this PR? Would love to potentially try to rebase and get a bit more into this feature over the Holidays, as there's sadly not too much activity here.

[bruvzg]

What's the status on this PR?

Needs testing on various platforms and architectures, it works fine on arm64 and x86-64, but extremely buggy on 32-bit x86, and not tested on any other architectures.