Bypass JVM for native->Java->native calls

[ws909]

FT_New_Library requires an instance of FT_Memory. FT_Init_FreeType is not an option due to its threading limitations. FT_Init_FreeType does not support reference counting of the library, and so is not an option if that's a requirement. Pardon my initial misunderstanding, believing there was a multithreading difference between the two functions.

FT_Memory is a structure whose members are function pointers for alloc, free, and realloc, as well as a void* pointer to user data. There is no default instance of this structure, in either Java or FreeType. org.lwjgl.system.libc.LibCStdlib provides Java and Java native methods for these functions, however, their addresses are not available.

It is possible to supply lambdas that allocate native memory from Java, however, going via Java for native-to-native memory management, is likely a big performance obstacle.

Another very cumbersome way to access these functions, is through the FFI API currently in preview, though, downcall handles are not yet supported on AArch64, which creates a compatibility concern. I did not yet test if the function pointers are retrievable.

How can a high-performance, initialized FT_Memory instance be acquired?

[ws909]

Apparently, acquiring the pointers was quite easy, but injecting them into the FT_Memory instance was not so great.

final var allocator = MemoryUtil.getAllocator();
final var alloc = allocator.getMalloc();
final var free = allocator.getFree();
final var realloc = allocator.getRealloc();
            
final var memory = FT_Memory.malloc().user(0);
final var address = memory.address();
    
memPutAddress(address + FT_Memory.ALLOC, alloc);
memPutAddress(address + FT_Memory.FREE, free);
memPutAddress(address + FT_Memory.REALLOC, realloc);

This seems to work, so far. The problem is that LWJGL's FT_Memory class expects the function pointers to be functions generated by LibFFI, sourced from FT_Alloc_Func, FT_Free_Func and FT_Realloc_Func.

So doing ((FT_Memory) ...).alloc() invokes this chain:

public FT_Alloc_Func alloc() { return nalloc(address()); }
...
public static FT_Alloc_Func nalloc(long struct) { 
    return FT_Alloc_Func.createSafe(memGetAddress(struct + FT_Memory.ALLOC)); 
}
...
public static FT_Alloc_Func createSafe(long functionPointer) {
    return functionPointer == NULL ? null : create(functionPointer);
}
...
public static FT_Alloc_Func create(long functionPointer) {
    FT_Alloc_FuncI instance = Callback.get(functionPointer); // Does this create any problems?
    return instance instanceof FT_Alloc_Func
        ? (FT_Alloc_Func)instance
        : new Container(functionPointer, instance);
}

This crashes the JVM:

Exception in thread "main" java.lang.NullPointerException: Cannot invoke "org.lwjgl.system.libffi.FFIClosure.user_data()" because the return value of "org.lwjgl.system.Callback$ClosureRegistry.get(long)" is null
	at org.lwjgl@3.3.2-snapshot+8/org.lwjgl.system.Callback.get(Callback.java:190)
	at org.lwjgl.freetype@3.3.2-snapshot+8/org.lwjgl.util.freetype.FT_Alloc_Func.create(FT_Alloc_Func.java:31)
	at org.lwjgl.freetype@3.3.2-snapshot+8/org.lwjgl.util.freetype.FT_Alloc_Func.createSafe(FT_Alloc_Func.java:40)
	at org.lwjgl.freetype@3.3.2-snapshot+8/org.lwjgl.util.freetype.FT_Memory.nalloc(FT_Memory.java:242)
	at org.lwjgl.freetype@3.3.2-snapshot+8/org.lwjgl.util.freetype.FT_Memory.alloc(FT_Memory.java:82)

[ws909]

Even just running that first section of code:

final var allocator = MemoryUtil.getAllocator();
final var alloc = allocator.getMalloc();
final var free = allocator.getFree();
final var realloc = allocator.getRealloc();
            
final var memory = FT_Memory.malloc().user(0);
final var address = memory.address();
    
memPutAddress(address + FT_Memory.ALLOC, alloc);
memPutAddress(address + FT_Memory.FREE, free);
memPutAddress(address + FT_Memory.REALLOC, realloc);

final long library;
try (final var stack = MemoryStack.stackPush()) {
    final var pointer = stack.mallocPointer(1);
    final var returnCode = FreeType.FT_New_Library(memory, pointer)
    assert returnCode == 0;
    library = pointer.get(0);
} 
FreeType.FT_Add_Default_Modules(library);

crashes the JVM.
Attempting to load a face without first calling FreeType.FT_Add_Default_Modules(library) correctly results in an FT error.

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGBUS (0xa) at pc=0x000000a24000dc80, pid=93911, tid=259
#
# JRE version: OpenJDK Runtime Environment (19.0+36) (build 19+36-2238)
# Java VM: OpenJDK 64-Bit Server VM (19+36-2238, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, bsd-aarch64)
# Problematic frame:
# C  0x000000a24000dc80

[ws909]

Using this does in fact work:

final var memory = FT_Memory.malloc().user(0);
memory.alloc((ftMemory, size) -> org.lwjgl.system.libc.LibCStdlib.nmalloc(size));
memory.free$((ftMemory, address) -> org.lwjgl.system.libc.LibCStdlib.nfree(address));
memory.realloc((ftMemory, currentSize, newSize, address) -> org.lwjgl.system.libc.LibCStdlib.nrealloc(address, newSize)); 

FreeType.FT_New_Library(memory, ...)

But we're back to the problem of executing via the JVM.

This also makes it quite obvious why the native C malloc, free and realloc functions don't work: they don't expect the FT_memory and cur_size arguments, and the flipped argument order of realloc.

[ws909]

FT_New_Memory is an internal function of FreeType, in ftsystem.c, and is what FT_Init_FreeType uses.

[ws909]

Is it possible to use LibFFI or Java's FFI module to set up a native function that wraps a method, discards 0-2 arguments, and reorders arguments, and that doesn't interact with the JVM? Will LibFFI or FFI be able to wrap such a native function in a dynamically created Java lambda or similar? If yes, will this be handled internally by LWJGL (currently calls into LibFFI) when acquiring Java instances of native function pointers (f.ex FT_Alloc_Func)?

[ws909]

I was looking into writing a custom subclass of FT_Memory that handles this, however, that also relies on custom subclasses of the callback interfaces.

I tried this, but came short as the super class's constructor is package-private.

class NativeAllocFunc extends FT_Alloc_Func {
    
    public NativeAllocFunc(final long address) {
        super(address); // error
    }

    @Override
    public long invoke(final long memory, final long size) {
        return invokePPP(memory, size, super.address());
    }
}

[Spasi]

Hey @ws909,

• Using MemoryUtil.getAllocator() to get the function addresses for malloc/free/realloc is correct. You could also grab a specific allocator, e.g. via JEmalloc.Functions.malloc.
• The issue is indeed that the FT_Memory callbacks do not have matching signatures with the standard allocation functions. The extra FT_Memory parameter must be dropped.
• LibFFI cannot be used to do this.
• A subclass would not offer anything as a workaround. The signature is hardcoded in the native library.

The possible ways to address this:

• Just use FT_Init_FreeType. Do you really need reference counting, i.e. multiple FreeType instances?
• If yes, use LWJGL's debug allocator (or a custom allocator) to log allocations/deallocations within FreeType. Does it really do so many allocations that the callback overhead becomes an issue? Is it possible that you're worrying about it prematurely?
• LWJGL could provide custom callback implementations that drop the parameter and call a user specified allocation function (with the standard signature). However, that requires custom native code and we have nowhere to put it. The FreeType shared library is a normal FreeType build, built outside the LWJGL repo (see LWJGL-CI/freetype). Also, the bindings are designed to be usable with any FreeType build. We'd need a second shared library for the custom implementation and I really don't want to do that.

Finally, the solution I've been considering for a while is having bindings to a native JIT compiler. It would solve this problem and create opportunities for many more improvements, both within LWJGL and in user programs. However, it's hard to find a solution that satisfies the following:

• The compiler must be small enough for deployment with the core LWJGL module. E.g. LLVM JIT is out of the question due to binary size alone.
• It must support all platforms/architectures that LWJGL supports.
• It should strike a good balance between compilation speed and runtime performance.

In the past, I've experimented with TinyCC and was fairly happy with it, but not entirely convinced. I've been looking for a better solution for a while, but there seems to be nothing out there that satisfies the above constraints as well as TinyCC. It's also been very actively maintained lately. Unless someone has a better suggestion that I've missed, I'll give it another chance soon.

[ws909]

Just use FT_Init_FreeType. Do you really need reference counting, i.e. multiple FreeType instances?

FT wasn’t designed for multithreading, and this is one of the most common recommendations for multithreading.

Is it possible that you're worrying about it prematurely?

Most likely.

I attempted the subclassing before I realized that the signatures are different. We all miss the obvious every now and then.

It’s quite interesting that you’ve been considering a custom JITC, and is something I’ve not been aware of at all. I do not possess knowledge of any JIT compilers, however. Seems like TinyCC compiles C code, which is not necessarily necessary. There seems to be JEP drafts such as this one to bring this functionality into the JDK. I can remember having read, possibly in a JEP, about future functionality to write bytecode at runtime. This could potentially be a solution. I looked for that JEP, but couldn’t find it.

[Spasi]

JVMCI would be useful if we wanted to implement a compiler. That would be a very difficult undertaking, outside the scope of LWJGL. What we need is another library to do the hard work for us (calling-convention/ABI details, register allocator, optimizations, etc).

C code isn't necessary, but that's what TinyCC uses as input. We could use LLVM bitcode, or WebAssembly, or whatever, as long as the constraints described above are satisfied.