Test Suite Overhaul and Improvement

[basshelal]

The current test suite is good but doesn't cover many cases well enough and sometimes not at all.

I feel the quality and coverage of the tests can and should be significantly improved to be able to:

• More confidently and quickly add new features
• Improve and refactor the existing code, including cleanups and simplifications, it is more valuable to remove code than to add new code but also much scarier
• Discover bugs or missing expected features, NoClassDefFoundError when callback method invoked #274 is a good example of this
• Better understand the codebase
• More easily debug bugs by using a specific test as the start of a debug session, having more specific tests makes this smoother
• Show example usage, since many tests will look similar to callers' code (though example code is better for this)

This relates partially to #253, since both are proposed so that we can have an easier time improving the existing code, especially regarding simplifying the code and improving and understanding performance.

This is a large endeavor that is more akin to multiple issues and will be fixed with multiple PRs each improving a small section of the test suite.

[basshelal]

Checklist (Subject to Change):

C types

• Numeric types (int, float, enum etc) Overhauled tests for numeric types #291
• Numeric pointers (int *, float *, void** (pointer of pointers, since pointer is merely a number)) Overhauled tests for pointers containing numeric types #293
• Structure types (struct, union)
  • Containing numerics and numeric pointers (int, int *, void * etc) Overhauled tests for structure types containing numeric types #295
  • Containing structures and structure pointers (struct, union, struct *, union * etc) Overhauled tests for structure types containing structure types and pointers to them #296
  • Containing function pointers (int (*function)(int), struct*(*function)(struct *) etc)
  • Containing arrays (int[], struct[], struct*[], void*[] etc)
• Structure pointers (struct *, union *)
• Function pointers (int (*function)(int), struct*(*function)(struct *) etc)
• Arrays (int[], struct[], struct*[], void*[] etc)

Global Variables

• Numeric types and their pointers (int, float, enum, int* etc)
• Structure types and their pointers (struct, union, struct*, union*)
• Arrays (int[], struct[], struct*[], void*[] etc)
• Function pointers (int (*function)(int), struct*(*function)(struct *) etc)
• Consider const correctness for these as well especially pointers which have 2 const-nesses, const int* vs int* const and the combined const int* const

Function Invocation

• Pass and return by pointer (Numerics, Structure and Function)
  (int *function(int *a, struct *b), struct *function(callback_t a, void *b) etc)
• Pass and return by value (Numerics, Structures and Arrays) (we don't support some of this yet but we can write the tests and have them disabled with a message saying not yet implemented)
  (int function(int a, struct b), struct function(int[] a, struct[] b) etc)

Memory

• Pointer class
• Struct <==> Pointer conversion
• All memory types (DirectMemoryIO, BoundedMemoryIO etc)
• Memory lifecycle with regards to JVM GC

Library Loading

• All LibraryOptions
• Both loading strategies (Reflection/Proxy or Asm/Bytecode)
• Library path selection behavior
• Multi-library single interface mapping, putting mappings of multiple libraries into a single Java interface (haven't tried it but should be possible I think?)
• Bad mapping errors
• Library load and symbol not found errors

Annotations

• Parameter annotations (@In, @Pinned etc)
• Type annotations (@size_t, @u_int64_t etc)
• Library annotations (@IgnoreError, @StdCall etc)

Final Touches

• Remove added test suite TODOs (any I may have added)
• Code cleanup and/or clarification (make tests that were from different PRs reuse code and add more clarifications when needed)
• If the tests begin to test platform-dependent functionality (which is inevitable) we should see if we can add different platforms on our GitHub Actions workflow

After Completion

Probably turn these into their own issues upon completion:

• Improve user documentation based on specification made from the tests
• Improve in-code documentation and comments based on findings from tests, tests can be individually used as the start of debug sessions which allows for more finely tuned debugging and understanding the code flow
• Create example scenario(s) as tests within the repository, this enforces a tighter coupling (regarding maintenance) of the examples to the main code which I think is better for long term usability by users, in addition to being a "real-world" test case.

[headius]

Great list and great initiative to start pushing this forward! We can swipe examples from the various places we have them and turn them into nice readable tests. This will improve documentation as well as coverage. Let me know where I can help!

[paulocabrita-ionos]

Hello.

Is there any news related to a struct being passed by value? Returning a struct and passing a struct into a parameter to a native function?

I know it's related to:

• Add support for Struct passed or returned by value? #262 ;
• Return a struct by value #14 ;

Right now, I'm kinda stuck because I have to call some native functions and pass structs as value.

I have the source code of the native library (it's an open source project) but making changes to the code will be a nightmare.

I love JNR (congrats! :)) but I'm also thinking on mixing with JNA. JNA deals with struct by value: https://segmentfault.com/a/1190000041817628/en

What do you think about the idea of combining JNR and JNA? Using JNA just to make the calls of the methods having struct by value.

[headius]

I think it's probably best for us to have the missing functionality in JNR. I am not opposed to working toward merging or sharing some backend logic for JNR and JNA, and of course we're all looking forward to Project Panama.

[paulocabrita-ionos]

We have a project that we have to create a Java Binding of an IoT solution which is written in C. I'm gonna share a little bit of our experience so far:

1. Panama: promising technology but... very poor documentation. We didn't have success in some forms of accessing the native library and we didn't like the way of doing a "jextract" before. Yeah, the stub creation is great but.... Also, we couldn't go to the Panaman because we had to go to the JDK 17 version;
2. JNR: We are very happy with the JNR (configuration and usage) but.... does not manage the "struct by value"
3. JNA: After using JNR, we tried JNA but... we started to have errors that we didn't have with JNR.

So... no perfect solution. What we did is: go back to JNR and change some native functions (we have the source because it's a open source project). We still don't know if at the end of the day we will have success but it's the path that we have chosen. Also, I don't like to apply a patch to an open source project before using JNR.

If you decided to carry on with the implementation of the "struct by value" in JNR maybe we can help.