Provide support for JEP424 foreign functions

[tianxiadys]

At its core, the JNA project is supported through JNI. Since Java 17, Java has already provided foreign function features, although it is only in preview, but I think JNA can also be launched with an alpha version to follow up testing

[tianxiadys]

If this is one of JNA's goals, I can contribute to it

[dbwiddis]

I changed the title to reference JEP 424 rather than Java 17. JDK 17 (LTS) support was still incubating, JDK 19 is in preview, and I think we should target support for JDK 21 LTS.

Some other comments:

• this will have to be developed in a separate branch and separate core features from platform mappings similar to the existing project
  • we may even consider a separate project to JNA under the same java-native-access organization
• as JPMS capability can be assumed, we should consider designing for modules at the outset

All that said, happy to also help contribute here.

[tianxiadys]

If we assume that at JDK21, this feature reaches GA, there are several ways to implement it
1 Create a new project, e.g. jna2
2 Non-backward compatible version upgrade, 5.x compatible JDK8-JDK20, 6.X+ compatible JDK21
3 contains both implementations and detects the JDK version at runtime

I personally prefer the second option, but the third may be more acceptable to everyone

[dblock]

Ideally JNA would offer a path where users of platform* do not have to do anything other than upgrade to take advantage of JEP 424. I believe (2) is a perfectly fine start and we can evaluate a path for (3) and whether it's worth it when there's a working PR.

[matthiasblaesing]

Before we get trigger happy, I think questions that need to be answered:

a) What benefit gives you a JNA based on JEP424?
b) Is there really a performance benefit in JEP424, if you use the same integration JNA currently uses? (Copying all memory for structures on all invocations, reading all data always, exposing all fields)?
c) Are JEP424 methods protected and need runtime flags? In the past reflection could take you everywhere in the JVM, with the module system this is very restricted, I doubt that methods, that allow to poke everywhere in memory are accessible without restrictions.
d) Would it be even feasible to create a package that retains the libffi based invocations, the same backwards compatibility and can use JEP424? I implemented method handle support using reflection. That worked, but has a high price: it is difficult to implement and will loose speed benefits, as now reflection is in the way of the hotspot compiler.

[dbwiddis]

What benefit gives you a JNA based on JEP424?

As a downstream library maintainer, I want an improvement over what I can do with JNA. I would consider JEP424-on-top-of-existing-JNA to be worse. If there is a connection in terms of "fallback implementation" would hope it could be in terms of an optional dependency that is not brought in unless desired for things that can't be done in JEP424.

Is there really a performance benefit in JEP424, if you use the same integration JNA currently uses?

Given the "default", yes, as demonstrated in a 5x speed improvement (non-formal benchmark) for basic function invocation and structure handling. I think the main benefit was likely in not using reflection, instead using pre-calculated offsets with the JEP424 syntax. This is contrary to @dblock's ideal of "same API" although I suppose there could be a "one time" translation between those formats (but that one-time reflection could also be done in existing JNA to optimize Structure). There's also a benefit of JEP 424 able to handle structure bitfields, a capability JNA lacks, but perhaps could inform a new feature.

JEP424 also provides more flexible/re-usable/cache-able memory allocations in a given scope... again, something that could be done with the Memory class, but that does not yet exist.

Are JEP424 methods protected and need runtime flags?

At present I need to use --enable-preview which will go away with JDK21, and --enable-native-access=module.name.here which is probably the restriction you're asking about. After that everything (at least the basic function mapping and memory layout code) just works.

Would it be even feasible to create a package that retains the libffi based invocations, the same backwards compatibility and can use JEP424?

Possibly with an MRJAR implementation, but a lot more complexity for very little gain.

My ideal situation differs from this comment. I just want a central library somewhere with common idioms like the ByReference implementations, and platform mappings similar to jna-platform, and some common util functions like Advapi32Util, and so on. I'm totally happy doing the primary/fallback implementation in my own code and not trying to cram the kitchen sink into a single library. I would just like to contribute the work I'm already doing "upstream" somewhere to share it with others, and this organization seems a good fit for it.

I think hosting a JEP424-only set of mappings in a same-org/separate-project here (option 1) without any libffi, with design-for-modularity and minimum JDK21 compatibility is a reasonable option. If the opinion of the maintainers here is that such an implementation doesn't quite belong here, that's fine, and I'm also happy to create a new org/project/etc. anywhere else.

TLDR: Someone needs to start collecting platform mappings somewhere. The users/contributors of this project have a lot of overlap with the users (now and in a decade when JDK21 is ancient) of JEP424. Let's decide where that somewhere is.

[ExE-Boss]

The JEP where this feature became finalized is JEP‑454 (released in JDK 22).

[boulder-on]

I've been working on a JNA like library that uses JEP424. For simple cases the code I have is very close to a drop in replacement. I have versions of the library that work on each Java version from 16-21.

https://github.com/boulder-on/JPassport/tree/Java_22

The library works in one of two modes:

1. Class writing - will reflect on your interface then write, compile and load a class that uses JEP424 to make native calls
2. Proxy - builds an interface proxy and uses reflection to convert arguments and call JEP424.

Class writing is slower to start up, but much more efficient overall. The Proxy object starts very quickly, but requires so much reflection and so many switch statements that performance is a problem. Ultimately, the Classfile API (JEP 457) that is being added to JDK 22 would be a better solution since it would cut out the compile step.

I'm not sure what the simplest architecture is moving forward for JNA. MRJAR's are designed for this sort of thing, but they do add significant complexity. JEP424 likely has enough new and powerful features to warrant a JNA2. The extra benefit is that since you would already require at least JDK 22 you could make use of a large number of new language features that are currently off limits.