[Native-image] Build a native-image for pulsar-admin

[nodece]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Using https://www.graalvm.org/ to build a native-image for pulsar-admin reduces the running time.

Comparison of running time:

❯ ./native-pulsar-admin.sh
+ echo native-pulsar-admin
native-pulsar-admin
+ ./target/./native-pulsar-admin ../conf/client.conf tenants list
public
pulsar

real    0m0.264s
user    0m0.208s
sys     0m0.049s
+ echo ----------------
----------------
+ echo 'default pulsar-admin'
default pulsar-admin
+ ../bin/pulsar-admin tenants list
2024-03-21T08:27:30.697167Z main WARN The use of package scanning to locate plugins is deprecated and will be removed in a future release
2024-03-21T08:27:30.703654Z main WARN The use of package scanning to locate plugins is deprecated and will be removed in a future release
2024-03-21T08:27:30.729035Z main WARN The use of package scanning to locate plugins is deprecated and will be removed in a future release
2024-03-21T08:27:30.732848Z main WARN The use of package scanning to locate plugins is deprecated and will be removed in a future release
public
pulsar

real    0m2.429s
user    0m4.667s
sys     0m0.318s

This native-pulsar-admin reduces real by 9 times compared to pulsar-admin.sh, this is a benefit.

BTW, Graalvm cannot support cross-compilation, we must use different platforms to build different native-image.

Solution

No response

Alternatives

No response

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[lhotari]

Yes, this could be useful. Also for pulsar-shell (/cc @nicoloboschi). The only concern is about the increased build time and I wouldn't like to increase CI build times for building the native images. How long does it take to build the native image?

[nicoloboschi]

I've tried in the past to build a native binary. The main issue is that all the reflection calls must be intercepted and declared in the configuration file. There's a tracing mechanism in graalvm to auto-generate those configurations, however not all the code paths are triggered and therefore some runtime errors could be generated at runtime (blocking users in making some operations).

That's why I think it's not worth it and instead the Pulsar community should start using Pulsar shell as single CLI tool (and that's why it was born)

My attempts were made more than 1 year ago and things could have changed meanwhile; however I've read the same complaints in other ASF projects

• Pulsar Shell: native image with GraalVM #17211
• Pulsar Shell: native image with GraalVM #20095

[nodece]

How long does it take to build the native image?

Environment:

MacBookPro16,1
6H Intel Core i7
2.6 GHz

Only build for pulsar-admin: [INFO] Total time: 05:04 min

Pulsar admin is a commonly used tool, so I plan to support that. Or we can export the native-image config to the jar, the developers can use the Graalvm to build the native-image tool.

https://github.com/remkop/picocli/tree/main/picocli-codegen can help us to generate the CLI config of native-image, it is simpler, I have already done it, and I can open a PR to preview.

[lhotari]

Only build for pulsar-admin: [INFO] Total time: 05:04 min

Pulsar admin is a commonly used tool, so I plan to support that. Or we can export the native-image config to the jar, the developers can use the Graalvm to build the native-image tool.

It would be worth checking with GitHub Actions CI. Please run experiments in your own fork.
One possibility is to setup the CI in a way that this isn't run for PR builds at all and let the scheduled build handle building the native image & running tests for it for catching possible regressions.

[lhotari]

Have you tested CDS? That would be easy to add.
https://docs.oracle.com/en/java/javase/21/vm/class-data-sharing.html

For example, Spring is leveraging this. Blog post https://spring.io/blog/2023/12/04/cds-with-spring-framework-6-1 .

As mentioned by Brian Goetz in his Project Leyden talk at Devoxx: “Most people don’t use CDS today but probably should since they can get a reasonable startup improvement for relatively little work”.

[nodece]

Let me try CDS.