fix(allocated/buffer): release allocated direct memory on close

[romansmirnov]

Description

Release allocated direct memory when closing the dispatcher.

Related issues

closes #8509

Definition of Done

Not all items need to be done depending on the issue and the pull request.

Code changes:

• The changes are backwards compatibility with previous versions
• If it fixes a bug then PRs are created to backport the fix to the last two minor versions. You can trigger a backport by assigning labels (e.g. backport stable/0.25) to the PR, in case that fails you need to create backports manually.

Testing:

• There are unit/integration tests that verify all acceptance criterias of the issue
• New tests are written to ensure backwards compatibility with further versions
• The behavior is tested manually
• The change has been verified by a QA run
• The impact of the changes is verified by a benchmark

Documentation:

• The documentation is updated (e.g. BPMN reference, configuration, examples, get-started guides, etc.)
• New content is added to the release announcement

[Zelldon]

Thanks @romansmirnov good catch ! 🎖️

I know that we removed the cleaner stuff at some point, because it was a bit hard to maintain and should actually be done by the jdk (on gc). At that time agrona didn't support that functionallity afaik. I think agrona uses the same logic we used, which depends on unsafe (another reason why we removed it). The jdk has currently no option to free memory in an easier way. @oleschoenburg do you think this might be an issue for the jdk 17 migration?

If ole has sees no issues than I think it is fine :)

[lenaschoenburg]

As we are already running on JDK 17 it's probably fine. I don't expect any issue due to raising the language level.
I also like that we can just rely on agrona to maintain this 👍

[romansmirnov]

Thanks for your input. It seems that this change causes test failures. I am looking into it.

[romansmirnov]

Hm... now what could happen is the CommandApiRequestHandler tries to write to the Dispatcher which got freed. Meaning, the write buffer might still be in use by other components while the direct memory is freed. In that case, the request handler writes a command to an illegal address which results in exiting the JVM.

The direct byte buffer would need to be freed only when it is sure that nobody tries to write to it.

@Zelldon & @oleschoenburg, do you see any other option?

[romansmirnov]

Closing this PR for now without merging it, because it causes some subsequent issues. We will come back to the issue again.