Hazelcast EntryProcessor

[rene-m-hernandez]

Hello,

I noticed Bucket4j HazlecastEntryProcessor does not implement the Offloadable interface.

The Hazelcast documentation seems to suggest that we might stand to benefit from implementing it. If I am reading documentation correctly, I think it might be possible to expose this feature in backwards compatible manner. WDYT?

[vladimir-bukhtoyarov]

Hello @rene-m-hernandez

In my understanding Offloadable feature is designed for moving long running entry-processors outside partition thread. Bucket4j HazelcastEntryProcessor does not perform long-running computations, so I don't think that any benefit can be achieved, moreover I suppose that latency will became worse because of moving computation to another thread from thread that was ready to perform computation in-place.

Anyway, feel free to make an benchmark. The most easy way to benchmark Offloadable feature without library code modification will be decoration of IMap and wrapping incoming entry processor inside executeOnKey and submitToKey methods

import com.hazelcast.core.Offloadable;
import com.hazelcast.map.EntryProcessor;

import java.util.Map;

public class HazelcastEntryProcessorOffloadableWrapper<K, T, R> implements Offloadable, EntryProcessor<K, T, R> {

    private final EntryProcessor<K, T, R> targetProcessor;

    public HazelcastEntryProcessorOffloadableWrapper(EntryProcessor<K, T, R> targetProcessor) {
        this.targetProcessor = targetProcessor;
    }


    @Override
    public String getExecutorName() {
        return "my-executor";
    }

    @Override
    public R process(Map.Entry<K, T> entry) {
        return targetProcessor.process(entry);
    }

    @Override
    public EntryProcessor<K, T, R> getBackupProcessor() {
        return targetProcessor.getBackupProcessor();
    }
}

import java.util.*;
import java.util.concurrent.CompletionStage;
import java.util.concurrent.TimeUnit;
import java.util.function.BiFunction;
import java.util.function.Function;

public class IMapOffloadableDecorator<K, V> implements IMap<K, V> {

    private final IMap<K, V> targetMap;


    public IMapOffloadableDecorator(IMap<K, V> targetMap) {
        this.targetMap = targetMap;
    }

    @Override
    public <R> CompletionStage<R> submitToKey(K key, EntryProcessor<K, V, R> entryProcessor) {
        return targetMap.submitToKey(key, new HazelcastEntryProcessorOffloadableWrapper<K, V, R>(entryProcessor));
    }

    @Override
    public <R> R executeOnKey(K key, EntryProcessor<K, V, R> entryProcessor) {
        return targetMap.executeOnKey(key, new HazelcastEntryProcessorOffloadableWrapper<K, V, R>(entryProcessor));
    }
    
    ...
    another methods in same style. Also Lombok can be used for fast decorator implementation.
    ...
}

[rene-m-hernandez]

Thank you for the quick response and the decorator suggestion!

I ran some rough benchmarks today and it seemed as though using an offloaded executor helped with the 99th percentile tail latency in my test (the very thing I was trying to improve).

A rough picture of what I'm running: a gRPC service (Envoy RLS) which calls AsyncBucketProxy.tryConsume(1) after some some slight parsing of the gRPC request (and nothing else).

I need to run the tests a few more times to say for certain, but initial results seem promising. Unfortunately, I have not been very scientific in my load testing as I may have changed more than one variable (code change) between tests, so I will try isolate the test differences to only the decorated IMap.

More context: I am currently only running a single m6i.xlarge EC2 instance. Load test aiming for 7k rps. Prior to using decorated IMap, i was seeing 99th percentile of ~100+ ms. With decorated IMap, i was seeing ~50ms 99th percentile.

[vladimir-bukhtoyarov]

Good. If testing show some small impact, it is not hard for me to extend HazelcastProxyManager. It will be done in 7.5 release, which proposed release date is between 3-10 May.

[vladimir-bukhtoyarov]

Also I found one use-case where moving computation to external thread can be theoretically helpful - it is case of bucket with huge contention optimized with Batching Optimization. It is use-case of wide limiters when single limiter is applied for all users instead of limiter per user scenario. Example of usage of batching optimization can be found here

[vladimir-bukhtoyarov]

@rene-m-hernandez ability to specify custom executor has been released with version 7.5.0

see https://github.com/vladimir-bukhtoyarov/bucket4j/blob/master/bucket4j-hazelcast-all/bucket4j-hazelcast/src/main/java/io/github/bucket4j/grid/hazelcast/HazelcastProxyManager.java#L76

[rene-m-hernandez]

Thank you! 👍