Documentation update: LRU, newest not oldest expired out in edge cases

[neilstevenson]

Newest entries are deleted not oldest, I presume as expirationTime is set to Long.MAX_VALUE for all

Reproducer, tested on 4.2.2 and 5.0:

    public static void main(String[] args) throws Exception {
    	String mapName = "m";
    	Config config = new Config();
        MapConfig mapConfig = new MapConfig(mapName);
        EvictionConfig evictionConfig = new EvictionConfig();
        evictionConfig.setEvictionPolicy(EvictionPolicy.LRU);
        evictionConfig.setMaxSizePolicy(MaxSizePolicy.USED_HEAP_SIZE);
        evictionConfig.setSize(3);// 3MB
        mapConfig.setEvictionConfig(evictionConfig);
    	config.getMapConfigs().put(mapConfig.getName(), mapConfig);
    	HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(config);
    	IMap<Long, byte[]> iMap = hazelcastInstance.getMap(mapName);
    	System.out.println("Used memory: " + (Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()));
    	byte[] oneMb = new byte[1024 * 1024];
    	for (long l = 0; l < 5; l++) {
    		iMap.put(l, oneMb);
    		TimeUnit.SECONDS.sleep(10L);
    	}
    	// Allow eviction to run
    	TimeUnit.MINUTES.sleep(1L);
    	for (Long key : new TreeSet<>(iMap.keySet())) {
    		EntryView<Long, byte[]> entryView = iMap.getEntryView(key);
    		System.out.println(key + " : creationTime=" + entryView.getCreationTime()
    			+ ",last access=" + entryView.getLastAccessTime()
    			+ ",expirationTime=" + entryView.getExpirationTime());
    	}
    	Hazelcast.shutdownAll();
    }

gives

0 : creationTime=-1,last access=1635439097000,expirationTime=9223372036854775807
1 : creationTime=-1,last access=1635439107000,expirationTime=9223372036854775807

Last access time seems also to have lost millisecond granularity.

Related hazelcast/hazelcast#18614

[mmedenjak]

@ahmetmircik can you take a quick look?

[ahmetmircik]

This is expected by design.

Eviction happens per-partition basis. So when map needs eviction, it sorts entries in a partition to apply LRU and removes most eligible one. In above test case, all entries go in a different partition. All the keys that are being put is the only key in matching partition. So if map decides eviction is required, it removes that only 1 entry from the partition.
To verify eviction works as expected and not removing lately put entries, partition count can be set to 1:

Config config = new Config();
config.setProperty(ClusterProperty.PARTITION_COUNT.getName(),"1");

Regarding time resolution, it is in seconds after 3.11.

[neilstevenson]

From https://docs.hazelcast.com/hazelcast/5.1-snapshot/data-structures/managing-map-memory#understanding-map-eviction we would expect each partition to get a maximum size of 1.

If we insert keys 4,3,2,1,0 these go to partitions 179,227,5,31,11.
Keys 1 and 0 are deleted.

If we reverse the sequence, insert 0,1,2,3,4 these go to partitions 11,31,5,227,179.
Keys 3 and 4 are deleted.

So the observed behaviour is newest inserts are deleted if all partitions are the same size.

[ahmetmircik]

f we insert keys 4,3,2,1,0 these go to partitions 179,227,5,31,11.
Keys 1 and 0 are deleted.
If we reverse the sequence, insert 0,1,2,3,4 these go to partitions 11,31,5,227,179.
Keys 3 and 4 are deleted.

Eviction process only checks the partition of the put operation, there is no coordination between partitions.
In both cases in your example, newest entries are evicted(0,1 in first and 3,4 in second). This is because each partition has 1 entry when latest puts detect memory exceedance. A later put only find itself in partition to evict and deletes it.

[neilstevenson]

Thanks @ahmetmircik

[neilstevenson]

Reopened as placeholder for documentation clarification.

Hazelcast is working correctly, but without understanding of the above explanation, it would seem counter-intuitive