Question about io.netty.maxDirectMemory configuration property

[andsel]

With commit 3d29bcf was introduced the property io.netty.maxDirectMemory to limit the usage of direct memory. As I can understand this limit defines a limit for all Netty's direct memory allocation both pooled and unpooled.
Regarding the case < 0:

netty/common/src/main/java/io/netty/util/internal/PlatformDependent.java

Lines 153 to 156 in dc16c58

	// Here is how the system property is used:
	//
	// * < 0 - Don't use cleaner, and inherit max direct memory from java. In this case the
	// "practical max direct memory" would be 2 * max memory as defined by the JDK.

Given that there is an hard limit imposed by the JVM (MaxDirectMemorySize) on how much direct memory the process can use, how is it possible that is uses twice the max memory as defined by the JDK?

[chrisvest]

The JDK limit only applies to direct ByteBuffers. Other allocation paths to native memory, like JNI and Unsafe, are not accounted in that JDK limit.

[andsel]

Thanks a lot @chrisvest for calrification, I missed that Unsafe allocation happens outside of that limit 👍

[andsel]

I've another doubt. If all allocations in Netty happens with Unsafe, in the case of io.netty.maxDirectMemory == 0 says:

Netty will not enforce max memory, and instead will defer to JDK.

if Netty uses one of the other allocations paths, not the direct buffers one, how the JDK can check that MaxDirectMemorySize is respected?

[andsel]

Regarding the original question, about how the memory is accounted when io.netty.maxDirectMemory < 0, I created a small test: https://gist.github.com/andsel/f22c88e3cf4628dfa61454ea5e28a8ba.

The script is launched with MaxDirectMemorySize=8MBand io.netty.maxDirectMemory=-1. In this case it would mean that we have 8MB for Netty which is allocated using Unsafe and aren't accounted in the MaxDirectMemorySize, plus other 8Mb for NIO direct ByteBuffer.

The test allocate and check the allocated direct memory (using JMX). The test executes these steps:

• allocate 4MB with PooledByteBufAllocator.DEFAULT.directBufferand NIO direct memory spy should report 0MB allocated, but print 4MB
• allocate a small 8KB as NIO direct buffer
• allocate other 2MB as NIO direct buffer, at this point the spy should report ~2MB allocated, but instead is ~6MB
• try to allocate other 2MB as NIO direct buffer. At this point the operation should be feasible because we have 4MB out of 8MB allocated with Unsafe (which shouldn't be accounted as NIO direct memory), other ~2MB out of 8MB in NIO direct space, so allocating other 2MB should successful, but fail with:

java.lang.OutOfMemoryError: Cannot reserve 2097152 bytes of direct buffer memory (allocated: 6299649, limit: 8388608)

Is not clear to me why the direct memory that Netty allocates with Unsafe is accounted under the NIO direct space when io.netty.maxDirectMemory < 0.

[chrisvest]

That "Netty allocates with Unsafe" isn't the whole story. It depends on what kind of access we have available at runtime.

See https://github.com/netty/netty/blob/4.1/buffer/src/main/java/io/netty/buffer/PoolArena.java#L719-L722 and https://github.com/netty/netty/blob/4.1/buffer/src/main/java/io/netty/buffer/UnpooledByteBufAllocator.java#L87-L97 for places where we make decisions about this.

Sounds like in your experiments we were allocating native memory through direct ByteBuffers.

[andsel]

I run the test with the following flags:

-Dio.netty.maxDirectMemory=-1 -Dio.netty.allocator.numDirectArenas=1 -Dio.netty.allocator.numHeapArenas=0

where -Dio.netty.allocator.numDirectArenas=1 -Dio.netty.allocator.numHeapArenas=0 force Netty to use just one direct arenas a no Javaheap.
io.netty.maxDirectMemory=-1 as discussed before, means "use a native heap space different from the one used by NIO direct allocation, equal in size to it (aka MaxDirectMemorySize)".

I run on JVM

openjdk version "17.0.8" 2023-07-18
OpenJDK Runtime Environment Temurin-17.0.8+7 (build 17.0.8+7)
OpenJDK 64-Bit Server VM Temurin-17.0.8+7 (build 17.0.8+7, mixed mode)

and after enabling some logs as defined in the following log4j file:

status = error

appender.console.type = Console
appender.console.name = plain_console
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %m%n


logger.netty.name = io.netty
logger.netty.level = debug

I got this logging

10:37:01.588 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using Log4J2 as the default logging framework
10:37:01.590 [main] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.level: simple
10:37:01.590 [main] DEBUG io.netty.util.ResourceLeakDetector - -Dio.netty.leakDetection.targetRecords: 4
10:37:01.595 [main] DEBUG io.netty.util.internal.PlatformDependent0 - -Dio.netty.noUnsafe: false
10:37:01.595 [main] DEBUG io.netty.util.internal.PlatformDependent0 - Java version: 17
10:37:01.596 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
10:37:01.596 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
10:37:01.596 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.storeFence: available
10:37:01.596 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
10:37:01.596 [main] DEBUG io.netty.util.internal.PlatformDependent0 - direct buffer constructor: unavailable: Reflective setAccessible(true) disabled
10:37:01.597 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: available, true
10:37:01.597 [main] DEBUG io.netty.util.internal.PlatformDependent0 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable: class io.netty.util.internal.PlatformDependent0$7 cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module @6bedbc4d
10:37:01.597 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.DirectByteBuffer.<init>(long, int): unavailable
10:37:01.597 [main] DEBUG io.netty.util.internal.PlatformDependent - sun.misc.Unsafe: available
10:37:01.597 [main] DEBUG io.netty.util.internal.PlatformDependent - maxDirectMemory: 8388608 bytes
10:37:01.598 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.tmpdir: /var/folders/f2/6ln9srr13hsdp3kwfz68w3940000gn/T (java.io.tmpdir)
10:37:01.598 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
10:37:01.598 [main] DEBUG io.netty.util.internal.PlatformDependent - Platform: MacOS
10:37:01.598 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.maxDirectMemory: -1 bytes
10:37:01.598 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.uninitializedArrayAllocationThreshold: -1
10:37:01.599 [main] DEBUG io.netty.util.internal.CleanerJava9 - java.nio.ByteBuffer.cleaner(): available
10:37:01.599 [main] DEBUG io.netty.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
10:37:01.599 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.numHeapArenas: 0
10:37:01.599 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.numDirectArenas: 1
10:37:01.599 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.pageSize: 8192
10:37:01.599 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxOrder: 9
10:37:01.599 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.chunkSize: 4194304
10:37:01.599 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.smallCacheSize: 256
10:37:01.599 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.normalCacheSize: 64
10:37:01.599 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxCachedBufferCapacity: 32768
10:37:01.599 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.cacheTrimInterval: 8192
10:37:01.599 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.cacheTrimIntervalMillis: 0
10:37:01.599 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.useCacheForAllThreads: false
10:37:01.599 [main] DEBUG io.netty.buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxCachedByteBuffersPerChunk: 1023
10:37:01.601 [main] DEBUG io.netty.util.internal.InternalThreadLocalMap - -Dio.netty.threadLocalMap.stringBuilder.initialSize: 1024
10:37:01.601 [main] DEBUG io.netty.util.internal.InternalThreadLocalMap - -Dio.netty.threadLocalMap.stringBuilder.maxSize: 4096
10:37:01.605 [main] DEBUG io.netty.buffer.AbstractByteBuf - -Dio.netty.buffer.checkAccessible: true
10:37:01.605 [main] DEBUG io.netty.buffer.AbstractByteBuf - -Dio.netty.buffer.checkBounds: true
10:37:01.605 [main] DEBUG io.netty.util.ResourceLeakDetectorFactory - Loaded default ResourceLeakDetector: io.netty.util.ResourceLeakDetector@6aa61224
10:37:01.606 [main] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.maxCapacityPerThread: 4096
10:37:01.606 [main] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.ratio: 8
10:37:01.606 [main] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.chunkSize: 32
10:37:01.606 [main] DEBUG io.netty.util.Recycler - -Dio.netty.recycler.blocking: false
10:37:01.608 [main] DEBUG io.netty.util.internal.PlatformDependent - org.jctools-core.MpscChunkedArrayQueue: available

Where the line

10:37:01.597 [main] DEBUG io.netty.util.internal.PlatformDependent - sun.misc.Unsafe: available

indicate that Unsafe is available. So why if Unsafe is available and I would expect 8MB for NIO direct heap and 8MB for Unsafe, Netty allocate just 8MB on the NIO direct?

[chrisvest]

Try stepping through the code in a debugger to see how the decisions are made.

[andsel]

In the PlatformDependent the point that set USE_DIRECT_BUFFER_NO_CLEANER

netty/common/src/main/java/io/netty/util/internal/PlatformDependent.java

Lines 162 to 165 in 5cb4974

	if (maxDirectMemory == 0 || !hasUnsafe() || !PlatformDependent0.hasDirectBufferNoCleanerConstructor()) {
	USE_DIRECT_BUFFER_NO_CLEANER = false;
	DIRECT_MEMORY_COUNTER = null;
	} else {

is conditioned by the false value of PlatformDependent0.hasDirectBufferNoCleanerConstructor() which is linked to the log line

10:37:01.596 [main] DEBUG io.netty.util.internal.PlatformDependent0 - direct buffer constructor: unavailable: Reflective setAccessible(true) disabled

So if I understand correctly this due to the fact that setAccesible is not anymore available (which happens since JDK 9, if I'm not wrong).
Does this means that if Netty is used on JDK 9+ the allocation is done only using NIO direct byte buffers and no Unsafe or JNI?

[chrisvest]

Yeah, it's disabled by default in certain cases. You can enable it with -Dio.netty.tryReflectionSetAccessible=true.