How PoolArena and PoolChunk handles memory

[andsel]

Hi team,
in the path to understand how Netty manages the pool of memory buffers, I tried the following simple experiment:

PooledByteBufAllocator allocator = PooledByteBufAllocator.DEFAULT;

List<ByteBuf> buffers = allocateUptoWithBufferOfSize(124 * MB, 8 * KB, allocator);

List<ByteBuf> smallBuffers = allocateUptoWithBufferOfSize(8 * KB, 512, allocator);

// free small buffers
smallBuffers.forEach(ReferenceCounted::release);

// allocate 1 big 4MB buffer
ByteBuf bigBuffer = allocator.buffer(4 * MB); // KO
// allocateUptoWithBufferOfSize(4 * MB, 8 * KB, allocator); // KO
// allocateUptoWithBufferOfSize(4 * MB, 512, allocator); // OK

/**
 * User allocator to allocate as many ByteBuf of payloadSize up to spaceToFill.
 *
 * @return the list of allocated ByteBuf
 */
private static List<ByteBuf> allocateUptoWithBufferOfSize(int spaceToFill, int payloadSize, PooledByteBufAllocator allocator) {
    int iterationsToFill = spaceToFill / payloadSize;
    List<ByteBuf> buffers = new ArrayList<>(iterationsToFill);
    
    // allocates almost all the memory leaving spaces for little less than 4MB
    for (int i = 0; i < iterationsToFill; i++) {
        buffers.add(allocator.buffer(payloadSize));
    }
    return buffers;
}

The test essentially allocates 124MB of ByteBuf each of 8KB size.
Allocate other 8KB with ByteBuf of 512 byte each.
Free these last 8KB.
Ask to allocate 4MB in a unique ByteBuf but fails with OOM.
Ask to allocate 4MB in slices of 8KB ByteBuf but fails with OOM.
Ask to allocate 4MB in slices of 512 ByteBuf and successed.

Failure trace

Exception in thread "main" java.lang.OutOfMemoryError: Cannot reserve 4194304 bytes of direct buffer memory (allocated: 134217729, limit: 134217728)
        at java.base/java.nio.Bits.reserveMemory(Bits.java:178)
        at java.base/java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:121)
        at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:332)
        at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:701)
        at io.netty.buffer.PoolArena$DirectArena.newChunk(PoolArena.java:676)
        at io.netty.buffer.PoolArena.allocateNormal(PoolArena.java:215)
        at io.netty.buffer.PoolArena.tcacheAllocateNormal(PoolArena.java:197)
        at io.netty.buffer.PoolArena.allocate(PoolArena.java:139)
        at io.netty.buffer.PoolArena.allocate(PoolArena.java:129)
        at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:396)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)
        at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
        at io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:116)

The script is launched with the following JVM settings, limiting direct memory to 128MB and forcing the use of only 1 arena:

XX:MaxDirectMemorySize=128m
-Dio.netty.maxDirectMemory=134217728 // limit the direct memory to 128MB
-Dio.netty.allocator.numDirectArenas=1
-Dio.netty.allocator.numHeapArenas=0 // force 1 arena for all the allocable direct memory

My understanding of how Netty allocates memory is by dividing the available memory in Arenas (with this formula

netty/buffer/src/main/java/io/netty/buffer/PooledByteBufAllocator.java

Lines 114 to 116 in 53fa91f

	(int) Math.min(
	defaultMinNumArena,
	PlatformDependent.maxDirectMemory() / defaultChunkSize / 2 / 3)));

) and associate each arena to an event loop.

Each arena contains multiple chucks, which by default are 4MB wide each. Each chunk is assigned to a length size, so that 8KB ByteBuf-s are borrowed from one chunk while others (for example 2KB) are from a different chunk.

Now I've 2 questions:

• considering the test above, why once freed the small byte byffers (512 byte length), re-establishing a 4MB wide space, the chunk that was allocate for 512 byte block is not returned back to be used for other sizes?
• what happens if I allocate a 5 KB byte buffer? Is it brought from 8KB chunk, creating an internal fragmentation of 3KB?

[laosijikaichele]

considering the test above, why once freed the small byte byffers (512 byte length), re-establishing a 4MB wide space, the chunk that was allocate for 512 byte block is not returned back to be used for other sizes?

When doing smallBuffers.forEach(ReferenceCounted::release);, there will be one PoolSubpage left and not be returned back, please see https://github.com/netty/netty/blob/4.1/buffer/src/main/java/io/netty/buffer/PoolSubpage.java#L139, so it still consume some memory, and it can be reused for 512 byte but not 8 * KB / 4MB. @andsel

[andsel]

@laosijikaichele so what happens if I release other 4Mb from buffers list, can I:

• could I allocate just 4MB in chuck of 2KB ?
• or could allocate just 8MB in chuck of 2KB ?

[laosijikaichele]

I think the first one will success, you can run the test to verify it.

[andsel]

Adding these lines, to free 4MB of buffers that were allocated in chunks of 8KB:

// free 4MB form 8k slice buffers
Collections.reverse(buffers);
buffers.stream().limit((4 * MB) / (8 * KB)).forEach(ReferenceCounted::release);

after the free of the small buffers, doesn't change the effects.
From memory metrics I have used: 134217728 bytes pinned: 125829120 bytes. So the space to allocate a 4MB buffer is available, but the allocator try to allocate over the 128MB limit despite it has 8MB free.

Please could someone provide some insights on how the arenas in allocator works. I'm developing an application that when the pinned memory is close to the max direct limit, it starts to close connections and responding with negative acks. I think the problem of recognizing the direct memory exhaustion is a common problem, for example an HTTP server that's consuming almost all resources starts to return 5xx errors. If I don't do this the server dies for an OOM.